Volume 27, Number 3
Using Research to Predict Great Teachers
Uplift Education, a charter network in Dallas, is using predictive research to identify effective teachers like Daniel Polk (above) more easily.
What if you could spot a top teacher candidate from an e-mail?
It may sound too good to be true, but statistically speaking, it can be surprisingly effective, according to one charter school organization requiring applicants to answer hypothetical e-mails as part of its interviewing process beginning this summer.
The e-mail exercise is just part of a larger effort by Uplift Education
of Dallas, Texas to use predictive research to help them identify teachers who are most likely to be effective with students and remain with the organization.
Predictive research is not new; it’s how Amazon or iTunes knows what products to recommend to you. But it’s attracting more interest in education circles. The Gates Foundation recently announced a $1 million
grant to use predictive analytics to identify variables in student school success. Now, Uplift Education is using predictive research to tackle one of the most vexing riddles facing school leaders everywhere: how do you hire a good—even great—teacher when faced with mountains of rèsumès?
Given its plans to expand from 17 to 37 schools in five years, Uplift CEO Yasmin Bhatia says her organization needed a better way to “increase our hit rate of having a great teacher.” For any school, making a bad hire is costly. Even though charter schools can fire an ineffective teacher mid-year, precious time is lost for students.
“When you are trying to close the achievement gap, you have to get the teacher right from the start,” says Bhatia, noting that this past year they hired 107 teachers for which they received 5,472 applications. So, more than a year ago, Uplift began working with consultants from the 3D Group of Emeryville, Calif. to develop and validate new hiring tools based on the characteristics of successful Uplift teachers.
The result is a multi-step interview that scores candidates on how they align with 29 key characteristics—a combination of knowledge, skills, abilities, and other qualities— including the ability to differentiate instruction, do detailed planning, and create remediation strategies to close achievement gaps.
The new hiring approach shifts the onus of sorting resumes from principals to a human resources department. At each step along the way, candidates are asked scripted questions with follow-up probes—all scored on a five-point scale according to specific rubrics. Most important, the questions themselves are not general in nature, but rather require candidates to cite specific instances when, for example, they had to gain student trust, manage deadlines, or cope with difficult feedback. Only candidates who score well on an initial phone interview get to teach a model lesson and interview with the school’s principal and other Uplift leaders.
While the process this year included the phone screening, model lesson, and the interview with school leaders, two more elements will be part of the upcoming hiring cycle. These include the e-mail exercise and a task requiring candidates to review student data and write an analysis.
In the e-mail exercise, applicants are asked to respond to an e-mail from a fictional parent who is upset with her son’s biweekly progress reports, which show he is doing poorly in reading. “Please let me know what thoughts you have on what my son can do to turn things around,” says a sample prompt released by the 3D Group. The real prompt given to candidates might be even more “accusatory” says Andrew English, a senior consultant for 3D.
English says he was impressed by how much the e-mail exercise revealed. “It taps into the ability of the teacher to deal with an upset parent in a way that shows sensitivity and the ability to admit mistakes and not be defensive,” says English. The task also calls on “the ability to reflect on the situation and to see the bigger picture,” he says.
To be certain that the new hiring tools, including phrasing of questions and scoring, actually worked, 59 top Uplift teachers went through the process to validate the tools. As researchers hoped, they scored well on the measures comparing their responses to items calibrated to reflect those 29 key characteristics identified as critical to success at Uplift. The result, says English, is a process that is statistically much more reliable than unstructured interviews in figuring out which candidates will be the most effective teachers.
“Literally, we are predicting teacher performance,” he says. Traditionally, across industries, “individuals believe they can sit down and figure out if someone is good based on an unstructured interview—and we know from research that’s not true,” English says.
Bhatia, the Uplift CEO, says that as they began using the tools this year, it has required a “cultural mind shift” to move away from expecting school leaders to spot teaching talent to relying on a more studied and statistical method. Jacqueline Lovelace, director of new schools at Uplift, who helps judge model lessons and score final interviews, says the tools solve a key problem: some people are good talkers, but lack actual skills.
Lovelace says they used to ask fairly general questions such as, “What is your philosophy of teaching?” Unfortunately, she says, “You could go onto the Internet and pull standard interview questions for teachers. What we found is we had a lot of people who could fake through the interview and be very general about what they would do, but when it came to performing—they didn’t.”
Another bonus, says Lovelace, a 16-year veteran of the Dallas Independent School District before joining Uplift six years ago, is that those they hire now “know what they are getting into.” That means a better match with the organization’s values.
“You have to get the right people in front of the scholars,” she says, noting that if you expect students to work together and mediate conflicts, for example, you can’t have teachers arguing in the hallway. Better interviewing tools can flag problems before someone is hired, she says.
While the interview process may sound intimidating, Kascie Smith, recently offered a middle school art teacher position at Uplift’s Peak Preparatory, really liked it. “I didn’t feel like they were just trying to fill a position,” he says. “It felt like they were trying to get a good fit.”
That experience was much different than last August when he was hired as an eighth grade language arts teacher at a South Dallas middle school. During a 30-minute interview, the principal asked if he had ever taught in an urban setting. Smith, who had taught at private schools, had not, but had been a police officer and coach. The principal urged that, “you have to motivate these kids” and told him to “go through HR and make sure all the paperwork is in.” Smith left feeling that, “they just needed a teacher and I needed a job.”
For Uplift, Smith, who majored in art at Baylor University, prepared and taught a 40-minute class on pointillism. Detailed interview questions hit on classroom management and deadlines, and he says, “gave me the check mark that this place is top of their game. You know their expectations instead of trying to figure them out as the year goes along.”
There may be no bigger conversation in education reform at the moment than teacher quality. While there are pitched battles over teacher evaluation, value-added measures, tenure rules, and LIFO, some education leaders are coming to the “aha” conclusion that it’s critical to hire smarter in the first place.
Plus, says Bhatia, better hires elevate the whole school culture. “The buzz around our organization now is, ‘I wonder if I could get hired?’’ says Bhatia. “It has sent a signal that when we talk about rigor, we are translating that all the way down to the hiring process.”
Laura Pappano is an education journalist based in New Haven, Conn. She is the author of Inside School Turnarounds: Urgent Hopes, Unfolding Stories (Harvard Education Press, 2010).