The most common open-ended response on the High School Survey of Student Engagement is perhaps the most revealing.
“Many kids say, ‘Why are we taking this survey? No one will listen to us,’” says Ethan Yazzie-Mintz, survey director at the Center for Evaluation and Policy at Indiana University, whose 2008 High School Survey of Student Engagement reached 68,000 students in 29 states. Although it may sound like a throwaway line, says Yazzie-Mintz, their comment nails the problem precisely: students don’t feel heard.
While principals, teachers, and education policymakers are constantly parsing data on student achievement, there is growing concern that data on student engagement—and other aspects of students’ experience—are missing. A significant body of evidence links student engagement to graduation rates and academic success. But while educators may think they are involving students intellectually, socially, and emotionally, students often see it differently.
Large-scale student surveys are fast becoming valuable tools for educators. Access to student perceptions, proponents say, offers better information on everything from how welcoming a school is to how well students understand the nuts and bolts of the college application process.
Bored Every Day
What do students think about school? The picture from national surveys is not encouraging.
Yazzie-Mintz says data and his experience working with high schools suggest that about 10 percent of high school students are highly engaged and 15 percent are disengaged. Those in between lack strong connections with teachers and find school work irrelevant or dull. Results from the center’s 2007 survey show that two-thirds of students are bored in class every day, and 17 percent are bored in every class.
The latest My Voice survey designed by the Quaglia Institute for Student Aspirations in Portland, Maine, echoes this sense of disaffection. Administered between the fall of 2006 and the spring of 2008 to 414,243 students in grades 6 through 12 in 32 states, the survey found that just 45 percent of students believe teachers care if they are absent from school.
“It would be wrong to say that no schools have paid attention to what kids have been experiencing,” says William Damon, whose book The Path to Purpose: Helping Our Children Find Their Calling in Life describes a generation of youth adrift. But, says Damon, director of the Stanford University Center on Adolescence, “too many schools have neglected the motives and feelings and experiences of the students and have relied on that old behaviorist model that you stimulate them and expect a response.”
Jeremy D. Finn, professor of education at the University at Buffalo SUNY and an expert on dropouts and student engagement, says surveys can tell schools how well they are connecting with students (or not), but the critical piece is that schools take the students’ responses seriously.
In a key 1989 paper, Finn described students’ decisions to drop out of school not as a single momentous action but as the culmination of a long path of disengagement from school. Schools must stop looking at what kids are doing—the risk factors or attributes that make them likely to disengage—and look at what the school is doing, he says. Does your school make kids feel anonymous? Impose excessively strict discipline? Teach courses that feel irrelevant to students? Fail to support students academically or socially in the classroom? “Surveys,” he says, “are ways to find out all of these things.”
A Range of Instruments
A growing array of surveys is available to give students a voice in their education—and educators insight into students’ worlds. These range from homegrown school or district questionnaires to large-scale instruments like the High School Survey of Student Engagement (HSSSE). In 2008, 135 high schools paid to have the HSSSE administered to their students. The 35-question survey may be given once a year or, in some districts, every two or more years. The survey asks concrete questions, such as how many papers students write and how much time they spend on homework, as well as posing more philosophical queries, such as, “Why do you go to school?”
The Tell Them From Me survey designed by The Learning Bar, a Canadian company that has gathered data on 100,000 students in the past three years, can give a school principal continuous feedback—even week to week—on the school climate and other indicators, such as students’ sense of belonging and interest in classroom learning over time. (Schools can also opt for a twice-a-year “snapshot.”) As with the HSSSE, school officials can compare answers with national responses and with schools that have similar characteristics.
The My Voice survey designed by the Quaglia Institute builds its questions around eight themes the institute believes are critical to student engagement and success, with seven questions on each topic. The themes are “belonging,” “heroes,” “sense of accomplishment,” “fun and excitement,” “curiosity and creativity,” “spirit of adventure,” “leadership and responsibility,” and “confidence to take action.” The 15-minute survey can be paired with a staff survey to uncover what institute director and founder Russell J. Quaglia describes as a gulf between teachers’ and students’ perceptions. For example, he says, national survey results show that while 96 percent of teachers say they are excited to work with students, only 56 percent of students believe teachers enjoy working with them. “We are not communicating with kids very well,” says Quaglia. “It’s almost as if we are in two different worlds.”
Surprised by the Answers
As principal of the Portsmouth Middle School in New Hampshire for the past 27 years, John M. Stokel believes that if kids don’t feel cared for and connected, they won’t perform well scholastically. A few years ago, he partnered with the nonprofit Quaglia Institute and had students complete the My Voice survey. He also created a faculty committee on student aspirations, which conducted student focus groups to supplement the survey’s findings.
The results jumped out at him. Just 57 percent of students said they felt teachers cared about their problems and feelings. Focus groups helped clarify the matter. Even though teachers thought they had showed interest by speaking and relating to students’ parents, kids didn’t interpret that as concern for them. “The teachers feel like, ‘I am talking all the time, I’m talking to the parents,’” says Stokel. “But the kid perceives it in a different way.”
This and other results spurred Stokel to create a 30-minute advisory program on Monday mornings. Last year, the school’s results on the My Voice survey in the “belonging” category showed big improvement in student perceptions, including an 11-point bump (from 63 percent to 74 percent) in positive responses to the statement, “School is a welcoming and friendly place.”
However, Stokel says the survey also revealed other concerns that are not so easily addressed, including students’ negative feelings about the school’s offering both standard and advanced math and English beginning in seventh grade. “That seems to hit the kids. They look at it as [if] they are being sorted out after sixth grade,” says Stokel. Yet, he says, homogenous versus heterogeneous grouping is “such a political issue” that it will take time to craft a solution.
If You Ask, Listen
Surveys, in other words, don’t just provide information you want. Sometimes they reveal things that are difficult to hear—or fix. Yet if you ask students for their opinions, survey designers say, there is an obligation to act on them.
“We say, ‘If you are not going to do anything about it, you may as well not do the survey,’” says J. Douglas Willms, Canada Research Chair in Human Development at the University of New Brunswick and president of The Learning Bar, whose Tell Them From Me survey will be available in the United States this year. Willms says school leaders lose credibility if they don’t respond—even if it’s just doing the easy things, like fixing broken bolts on bathroom stall doors.
One school in Saskatchewan, he says, reports survey results to students on a large video screen at the front of the school. The format they use (“We asked you . . . ,” “You said . . . ,” and “We’re doing this . . .”) lets students see that their input has an effect on their school environment, he says.
Educators must often stifle the urge to reject negative feedback. “They say, ‘The kids are making this up,’” reports Yazzie-Mintz. “That is something to examine. If you are going to give students a perceptions survey, you have to start with, ‘I’m going to believe what they say.’”
Yazzie-Mintz recalls one high school in an affluent suburb in the Northeast that had made a major investment in technology. When students reported being bored in class, he says, the principal responded, “The students are lying on this thing. They are just trying to make us look bad.” Yet when Yazzie-Mintz looked at students’ open-ended comments, he saw that they perceived the new technology not as improving teaching and learning but as image enhancement. “A lot of the responses were, ‘This school only cares about its image and only does things to look good to outsiders,’” he says. “The school has a lot of work to do.”
Despite concerns that students don’t take surveys seriously, survey designers say that is rarely a problem. Because students don’t disclose their identities, they are free to be frank. The results, notes one survey creator, “are like an anonymous note going under the principal’s door.”
Interpreting the Data
Survey results can be overwhelming. What answers merit a response? Should results be vetted by a data person in every school or district? Or should they be accessible to everyone?
The right answer varies by school, survey—and what you are trying to discover. Some surveys yield results that can be easily benchmarked (e.g., what percentage of students are involved in clubs or sports compared with other schools). And even if only a few people in a district have the skills to analyze data fully, others can still scan the results and act on them, perhaps by making lessons more interactive if the survey shows that students are bored in class. Yazzie-Mintz advises that administrators focus on one or two areas of concern so the results don’t feel like yet another set of charts.
As principal of the Langstaff Secondary School in Canada’s York District, Peter Milovanovic surveyed students at regular intervals using the Tell Them From Me survey, but admits being confused by the data at first. Initially he gravitated to open-ended responses because they were easiest to grasp. Once he became familiar with the data, he says, it became more obvious how to use it. For example, survey results revealed that students were grappling with dramatically higher rates of depression and anxiety than he or school counselors had thought. This spurred him to expand counseling and outreach throughout the school.
Milovanovic says the survey results also gave him important ammunition for solving some long-standing problems in the cafeteria. He brought student feedback to the food service company, which responded by adding more vegetarian meals, adding lower-priced lunch choices, and even replacing three staff members and a manager who clashed with students.
Other results may require additional data or focus groups to decipher. At the Chesterfield County Public Schools in Virginia, all sophomores and seniors at the 10 high schools in the county complete the HSSSE every two years. School improvement manager Glen Miller notes that administrators use the results in conjunction with other information—and common sense.
For example, when results showed students were writing fewer papers than their peers in other schools, Miller says, they realized after some discussion that the survey was given before reaching the writing-intense portion of the English curriculum. “We had to look at the data and see what really applied to us,” he says.
At the Helen Tyson Middle School in Springdale, Ark., school personnel are focused on helping more students get to college. Each year, they question a sample of 100 students using a survey designed by the PALMS (Postsecondary Access for Latino Middle-Grades Students) program. In 2006, principal Todd Loftin says, 94 out of 100 students in his Hispanic-majority school said they planned to attend college, but only half said they knew how to prepare, including which courses to take in later grades.
These results told Loftin that students needed to grasp not just the goal of getting to college but strategies for achieving it. He brought in counselors from two high schools and several colleges, including the University of Arkansas, to give talks about topics like Advanced Placement courses and financial aid. Some of the talks were given for parents, in Spanish.
“In schools, we adults are used to making all the decisions,” says Yazzie-Mintz. Student surveys, he says, require “an ideological shift”—the recognition “that kids are smart, kids have insight, kids can actually help us to do this work [that] we need to do better.”
Laura Pappano writes about education and is writer-in-residence at the Wellesley Centers for Women at Wellesley College.
