A few years ago, Milbrey McLaughlin, professor emeritus of Stanford University’s School of Education, presided over a briefing with a group of concerned youth services workers in northern California. The group had convened to discuss the results of a study exploring the relationship between court dependency and school performance. Emotions in the room rose as the evidence was presented. Court-dependent youth were falling behind their nondependent peers in category after category. Children in foster care received lower standardized test scores than nondependent youth. They had higher rates of absence, of mobility, of grade retention. They were much more likely to be suspended or expelled.
The study also yielded more nuanced results. It showed that court-dependent children fared worst in school during the first six months of being in the system, when they were making the transition to foster care. Once they had adjusted, they seemed to do better. But if their stay in foster care went on for more than two years, performance declined again.
Depressing as the analysis was, the caseworkers were empowered by data that confirmed what they had known anecdotally and intuitively for years, McLaughlin says. A county commission was ultimately formed to explore ways to intervene to help this vulnerable population succeed in school, especially during the critical first six months of dependency. But the analysis itself could never have happened without the use of the Youth Data Archive, an innovative database that links records from several government agencies and community-based organizations.
Developed by researchers at Stanford’s John W. Gardner Center for Youth and Their Communities (JGC), the archive allows researchers to explore youth issues across many different agencies in ways that were previously next to impossible because of bureaucratic boundaries and logistical impediments. “The schools didn’t know these students had a history,” says McLaughlin. “The child welfare people didn’t know there was such a huge disproportion in poor outcome and suspension. Each agency simply handed off these kids to another. It was a blank slate.”
It can be a monumental task to build such a data warehouse. Most schools already track student information like test scores and grades. Similarly, such public agencies as health and human services, the juvenile justice system, and child protective services maintain their own databases. The data exists; it just hasn’t been put together. “The challenges are no longer technical. They are political,’’ says McLaughlin, who is also the director of JGC. Such efforts require convincing folks from agencies to collaborate with each other and to share sensitive data. Trust must be built. Relationships must be forged (see sidebar “How to Build a Better Database”). Close SidebarHow to Build a Better Database
The single most important factor in building an interagency data archive isn’t technology. It’s trust.
Researchers at Stanford University discovered right out of the gate that it was no easy task to convince government agencies to share their data. The process took time. Researchers found that they had to work with multiple players within each agency to gain permission. “If we got the buy-in of the superintendent, we might find that the gatekeepers of the data were [still] worried that if they gave away such sensitive data, they might get in trouble,’’ says Kara Dukakis, associate director of the John W. Gardner Center for Youth and Their Communities. Researchers had to convince many players at varying levels of authority that sharing the data was a win-win situation: research projects had the potential to yield results that could save money by eliminating the duplication of services and, more important, provide information on how to foster student success. “It involved being able to articulate why we need that data and what they are going to get out of it,” says Dukakis.
They have also forged a unique relationship with their partners. Members of the various agencies, not the researchers, determine the issues to study. The goal is to find actionable answers to real problems. In order to ensure trust, the team agrees not to publish the results without the explicit agreement of the agencies involved. That presents a unique challenge for researchers, as the goal is to remain neutral and to serve the community, not to drive scholarship. “We have to be careful, because they might feel they are going to get dinged if this information gets out,” says Dukakis. “That’s not our intention.”
That “no publish” arrangement presents a dilemma for some researchers, though. “The trading [off] of the possibility for publishing negative findings could be a big loss,’’ says Jeffrey Henig, professor of political science and education at Columbia University’s Teachers College.
Despite the obstacles, interagency data warehouses are becoming more common. The pioneering Kids Integrated Data System (KIDS) was launched in 2002 as a joint effort of the City of Philadelphia, the School District of Philadelphia, and the University of Pennsylvania. Archives like these have been used to influence policy and practices on issues ranging from kindergarten readiness to homelessness and post-?secondary success (see sidebar “Research Findings”). Close SidebarResearch Findings
Research findings using integrated databases in Philadelphia and northern California have been used to influence policy and practices on issues ranging from school readiness to postsecondary success. Here are some examples:
• A Philadelphia study underscored the relationship between a good, solid preschool experience and success in kindergarten and early elementary school. Researchers developed an early care and intervention interview, and now kindergarten teachers in Philadelphia meet with parents to gather data before the children begin school, enabling the teachers to have a touchstone at this critical time.
• Children who are in homeless shelters are at greater risk for out-of-home placement later down the line. As a result, the Philadelphia Department of Human Services put protective workers in shelters to ensure that these children get necessary services to prevent future negative outcomes.
• Students who participated in family engagement programs and extended learning opportunities at community schools in a northern California city reported feeling more cared for at school and had higher gains in math scores than nonparticipants.
• High school graduates who took a core English class in their first year in a local community college had a higher chance of finishing their degree. As a result, the community college set aside spots in core English classes for a group of students and agreed to follow those students over the years.
The study also yielded more nuanced results. It showed that court-dependent children fared worst in school during the first six months of being in the system, when they were making the transition to foster care. Once they had adjusted, they seemed to do better. But if their stay in foster care went on for more than two years, performance declined again.
Depressing as the analysis was, the caseworkers were empowered by data that confirmed what they had known anecdotally and intuitively for years, McLaughlin says. A county commission was ultimately formed to explore ways to intervene to help this vulnerable population succeed in school, especially during the critical first six months of dependency. But the analysis itself could never have happened without the use of the Youth Data Archive, an innovative database that links records from several government agencies and community-based organizations.
Developed by researchers at Stanford’s John W. Gardner Center for Youth and Their Communities (JGC), the archive allows researchers to explore youth issues across many different agencies in ways that were previously next to impossible because of bureaucratic boundaries and logistical impediments. “The schools didn’t know these students had a history,” says McLaughlin. “The child welfare people didn’t know there was such a huge disproportion in poor outcome and suspension. Each agency simply handed off these kids to another. It was a blank slate.”
It can be a monumental task to build such a data warehouse. Most schools already track student information like test scores and grades. Similarly, such public agencies as health and human services, the juvenile justice system, and child protective services maintain their own databases. The data exists; it just hasn’t been put together. “The challenges are no longer technical. They are political,’’ says McLaughlin, who is also the director of JGC. Such efforts require convincing folks from agencies to collaborate with each other and to share sensitive data. Trust must be built. Relationships must be forged (see sidebar “How to Build a Better Database”). Close SidebarHow to Build a Better Database
The single most important factor in building an interagency data archive isn’t technology. It’s trust.
Researchers at Stanford University discovered right out of the gate that it was no easy task to convince government agencies to share their data. The process took time. Researchers found that they had to work with multiple players within each agency to gain permission. “If we got the buy-in of the superintendent, we might find that the gatekeepers of the data were [still] worried that if they gave away such sensitive data, they might get in trouble,’’ says Kara Dukakis, associate director of the John W. Gardner Center for Youth and Their Communities. Researchers had to convince many players at varying levels of authority that sharing the data was a win-win situation: research projects had the potential to yield results that could save money by eliminating the duplication of services and, more important, provide information on how to foster student success. “It involved being able to articulate why we need that data and what they are going to get out of it,” says Dukakis.
They have also forged a unique relationship with their partners. Members of the various agencies, not the researchers, determine the issues to study. The goal is to find actionable answers to real problems. In order to ensure trust, the team agrees not to publish the results without the explicit agreement of the agencies involved. That presents a unique challenge for researchers, as the goal is to remain neutral and to serve the community, not to drive scholarship. “We have to be careful, because they might feel they are going to get dinged if this information gets out,” says Dukakis. “That’s not our intention.”
That “no publish” arrangement presents a dilemma for some researchers, though. “The trading [off] of the possibility for publishing negative findings could be a big loss,’’ says Jeffrey Henig, professor of political science and education at Columbia University’s Teachers College.
Despite the obstacles, interagency data warehouses are becoming more common. The pioneering Kids Integrated Data System (KIDS) was launched in 2002 as a joint effort of the City of Philadelphia, the School District of Philadelphia, and the University of Pennsylvania. Archives like these have been used to influence policy and practices on issues ranging from kindergarten readiness to homelessness and post-?secondary success (see sidebar “Research Findings”). Close SidebarResearch Findings
Research findings using integrated databases in Philadelphia and northern California have been used to influence policy and practices on issues ranging from school readiness to postsecondary success. Here are some examples:
• A Philadelphia study underscored the relationship between a good, solid preschool experience and success in kindergarten and early elementary school. Researchers developed an early care and intervention interview, and now kindergarten teachers in Philadelphia meet with parents to gather data before the children begin school, enabling the teachers to have a touchstone at this critical time.
• Children who are in homeless shelters are at greater risk for out-of-home placement later down the line. As a result, the Philadelphia Department of Human Services put protective workers in shelters to ensure that these children get necessary services to prevent future negative outcomes.
• Students who participated in family engagement programs and extended learning opportunities at community schools in a northern California city reported feeling more cared for at school and had higher gains in math scores than nonparticipants.
• High school graduates who took a core English class in their first year in a local community college had a higher chance of finishing their degree. As a result, the community college set aside spots in core English classes for a group of students and agreed to follow those students over the years.
A New Approach to Reform?
The possibilities for research are as deep as the data. What would happen if researchers could easily take a look at a school’s population to learn what agencies have served its students over the years? What if they could track patterns in students’ lives going back as early as prenatal care? What, then, if they compared these students with another cohort of similar students in a school with a better success record? They could look at interventions and programs that worked. They could examine programs that didn’t work. They could develop policies and practices to help these students succeed. With the increased sophistication of interagency databases, it is now easier to chart the relationship between out-of-school factors and in-school performance. Some policy makers see this as a forerunner of a new approach to education reform. They foresee a middle ground in the decades-old debate over the schools-only, accountability-?based efforts at education reform versus the notion that the nation must solve the poverty problem before tackling its troubled school system.
“We love these bipolar debates in education,’’ says Paul Reville, Massachusetts’ secretary of education. “Some say it’s all academics. Some say it’s all demographics. The reality isn’t in the [extremes]. It’s in the middle somewhere, and it’s going to be through some combination of urgent, dramatic improvement in quality of learning in the classrooms coupled with attention to the factors that prevent students from taking advantage of improved instruction.
“We are asking schools—six hours a day, 180 days a year—to close achievement gaps that are caused by factors that begin before birth and persist,” Reville says. “We need to develop systems of support that are similar to what an affluent family would [provide] to support their children for higher levels of achievement.”
In Massachusetts, however, this effort is still in the conceptual stage. In 2008, the governor established the Child and Youth Readiness Cabinet, jointly chaired by the state secretaries of education and of health and human services, to coordinate services among various agencies to collectively address the needs of the whole child. The cabinet, Reville says, has discussed pooling data to create a sort of medical record for each individual student, charting academic progress as well as interventions and support provided outside of school. The idea, he says, hasn’t “gotten very far,” given the vagaries of collaboration among various agencies, which can cling to their territory and authority and be resistant to change.
“It’s not neat, and it’s not comfortable,’’ says Reville. “It’s very difficult, but I think it’s essential.”
Legal barriers can be especially problematic. McLaughlin recalls one research project that involved outside mental health professionals working in schools. By law, these public employees were not allowed to tell the school what they had learned about the students. “It’s an institutional train wreck,’’ says McLaughlin. “No one can talk across boundaries.”
Shifting Ground
Despite the challenges, integrated data systems are drawing increasing interest from researchers. Two years ago, John Fantuzzo and his colleagues at the University of Pennsylvania launched Intelligence for Social Policy, an effort funded by the John D. and Catherine T. MacArthur Foundation that aims to proliferate the spread of integrated data systems to improve public policy and to create a professional network of such systems. The professional network currently represents nine communities.
There are other signs that change is in the wind. In April 2011, the U.S. Department of Education hired Kathleen Styles, its first chief privacy officer, in an effort to balance the need for student privacy with the need for data availability. At the same time, it released newly proposed regulations to the Family Educational Rights and Privacy Act. The new regulations aim to ease access to student data for legitimate research purposes while maintaining appropriate privacy for individuals. “The guidelines acknowledge that the data warehouse is a legitimate way to store educational records and to evaluate educational programs and their impact across multiple domains beyond education,’’ says Dennis Culhane, a KIDS founder and professor at the University of Pennsylvania’s School of Social Policy and Practice.
Nonetheless, privacy issues remain a concern. Some researchers envision creating an individual ID for each child, similar to a medical health record, that follows them as they proceed through school. “Many would say that is government intrusion,’’ says Jeffrey Henig, professor of political science and education at Columbia University’s Teachers College.
Henig and others are increasingly convinced that education reform cannot succeed unless out-of-school factors are taken into account, and integrated databases can be a valuable tool toward that end. At the same time, they contend that deeply entrenched ideological views against the public sector must shift before any change in a schools-only approach to reform can occur. “There is a prejudice against the public sector . . . and if your bias is that anything the public sector does is corrupt and incompetent, you blame the schools for any problem you see,” says Richard Rothstein, a research associate of the Economic Policy Institute, who has long contended that schools hold responsibility for only one-third of the achievement gap.
But for researchers like Fantuzzo, who has been working on integrated data systems since as early as 1998, a shift to the middle ground is urgent. “In large urban settings, you find mayors vexed by youth problems,’’ he says. “They care about dropouts and truancy and juvenile justice. They are worried about big kids with big problems. But it doesn’t take much rocket science to understand that the roots are the genesis of the fruits. You don’t get fruit from the tree if you don’t pay attention to the roots. And you won’t find kids who take root in education if we ignore real, substantial, contextual problems that go beyond the walls of the school.’’
Patti Hartigan, a former staff reporter for the Boston Globe, is a freelance journalist in Massachusetts.
The possibilities for research are as deep as the data. What would happen if researchers could easily take a look at a school’s population to learn what agencies have served its students over the years? What if they could track patterns in students’ lives going back as early as prenatal care? What, then, if they compared these students with another cohort of similar students in a school with a better success record? They could look at interventions and programs that worked. They could examine programs that didn’t work. They could develop policies and practices to help these students succeed. With the increased sophistication of interagency databases, it is now easier to chart the relationship between out-of-school factors and in-school performance. Some policy makers see this as a forerunner of a new approach to education reform. They foresee a middle ground in the decades-old debate over the schools-only, accountability-?based efforts at education reform versus the notion that the nation must solve the poverty problem before tackling its troubled school system.
“We love these bipolar debates in education,’’ says Paul Reville, Massachusetts’ secretary of education. “Some say it’s all academics. Some say it’s all demographics. The reality isn’t in the [extremes]. It’s in the middle somewhere, and it’s going to be through some combination of urgent, dramatic improvement in quality of learning in the classrooms coupled with attention to the factors that prevent students from taking advantage of improved instruction.
“We are asking schools—six hours a day, 180 days a year—to close achievement gaps that are caused by factors that begin before birth and persist,” Reville says. “We need to develop systems of support that are similar to what an affluent family would [provide] to support their children for higher levels of achievement.”
In Massachusetts, however, this effort is still in the conceptual stage. In 2008, the governor established the Child and Youth Readiness Cabinet, jointly chaired by the state secretaries of education and of health and human services, to coordinate services among various agencies to collectively address the needs of the whole child. The cabinet, Reville says, has discussed pooling data to create a sort of medical record for each individual student, charting academic progress as well as interventions and support provided outside of school. The idea, he says, hasn’t “gotten very far,” given the vagaries of collaboration among various agencies, which can cling to their territory and authority and be resistant to change.
“It’s not neat, and it’s not comfortable,’’ says Reville. “It’s very difficult, but I think it’s essential.”
Legal barriers can be especially problematic. McLaughlin recalls one research project that involved outside mental health professionals working in schools. By law, these public employees were not allowed to tell the school what they had learned about the students. “It’s an institutional train wreck,’’ says McLaughlin. “No one can talk across boundaries.”
Shifting Ground
Despite the challenges, integrated data systems are drawing increasing interest from researchers. Two years ago, John Fantuzzo and his colleagues at the University of Pennsylvania launched Intelligence for Social Policy, an effort funded by the John D. and Catherine T. MacArthur Foundation that aims to proliferate the spread of integrated data systems to improve public policy and to create a professional network of such systems. The professional network currently represents nine communities.
There are other signs that change is in the wind. In April 2011, the U.S. Department of Education hired Kathleen Styles, its first chief privacy officer, in an effort to balance the need for student privacy with the need for data availability. At the same time, it released newly proposed regulations to the Family Educational Rights and Privacy Act. The new regulations aim to ease access to student data for legitimate research purposes while maintaining appropriate privacy for individuals. “The guidelines acknowledge that the data warehouse is a legitimate way to store educational records and to evaluate educational programs and their impact across multiple domains beyond education,’’ says Dennis Culhane, a KIDS founder and professor at the University of Pennsylvania’s School of Social Policy and Practice.
Nonetheless, privacy issues remain a concern. Some researchers envision creating an individual ID for each child, similar to a medical health record, that follows them as they proceed through school. “Many would say that is government intrusion,’’ says Jeffrey Henig, professor of political science and education at Columbia University’s Teachers College.
Henig and others are increasingly convinced that education reform cannot succeed unless out-of-school factors are taken into account, and integrated databases can be a valuable tool toward that end. At the same time, they contend that deeply entrenched ideological views against the public sector must shift before any change in a schools-only approach to reform can occur. “There is a prejudice against the public sector . . . and if your bias is that anything the public sector does is corrupt and incompetent, you blame the schools for any problem you see,” says Richard Rothstein, a research associate of the Economic Policy Institute, who has long contended that schools hold responsibility for only one-third of the achievement gap.
But for researchers like Fantuzzo, who has been working on integrated data systems since as early as 1998, a shift to the middle ground is urgent. “In large urban settings, you find mayors vexed by youth problems,’’ he says. “They care about dropouts and truancy and juvenile justice. They are worried about big kids with big problems. But it doesn’t take much rocket science to understand that the roots are the genesis of the fruits. You don’t get fruit from the tree if you don’t pay attention to the roots. And you won’t find kids who take root in education if we ignore real, substantial, contextual problems that go beyond the walls of the school.’’
Patti Hartigan, a former staff reporter for the Boston Globe, is a freelance journalist in Massachusetts.
