Washington Post Staff Writer - Monday, July 21, 2008; 6:11 AM
And yet, because the career academy research by the New York-based MDRC (formerly known as the Manpower Demonstration Research Corp.) was so detailed and professional, we have just learned that the academies accomplished something perhaps even better than higher passing rates on reading exams. They produced young men who got better-paying jobs, were more likely to live independently with children and a spouse or partner and were more likely to be married and have custody of their children.
This is a remarkable finding. It has the power not only to revitalize vocational education but to shift the emphasis of school assessment toward long-range effects on students' lives, not just on how well they did in school and college.
The MDRC, long considered a leader in education and social policy research, started following 1,764 high school students in nine career academy programs in Maryland, California, Florida, Pennsylvania, Texas and the District in 1993. There were only about 400 public school career academies in the country then. These programs were organized as small learning communities to combine academic and technical instruction around a career theme, such as health or electronics or transportation, and create partnerships with local employers who would show the students what their jobs entailed.
When I first heard of the study, I was a financial reporter wishing to be an education reporter instead. I grabbed any business story that had a connection to schools, and this one seemed perfect. I wrote a long feature about career academies in Oakland, Calif., and did stories whenever MDRC released new data as it followed these students through school. But as the years went by, I lost interest because the academy students were not doing better on standardized tests than non-academy students. There seemed to be little else happening with the program that was very newsworthy.
If I were smarter, I suppose, I would have realized the MDRC effort deserved more patient attention. I forgot how unique the study was. It was randomized. That is the gold standard for education research, and very rare. When the project began, the nine participating academy programs had more applicants than they had spaces, so students were selected by lottery. The 55 percent of the 1,764 applicants who won admission became the study's academy group and the 45 percent who lost the lottery became the control group. The ethnic character of each group was about the same: more than 50 percent Hispanic and about 30 percent black. Both groups were regularly surveyed for the next 15 years.
This avoided a common weakness in comparative studies. Often the students drawn to new programs have special qualities that make them interested in doing better. Comparing their progress to that of students not in the new program, as is often done, overlooks the fact that those personal qualities, not the new program, may explain any comparative success they achieve. The distortion is removed if their results are compared only to those of other students who were similarly motivated to apply.
I also overlooked the potential power of a survey that continued long after the students left school. I was surprised to discover, in the report just released, that MDRC had interviewed the participants eight years after their high school classes had graduated. Even more astonishing, to a reporter who has tried tracking down young people in that mobile age group, MDRC located 1,428 of the former high school students -- 82 percent of the academy group and 80 percent of the control group.
This was expensive. The study has so far cost $11 million, funded by the U.S. departments of Education and Labor and 18 private foundations and corporations. But it was worth it. Here is how the latest report by James J. Kemple with Cynthia J. Willner summarized the results:
"The Career Academies produced an increase in earnings of $132 per month during the first four years of the follow-up period and $216 per month in the final four years. Both of these results are statistically significant, meaning that it is unlikely that the differences arose by chance. On average, this represents an increase of $174 per month over the full eight years following scheduled high school graduation and an 11 percent increase in monthly earnings over the non-Academy group's average of $1,561 per month.
"The Career Academies produced an average increase of $311 in real monthly earnings for young men. This amounts to a 17 percent increase over and above the average earnings of $1,792 per month of young men in the non-Academy group.
"In all, one-third of the Academy group were living independently with their children and a spouse or partner, compared with 27 percent of the non-Academy group. This impact of 6 percentage points from the Career Academies represents a 23 percent increase in two-parent households over and above the rates of the non-Academy group.
"Specifically, the Academies increased marriage rates for young men by 9 percentage points (from 27 percent for the non-Academy group to 36 percent for the Academy group) and increased custodial parenthood by nearly 12 percentage points (from 25 percent for the non-Academy group to nearly 37 percent for the Academy group."
These findings are good news for educators and policymakers who have been trying to do something for several decades about the poor job and family histories of young urban Americans, particularly men. "Employment rates for African-American men ages 20 to 24 have been declining steadily since 1970, from around 77 percent in 1969 to only 56 percent in 2003," the study noted. "Even in the peak economic period of 1999, African-American and Hispanic males aged 16 to 24 were far more likely to be neither working nor enrolled in school than white males of the same age (22.8 percent and 12.8 percent versus 8.7 percent, respectively.)"
Young women in the academies also showed gains in some categories, but the results for men were the most eye-catching. The MDRC report acknowledges that the reason why the academy experience might have had such an effect on job and family life is unclear. The report says researchers are analyzing the data for clues, looking at various theories. For instance, the personal contacts and internships with career professionals might have given the students an advantage when looking for full-time jobs. Their larger salaries and greater job security might in turn have had an impact on their ability and desire to form stable families.
No one really knows yet what is going on. We should remember the social scientist's frequent warning that correlation does not mean causation. But the MDRC study was conducted with such care that the connections it has revealed have to be greeted with some confidence.
Those of us who have been quick to promote test score improvement as the ultimate measure of a school's worth should, in light of this research, be looking for other programs that have had long-term impact on students' lives, and suggesting that more organizations invest in similar long-term, randomized studies.
Kemple estimates that the number of career academies has climbed to at least 2,500 in the last 15 years. Some are labeled that way but do not include all of the elements MDRC looked for in its study participants -- small size, technical and academic instruction, career focus and close participation by outside experts. Several experts told me the pressure for better test scores in reading, writing and math in recent years has led some schools to step away from vocational programs. In many cases, they were discarding shop and home economics classes that did not come close to the career academy model. But now that we know what works, it would be good to encourage more schools to try it, particularly if it is something students choose.
As the study makes clear, about half of the academy students eventually earned college or community college degrees, or a skills-training certificate or license. That was no better than what non-academy students did after high school. But Kemple told me that the academy students' job success might have had the same effect as more college credits. "The magnitude of the impacts on monthly earnings for young men exceed differences in earnings that have been found in other research comparing young workers who have two years of community college with those who have only a high school diploma," the report said.
Learning on the job may be, for some young people, the better way to go. Thanks to the MDRC, we now have a deeper sense of new ways to help urban youth. I hope the people who did this research have lots of imitators.
