This post reviews Ray Pennings, et. al., Cardus Education Survey (Hamilton, Ontario, Canada, 2011) [available here]
Phase 1 of the Cardus Education Survey was released a few weeks ago and has garnered significant national attention for its insights into private Christian schooling. Though not the report’s major emphasis, it also includes some very interesting information about homeschooling.
The survey aims to discover the outcomes of private Christian education by examining the lives and attitudes of graduates age 24-39, focusing especially on these graduates’ spiritual formation, cultural engagement, and academic achievement.
The Cardus data derives from two sets of surveys conducted by another firm (Knowledge Networks). Both sets of surveys were random samples. This is very important. The individuals providing information were not volunteers or recruits. They were randomly sampled from the entire national population (for full details see pp. 44-45 of the report)
For the purposes of this blog I’m going to ignore the fascinating information the survey reveals about private schooling and look only at the homeschooling data. The researchers were able to separate out the religious and non-religious homeschoolers based on whether or not their mothers attended religious services regularly. The report, concerned as it is with religious schooling, only gives results for the religious homeschoolers.
And what are the results?
1. This random sample of young adults who were religious homeschoolers looks a whole lot like young adults who attended Protestant schools when it comes to their spiritual lives. They go to church a lot, respect religious authority, have a private devotional life, believe in traditional theology and morality, and share a religious life with their spouse. Most of this has more to do with what their parents are like than with the act of homescooling itself, the authors claim (based on their rich data that allows them to control for variables like parental church attendance rates). On most such measures former homeschoolers score just a bit below young adults who attended Protestant day schools but well above graduates of Catholic or non-religious private schools.
When it comes to giving, however, religious homeschoolers look a lot more like Catholics and non-religious folks. They don’t give much money to their churches or to other religious or charitable causes.
2. On the topic of cultural engagement the results are much more interesting. Religious homeschoolers reported much higher rates of feelings of helplessness about dealing with life’s problems and of lack of clarity of goals and sense of direction. They get married at far younger ages than all other groups, have fewer children than the national average, and get divorced more than other private school graduates. The divorce finding is particularly provocative, for the researchers found a very large effect of homeschooling itself here, and not just demographic background variables. What that means is that lots of young adults who were homeschooled are getting divorced even though their parents weren’t.
As for politics, religious homeschoolers were pretty apolitical. Though former homeschoolers thought of themselves as pretty well connected to people in power, they didn’t do much by way of political action themselves.
3. Finally comes academic achievement. Former homeschoolers reported far lower rates of preparedness for college than all private school graduates. They were more likely to attend “open admission” universities (the least selective kind of higher education), less likely to attend prestigious universities, and in general less likely to attend college and especially graduate school. They also reported lower SAT scores than the other private schoolers in the sample.
So that’s the story. Very soon after the report was issued HSLDA put up a critique by Jeremiah Lorrig that dismissed the results because the sample of homeschoolers in the survey was so small. Lorrig instead would have us look to Brian Ray’s 2003 study of adult homeschoolers and similar 2009 study because they have much larger sample sizes. That they do, but their samples are not randomly generated. Instead, they are the result of aggressive recruitment by HSLDA, whose clear goal was to use the results of the survey as promotional material celebrating homeschooling’s wonders. The samples in these studies are so biased that they cannot really be generalized at all. [See my longer critique of Ray’s oevre here and here.]
That is not to say that this Cardus survey is necessarily an accurate reflection of what adult homeschoolers are like. The report itself does not give the raw numbers for how many respondents fit into the various categories of private school, so I asked David Sikkink, who was the principal researcher for the quantitative portion of this study about the sample size. My colleague Rob Kunzman at Indiana University asked him the same thing. Here’s what we learned:
Here are the raw numbers of individuals from each school background:
873 Public school
283 Catholic school
124 Conservative Protestant or “Christian School”
109 Nonreligious private school
61 Religious home school
21 Nonreligious home school
HSLDA’s point that a sample of 81 doesn’t capture the full picture is doubtless true, but the larger truth is that this is by far the best study we’ve ever had of adults who were homeschooled. 81 is actually pretty good! And you’ll remember that these are not volunteers. These were randomly selected individuals, which means we avoid the besetting bias of all the Ray studies that rely exclusively on recruits.
Having said that, there are a couple of points I want to make about the data itself. The first one was made very well by Rob Kunzman in an interview at the Jesus Creed blog. 2010 data about 24-39 year-olds is actually telling us about what homeschooling was like 10 to 20 years ago. And homeschooling has changed a lot since then. In particular, there’s a lot more fluidity between homeschooling and institutional schooling than there used to be, what with cybercharters, dual enrollment programs, part-time schooling, and much else. Grouping children into “homeschool” and “private school” categories simply fails to capture this fluidity. It also misses the fact that most homeschoolers don’t do it from k-12. Many, many children spend a year or two or more homeschooling and then return to public or private schools.
The authors of the Cardus survey are aware of this issue. Dr. Sikkink told me in his email that the group is working on a more careful analysis of the homeschool data that will take such issues into consideration. I hope this future study will also give us information about the non-religious homeschoolers in the sample.
Now let me say a word about the divorce issue. Let’s assume that Cardus has uncovered something that’s really happening–that young adults who were homeschooled in a religious setting are getting divorced at significantly higher rates than graduates of public or private schools. The report finds homeschooling itself to be the explanation for this. I doubt it. Lots of data on divorce correlates it to early marriage and educational level. People who marry young (especially if they have limited higher education) tend to get divorced, and the homeschoolers in this sample married young and had less higher education than the other groups. Perhaps we could say that “homeschooling” is a factor in all of this in that the sort of families that encourage their kids to get married young often are homeschooling families (think the whole “I Kissed Dating Goodbye” courtship thing), but to me that says more about the sort of people who were homeschooling in the 1990s than about homeschooling as such.
Finally, there remains the possibility that this sample of homeschoolers for whatever reason just may happen to be nonrepresentative of the general population. We’ve seen in the previous two NCES reports the estimated numbers of African American and Hispanic homeschoolers fluctuate wildly. Why? Probably because the sample size of these groups was so small that extrapolating from them to the entire population just doesn’t work. It could be that the researchers just happened to get more homeschoolers than average who didn’t do well academically, failed at marriage, have checked out of politics, and feel that their lives are adrift. Certainly there’s a lot of other evidence out there (though almost all of it is anecdotal and qualitative) that suggests homeschoolers are not on the whole like this. Maybe there is some weird sampling error going on.
Or maybe there isn’t and maybe this data catches the sorts of homeschoolers who don’t typically fill out the Brian Ray questionnaires. I don’t know. What I do know is that this study illustrates even more clearly the point I made last week–we need more, much more quantitative study of the long term results of homeschooling.
Milton Gaither
Messiah College
Thank you for reviewing this! I saw a sort of infographic about the study not long ago and was trying to get a handle on what the data was really saying. I’m also curious about whether there were differences between the religious and nonreligious homeschool groups. Those categories also seem to ignore a continuum of approaches.
As far as academic achievement, it seems as if homeschooling has dramatically improved in the last decade. Recent improvements wouldn’t show up in this study because not enough time has passed for the graduates to reach the age of 24, which was the minimum age for this study.
Secondly, homeschool graduates are more likely to come from large families that cannot afford expensive tuition charged by the more prestigious universities. They may settle for universities within an easy commute from home to forgo the cost of a dorm, or they may select a less prestigious university that offers a scholarship. It could also be that homeschool families are less likely to be impressed with “prestige”.
A good study should account for such factors as family size, and socioeconomic status. The study should have also looked at more recent graduates SAT scores and ability to earn scholarships. Otherwise, readers will draw wrong conclusions from the study.
I agree with the need for long-term logitudinal data on homeschooling. In such a study, I think we’d need far more than 80 children. Within our own local homeschool group (all Kansas Catholic homeschoolers), the 90 families differ extensively in use of curriculum, integration of faith in studies, education of parents, methods (classical, Charlotte Mason, unschooling, Montessori, etc.), access to resources, learning disabilities, autism, family size and all sorts of other variables that may (or may not) impact the future lives of homeschoolers. With a group so diverse, the sample size would have to be extremely large. But so fascinating!
I’m pleased that they are willing to invest in a serious academic study with random sampling. I hope there are more to come!
-Jeremiah Lorrig
I agree with californiaarchitect. The study does not show the whole picture and actually may vibrate inaccurate data.
Aside from the so-called data that Cardus collected, it still does not show the all- encompassing value that independent home study brings to the soul and mind.
Yeah – my family switched to homeschooling when I was in High School. So I didn’t really do it (a couple years) but my young siblings did and all the stats apply to them now – drifting, low education, unawareness of politics, early marriage (seemingly without much thought) and general lostness.