New Ray/HSLDA Study on Homeschooler Achievement

This post briefly reviews preliminary releases of the new study conducted by Brian Ray for HSLDA called “Homeschooling Across America: Academic Achievement and Demographic Characteristics.”  The full study is scheduled for release in November 2009.

While the full report has not yet been published, HSLDA has already posted a press release describing its scope and celebrating its finding that homeschoolers score on average 36-37 percentage points higher than public schooled children on a wide range of standardized tests.    You can read the press release here.  HSLDA has also issued a glossy PR piece complete with graphs and footnotes, which can be accessed here.

Both of the above linked documents describe how Ray used a cross-sectional sample of 11,739 participants from all 50 States, Puerto Rico, and Guam.  These participants submitted scores from the California Achievement Test, the Iowa Test of Basic Skills, or the Stanford Achievement Test along with demographic information.  Ray found that student scores were remarkably consistent, hovering in the 84-89 percentile range across all of the tests and all subjects.  Variables like family income, parent educational level, amount of money spent on homeschooling curriculum, and level of state government regulation all had little to no impact on outcomes.

Ray himself has issued a summary of his study at NHERI’s website, which you can read here.  His rhetoric is less celebratory but the same conclusions are reached.  Homeschool student achievement test scores are “exceptionally high” and do not vary significantly due to any of the variables mentioned above with the slight exception of parent education level (children of better educated parents scored about 2.5 % better than those of parents without formal degrees).

With the caveat that I have not yet seen the full report, let me make the following observations about this new study:

1. This study, though broader in scope than previous Ray studies, is characteristic of his previous work in that it relies on volunteers recruited by HSLDA for the study itself.  On HSLDA’s website you can still read the recruiting letter that was sent out in February of 2008.

Several months ago on this blog I paid extensive attention to Ray’s previous studies (post 1 and post 2).  What I said of his previous studies is true as well of this new one.  It is simply not legitimate to compare the scores of this selection of homeschoolers to national averages of public schooled students.  We are dealing here not with a representative sample of all homeschoolers but with a self-selecting sample of families recruited by HSLDA for this study.  These recruits are on the whole far better educated than the general population (62.5% of mothers had completed college, vs. 28% nationwide), are almost all white (92%), are two-parent households (98%), have full-time working fathers (98%) and stay-at-home mothers (81%).  It is not at all surprising that a sample with these demographics would compare very favorably against a sample of public school students that includes children from minority populations, non-English speakers, broken homes, unemployed fathers, working mothers, and so on.  Let me quote what I wrote in a previous post about these Ray studies:

What Ray ought to be saying and what other journalists and pundits who use his studies ought to be saying is not that homeschoolers outperform public schoolers. They ought to be saying that some middle-class, white, two-parent, conservative Protestant homeschoolers who volunteered for a research study that was pitched to them as a great opportunity to show off homeschooler success to the public, score in the 80^th percentile or above on standardized tests. That’s really all Ray’s studies show.

2. To what extent is homeschooling itself responsible for the success of these middle-class white students from intact two-parent families?  Well, according to Ray’s study itself, not much.  Ray says on his website that his study found “no statistically significant difference in achievement by whether a student has been home educated all his or her academic life…” The HSLDA PR piece notes that the study found “very little diffference between the achievement scores of students who had been homeschooled for their entire academic lives and those who had not.” (p.3)

These two small sentences make it clear that what Ray is measuring is not in fact the result of homeschooling.  Children of two-parent middle class white families are going to do well on tests no matter what sort of education they receive, because the secret to academic success is not the kind of education you receive but the kind of family you are born into.  Most homeschooled children are fortunate to have been born into families with fathers who are committed to their wives and children, with a home culture that values learning and literacy, with a network of extended family and friends who reinforce the family’s positive messages of love and discipline.  It’s contextual factors like these, not the curriculum or pedagogy of homeschooling, that lead to academic success.

The HSLDA press release tries to answer critics of this study by defending the idea of sampling.  But that is not the issue.  The issue is that Ray’s sample, like the Rudner study done a decade ago, simply cannot be compared to the national population.  If you compared Ray’s sample to a sample of white, two-parent, single income, multiple child families who send their children to public or private schools, the test score differential would evaporate.

When the full report comes out in November I’ll give it another look.  For now it seems to me that this report is just the latest in a long line of studies that do indeed show that volunteers recruited by HSLDA are doing a great job teaching their kids.  But the results are then, disingenuously, used as “evidence” to “prove” that homeschooling is better than public education.  Non sequitur.