Ray and Eagleson on Government Regulations and SAT Scores

This post reviews Brian D. Ray and Bruce K. Eagleson, “State Regulation of Homeschooling and Homeschoolers’ SAT Scores” in Academic Leadership: The Online Journal 6, no. 3 (14 August 2008).  [Available fulltext here]

Ray, founder and president of the National Home Education Research Institute (NHERI), and Eagleson, Chief of Emergency Medicine at a hospital in Lebanon, PA, here present the results of a study of over 6,000 homeschooled students’ SAT scores nationwide to argue that homeschoolers’ academic achievement is not affected by the degree to which homeschooling is regulated by the states. 

Ray and Eagleson begin with an introduction that summarizes the extant literature on homeschooler academic achievement (most of it done by Ray or other HSLDA-affiliated researchers).  They acknowledge that none of this literature meets the standards of rigorous social science–none of the studies control for key variables or have random sampling.  Given the absence of reliable research, both advocates and critics of homeschooling resort to theoretical arguments for why government should or should not regulate, arguments ably summarized in this paper.

But theoretical arguments are not what is needed.  We need good, solid evidence upon which policymakers can rely for guidance.  That is what Ray and Eagleson hope to provide here.  They divide the various state statutes and policies concerning homeschooling into three categories (categories that match those of HSLDA’s famous color-chart): states with low, moderate, and high levels of regulation.  They then compare the SAT scores of students from these differently-regulated states to determine if there is any relationship between level of regulation and SAT score.

The College Board (supplier of the SAT) provided Ray and Eagleson with the scores of all 6,170 students self-designating as homeschoolers who took the SAT in 2001.  They did not receive individual scores but only the average score for each state and the number of males and females from each state who took the test.

Ray and Eagleson took the data provided by the College Board (dropping North and South Dakota, both of which had fewer than seven homeschoolers take the test) and compared the SAT scores with the degree of regulation (low, moderate, and high).  They acknowledge that they must take it on faith that students self-designating as “homeschooled” were in fact homeschooled, though for how long or in what fashion they have no way of knowing.

Given these caveats, Ray and Eagleson found that there was no stastically significant difference in SAT performance between homeschoolers in states with low, moderate, or high levels of regulation.  In fact, in every case, states with the highest levels of regulation actually had the lowest test scores (though not enough to make it statistically significant).  This was true for states that hadn’t changed their laws in ten years and also for states who had not changed their laws for five years.  Ray and Eagleson provide statistically-literate readers with all of the charts and explanation needed to give them confidence that the data is legitimate.

What are we to make of the findings?  First, Ray and Eagleson acknowledge limitations:  we don’t know how long these students were homeschooled or how long they lived in the state wherein they took the test.  Noting, however, that far fewer children are homeschooled at the upper grade levels, Ray and Eagleson speculate that those who took the SAT were very likely longtime homeschoolers.  I’m inclined to accept this speculation on the whole, especially as the data comes from 2001, before the significant increase in homeschooled highschoolers brought about by the spread of cybercharters and other hybrids.

Second, Ray and Eagleson note that their results could be taken two ways.  Supporters of increased regulation could see the evidence and assert, “see, regulating homeschooling doesn’t hurt achievement.  Since it brings other goods (prevention of child abuse and neglect, for example), we should go ahead and regulate homeschoolers more.”  In contrast, opponents of regulation could say, “see, regulation doesn’t make homeschoolers do any better, and they’re doing fine without it.  Leave them alone.”  Ray and Eagleson of course fall into the latter category and conclude with a request that legislators not rush to regulate and test homeschoolers.

Finally, Ray and Eagleson end with an appeal to researchers of homeschooling to “make healthy and creative efforts to find ways to gather more complete data regarding these persons’ demographics and educational history.”  They do not note, however, that the easiest way to do this would be for states to collect data on all homeschoolers through some sort of registration process.  If every state in the country did so we’d have a hard figure for how many homeschoolers there are as well as data on race, SES, age, sex, religious affiliation, and whatever else was collected.  Philosophically Ray may be against requiring homeschoolers to register and fill out a form providing this sort of data, but Ray the researcher would no doubt be glad to get ahold of such information if it were available!

I’ll go one step further.  Ray and Eagleson repeatedly acknowledge what Ray’s critics have been saying for years now, that the research he and others have conducted on academic achievement lacks the sorts of controls and sampling good social science requires.  An easy remedy to this would be to require all children–homeschooled, private schooled, public schooled–to take some sort of nationally normed test.  I know homeschoolers would resist this with all their political might, but I for one would love to have that data, and I have a strong suspicion that homeschoolers would outperform their public-schooled peers.  A universal test like this would allow us to control for the variables that undermine all previous studies and would silence all academic criticism of homeschooling.  It would also smoke out the few bad apples who, when they hit the headlines, give homeschooling a bad name.

But all of that is just speculation.  The bottom line here is that Ray and Eagleson have produced here a fine study that acknowledges its limitations and does not overgeneralize from its findings.  It concludes soberly, not triumphantly.  While there are a few questions that remained for me (especially whether SAT scores among homeschoolers vary by region as widely as do those of public schooled children), and while it would have been nice to have the raw scores and numbers of students by listed by state, I am grateful to Ray and Eagleson for this valuable article.