This post reviews Brian D. Ray, “Academic Achievement and Demographic Traits of Homeschool Students: A Nationwide Study” in Academic Leadership Live: The Online Journal 8, no. 1 (February 2010). [Available Here]
This is the latest of a long line of nearly identical studies Ray has been performing for decades now at fairly even intervals. In two previous posts I reviewed this large body of work, which you can read here and here. This new study tries very hard to overcome one of the most persistent deficiencies of his previous work (and the 1999 Rudner study)–the near exclusive reliance on HSLDA’s advertisement to recruit subjects, leading to unrepresentative samples. This time around Ray tried to recruit families from outside of the HSLDA orbit. Did he succeed? Not really. At least 76.8% of his very impressive sample (11,739 subjects!) are some sort of conservative Protestant. 95.2% are some kind of Christian. But he did try. Rather than just recruit from HSLDA and its affiliates as in past studies, this time Ray used not only the Bob Jones University Press testing service popular with conservative Protestants but also the Washington-based Family Learning Organization, which bills itself as an “inclusive and non-judgmental” organization, the North Carolina-based Piedmont Education Services, (no help here. It’s at least as conservative Protestant as BJU), and the Virginia-based Seton Home Study School, which is a service used primarily by conservative Roman Catholics. The use of Seton helps account for the impressive 12.4% of Ray’s subjects who were Roman Catholic. Ray also tapped several smaller organizations, but it must be noted that many of these also cater to conservative Christians.
Of his 11,739 subjects, at least 91.7 % are white (some of those selecting “other” I have a feeling are also white). 97.9% are married. 68.1% have at least 3 kids. 42.2% have at least four. 6.3% have seven or more children.
80% of families have mothers who stay at home full time. 62.5 % of the mothers in Ray’s sample have a four-year college degree or higher.
45.9 % of these families make over $80,000 per year. 28.4% make over $100,000.
In short, Ray’s sample is way more conservative, way more Christian, and way, way whiter than the American norm. It is also richer and enjoys more intact family life (i.e. far fewer broken homes or single parent situations). It has a far higher number of children too. If the stereotype of the typical homeschooling family is that of the lily-white fundamentalist Protestant (or conservative Catholic) with a stay-at-home mom and lots of kids, this study completely reinforces it. It would only add that this family is doing quite well financially and has parents who have lots of higher education.
Not surprising given this demographic, the test scores are off the charts. On all the tests administered to this sample of homeschoolers, students performed on average in the 84th to 89th percentile. Ray has been reporting stellar test results for decades, but these results are even higher than anything he has found before.
There were slight differences between groups of homeschoolers based on some variables, and no difference based on others. Type of curriculum (complete school in a box or eclectic approach) didn’t make any difference. Neither did level of state regulation. Very slight improvement in scores resulted from being a girl, from having fewer brothers and sisters, from having parents who had never been certified to teach (!), and from having a more structured approach to homeschooling [Ray asked families to rank the level of formality of their homeschool on a 1-7 point scale]. A little bit more statistical improvement was connected to families being more wealthy and spending more on homeschooling curricula. But the only variable that really seemed to make a significant difference was parent education level. Kids with parents with at least a college education did noticeably better than kids whose parents had less formal education. But the main point here is not these slight variations. The main point is that all the subjects performed way, way above national norms.
Why? Ray considers many possible answers at the end of his report but not the one that is obviously the case. Children from these families do well on these test because children from families like this always do well on tests no matter where they go to school. Ray does acknowledge as much in a couple of places in his report:
homeschool families and their students do not appear to be a completely representative cross-section of all families in the United States. And it was not possible within the constraints of this study to confirm whether this sample is representative of the population of home-educated students.
Despite acknowledgments like this Ray repeatedly suggests that while his study doesn’t prove that homeschooling itself is what makes these kids do so well, “it may be that something about the typical nature and practice of home-based education causes higher academic achievement, on average, than does institutional state-run schooling.” As much as he may want to think this, there are actually two places in his own data that suggest otherwise.
The first comes in his comparison of students in his sample who were homeschooled their entire lives (59%) with those who have only homeschooled for a little while. He found no statistical relationship at all between number of years homeschooling and achievement. Later in the study Ray compares the scores of his homeschooled sample with scores from three of his large testing companies that include students enrolled in private Christian schools. The scores are nearly identical. From these two tidbits it is clear that what Ray is measuring is not a homeschooling effect but a sociological effect of stable, two-parent, middle-to-upper-middle class, white families whose parents are deeply committed to their children’s well-being. Children with advantages like this are going to shine no matter what kind of schooling they get.
Having said this, I want to end by praising the the study rightly interpreted. I know it is going to be misrepresented by many homeschooling advocates (possibly by Ray himself) as another study that proves homeschooling is better than public education. I know many unschoolers and others for whom the attraction of homeschooling is that it gets their families away from the standardized testing obsession are going to reject its very premises and see it, accurately, as ignoring their wing of the movement. But for all its demographic limitations (despite Ray’s praiseworthy efforts to broaden his sample, this is still the same basic group of homeschoolers he’s always studied, with the addition of some conservative Catholics) Ray has shown, yet again, that homeschooling does not by itself harm children academically. Middle class white kids from stable families who would excel in public or private schools perform at least as well on standardized tests if they’re homeschooled. Rather than insinuating that homeschooling is responsible for these impressive scores, I wish Ray would interpret his study as showing, as it so capably does, that homeschooling doesn’t make priviliged kids do worse on tests than they would have done had they gone to school. Homeschooling is no academic disadvantage. That’s newsworthy enough I think.
is the first scholarly book-length treatment of its theme. It is available for purchase
from 
Great analysis.
I think that most of us with helpful and loving neighbors and family members who seem somewhat hysterically concerned about the education or perceived lack of education of our homeschooled kids, will be happy with the general conclusion that homeschooled kids are not worse off than their school counterparts :).
I’m really not sure that he could get anymore diversity within his study groups – from where I sit the diversity really is a small segment – and many who fall within the more multicultural or secular group tend to have unschooling tendencies and therefore are not participating in testing (unless mandated by their states). What I find really interesting in these studies are the facts that the level of state regulation doesn’t seem to affect scores and the certification of parents in teaching affects them in the opposite manner of what might be expected. Now that people seem to be over the socialization argument, those two things are really what I hear the most in regards to homeschooling – that it should be regulated and that parents should be certified to teach.
Thank you for this analysis. I have often felt this to be the case with Brian Ray’s work and it is understood by many in the homeschool community that his reliance on HSLDA is a major issue with his results. Those of us who do not associate with HSLDA obviously would not be represented. The claim that homeschoolers are white, upper middle class, large families, and Christian is something we have been fighting against for decades.
What is also important to note; in the early stages of data gathering from the Family Learning Organization was their lack of getting release forms from their clients. For those of us who shy away from having our families studied, this was a huge breach.
All in all, I do not need this type of study to confirm my educational choice. I do not support using test scores as a means to measure the success of any educational method. In my mind, these are “fast food” methods. I’m comfortable knowing children are learners by nature and the likes of public school teaches them not to trust themselves and to rely too much on others. Just look at the average citizen – a product of over 100 years of public schooling.
I think the answer is more basic than either analysis implies. The children excel simply because there is a greater degree of expectation imposed; on the children, by their parents; by the parents, on themselves, to do whatever it takes to achieve a desired outcome. Learning is an inherently natural human characteristic. Parents provide a richness of opportunity according to their economic ability to do so, but also by resourcefulness in utilizing the world around them.
The only way for Ray to attain a more accurate demographic sampling would be to ask every school district to include anonymously, and voluntarily, submitted surveys with every application for exemption from Compulsory Attendance. His results are skewed by the fact that a majority of the submissions for sampling come not just from families within a narrow socio-economic profile, but also from families less opposed to being observed and evaluated according to equally narrow measurements.
The problem of the “Pedigree Sheet” outcome is that not everyone chooses to be defined by it. So it will always reflect only those willing to be driven by it.
[…] Recently I reviewed Brian Ray’s new study on homeschooler achievement. I didn’t bother making this point then, but it really is an important one. Ray’s study compares homeschooler scores on tests like the California Achievement Test to national averages. But, as Brodie’s description explains, homeschoolers are at a considerable advantage taking these tests. Even assuming that their mothers or siblings are not actually helping them fill out the test or offering subtle hints and corrections over their shoulders (and I wouldn’t be surprised at all if some actually do), such students are taking these tests in the comfort of their homes in a low-stress situation and, conceivably, without the same time constraints. They can also take them at their own pace rather than all at once. Julia took one section of the California Achievement Test per day for an entire week, meaning that she’d be fresh going into each section. Kids in schools are taking them one after the other. […]
Excellent critique of the conclusions that can be warranted by / drawn from sample-based research data.
@ Dan Overmitten | RE:
“All in all, I do not need this type of study to confirm my educational choice. I do not support using test scores as a means to measure the success of any educational method….”
— While I can personally relate to this sentiment, the point that this comment fails to miss is that the majority of (American) society *does* need to see these types of studies / results, based on popularly-believed-to-be-accurate “test scores”. Yes, homeschoolers are busy people, and not every family is willing to fight the battle for broad-scale societal/educational transformation. But the “I know what I know, and to heck with the rest of the world”-type comments, as Dan’s appears to be an example of (although I am not trying to pick on him), could possibly be viewed as an insular, selfish attitude on the part of homeschoolers who lack concern for the educational/schooling plight of those outside of the homeschool camp. If homeschooling works for you, why not do what you reasonably can to “evangelize” others, and “reach” them where they already are, e.g. with studies and test scores?
“Rather than insinuating that homeschooling is responsible for these impressive scores, I wish Ray would interpret his study as showing, as it so capably does, that homeschooling doesn’t make priviliged kids do worse on tests than they would have done had they gone to school. Homeschooling is no academic disadvantage. That’s newsworthy enough I think.”
Agreed. And if we have strong evidence supporting the notion that “Homeschooling is no academic disadvantage,” then we can begin to look at education on a broader societal level in terms of cost-effectiveness:
I would guess that the “cost per student” that homeschool families spend per year, per child is far less than what is spent per student at the average public school. If we have research-based evidenced that this lower-cost approach to education is not *less* effective than the more expensive institutional, public schools, then on the broader societal level we really have ground to question why we are willing to spend so much extra money on one approach (institutional/public schooling) that is not necessarily any more effective than a much lower cost alternative approach (home based education).
Most of the analysis I’ve seen comparing costs fails to even come close to showing an accurate comparison. To even start to make them more comparable, you have to factor in the amount of “unpaid” educational time parents apply to homeschooling or remove the teacher and facility costs from the student average. Essentially, it would have to be a comparison of curriculum and supply costs I’m thinking. In which case, I’m willing to bet I spend more on an annual basis because they can reduce cost numbers significantly through the reuse of materials. And don’t forget the amount of materials that parents are now additionally required to financially provide.
Hmmm.. has there been a study that looked at the scores of homeschooled ADOPTED kids?
That might at least tell us whether the scores are genetic or environmental.
Of course, if it gives the conclusion “having educated married parents helps kids achieve” I don’t really see how this can give us any public policy advice.
One thing I HAVE noticed among my homeschooling friends: We all, especially in the early years, aim for MASTERY. As in, we don’t move on until a child has mastered a skill, so all the kids get a good grounding in reading and basic math.
Also, homeschoolers are willing to ‘school’ on weekends and through summers, if necessary.
And, while the costs may end up a toss up when you figure in teacher salaries (Though I doubt it, as there’s a lot of other we don’t have, simply because, for instance, mortgages get paid whether or not you homeschool and the public library is there all day whether or not it’s in use), I wonder about TIME costs for the kids? I can achieve better than average results with my first grader in ONE hour of instructional time per day–which gives her more time to play, explore outside, go to musueums, do crafts, etc. etc.
One of the big downsides of school, and one of the reasons I started homeschooling, is wasted TIME. Whole day Kindergarten, for example, isn’t about raising achievement–it’s about free daycare. But if I’m home with the smaller kids, my daughter doesn’t NEED free daycare. First grade could probably be reduced to a half day as well—
So glad to read this as I’ve been struggling to find unbiased information regarding homeschooling and finding none. There seems to be a large battlefield and no one willing to walk across the field to discuss the truth of the matter. We’ve been handed from our son’s kindergarten teacher the ever so helpful knowledge that there would be gaps in his learning if we pull him out and homeschool him. The word “gaps” being some secret teacher speak for something that they evidently cannot seem to define for me. I think the bottom line that it certainly doesn’t hurt seems to be the best answer.
Find it hard to believe that the most we can take from Ray’s study is that homeschooling is not a disadvantage….
Another thought, perhaps more than ‘test’ scores should be considered when we think of advantages and disadvantages to children.