This post summarizes our research review, which provides a critical analysis of Rudner (1999). Click HERE to read a more in-depth version of the arguments presented.
In 1998, Michael Farris, the president of HSLDA, hired education researcher Lawrence M. Rudner to carry out a study on homeschoolers’ academic achievement. The results of this study were distributed in Home Schooling Works!, a 1999 summary of Rudner’s findings which is presented on HSLDA’s website. This summary, whose authors are unknown, misrepresents Rudner’s work in an attempt to further HSLDA’s agenda. Rudner’s actual paper, which was published in the peer-reviewed journal Education PolicyAnalysis Archives in 1999, is linked to by HSLDA and is available here.
These two publications are widely cited to support the claim that homeschoolers have higher levels of academic achievement than other children: Rudner reports that, for every subject and at every grade level, his participants’ average score was between 60% and 90%. The participants’ best subject was reading (where they scored 85.7% on average) and their worst subject was math (where they scored 75.1% on average). Around one-quarter of Rudner’s participants were ahead of their public schooled peers by at least one grade level, and they outpaced public schoolers as they advanced through the grade levels. Several demographic factors were found to have a significant positive effect on the participants’ scores, including family income, expenditure per child, and parents’ education level.
However, the design of the study severely limits our ability to draw conclusions from it. As Rudner himself says, “This study does not demonstrate that home schooling is superior to public or private schools…It does not indicate that children will perform better academically if they are home schooled.” The major impediments to drawing such a conclusion are:
1) Selection bias
Rudner’s sample of homeschoolers was not random. All of Rudner’s approximately 20,000 participants were recruited when they contracted with the evangelical Bob Jones University Press Testing and Evaluation Service, one of four major testing services for homeschoolers in 1998. However, only an estimated 16% of homeschoolers nationwide used such a testing service that year. It is likely that these homeschoolers were high achievers and had parents who were invested in their educations and expected them to do well on standardized tests. Unsurprisingly, Rudner’s participants scored around the 80th percentile—just as we would expect from a study designed only to test high achievers.
2) Non-representative sample
Rudner’s participants are not comparable to the population of school-age children in the United States, nor are they representative of all homeschoolers. Rudner’s sample of homeschoolers was disproportionately white (94%), Christian (94%), and young (only 11% of participants were in high school). They were also unequally distributed by state of residence. Most participants had married parents (97%) and large families (62% had families with three or more children), and their parents were highly educated (66% of their fathers and 57% of their mothers had at least a Bachelor’s degree) and had high incomes (only 8% of Rudner’s sample fell into the lowest income bracket, while 33% of homeschoolers fall into this category).
Rudner’s study tells us essentially nothing about homeschooled high schoolers, children of color, poor children, unschoolers, children with poorly educated parents, children being raised by single parents or by parents who both work, abused or educationally neglected children, or disabled or special needs children. The higher-than-average standardized test scores earned by Rudner’s highly privileged group of homeschoolers are only what we would expect from a study where nearly all disadvantaged children are excluded.
Rudner states very clearly in his introduction that “this study is not a comparison of home schools with public or private schools” [emphasis added], since homeschoolers are not representative of the general US population. This means that Rudner’s study does not show that homeschoolers score better than public schoolers. It merely assembles the scores of a particularly privileged group of homeschoolers and shows that they are capable of achieving high scores on standardized tests.
Unfortunately, this study’s findings have been continually misrepresented by HSLDA. Their online summary, entitled Home Schooling Works!, omits discussion of the study’s methodology and replaces Rudner’s conclusions with an op-ed by Michael Farris. In this piece, Farris claims that “It is clear that the average home school child performs significantly higher than the average public school child”—a finding which Rudner’s study explicitly does not show.
Farris further claims that “it would be contrary to the evidence to suggest that public school regulatory measures are justifiably imposed on home schoolers” because “it is only safe and fair to conclude that home education works well for those who are choosing this form of education.” But Farris has no evidence for this claim.
In the end, it doesn’t matter if Farris is deliberately deceiving the homeschooling parents who trust him for political gain, or if he is merely ignorant of what Rudner is actually saying. Either way, Farris is harming the cause of homeschooling by promoting it as a miracle cure for all families. Homeschooling alone does not produce better outcomes for children: supportive, involved families with high levels of resources produce better outcomes, no matter how their children are educated. Rudner’s evidence supports this hypothesis.
Correcting the Record: A Look at Rudner 1999
This post summarizes our research review, which provides a critical analysis of Rudner (1999). Click HERE to read a more in-depth version of the arguments presented.
In 1998, Michael Farris, the president of HSLDA, hired education researcher Lawrence M. Rudner to carry out a study on homeschoolers’ academic achievement. The results of this study were distributed in Home Schooling Works!, a 1999 summary of Rudner’s findings which is presented on HSLDA’s website. This summary, whose authors are unknown, misrepresents Rudner’s work in an attempt to further HSLDA’s agenda. Rudner’s actual paper, which was published in the peer-reviewed journal Education PolicyAnalysis Archives in 1999, is linked to by HSLDA and is available here.
These two publications are widely cited to support the claim that homeschoolers have higher levels of academic achievement than other children: Rudner reports that, for every subject and at every grade level, his participants’ average score was between 60% and 90%. The participants’ best subject was reading (where they scored 85.7% on average) and their worst subject was math (where they scored 75.1% on average). Around one-quarter of Rudner’s participants were ahead of their public schooled peers by at least one grade level, and they outpaced public schoolers as they advanced through the grade levels. Several demographic factors were found to have a significant positive effect on the participants’ scores, including family income, expenditure per child, and parents’ education level.
However, the design of the study severely limits our ability to draw conclusions from it. As Rudner himself says, “This study does not demonstrate that home schooling is superior to public or private schools…It does not indicate that children will perform better academically if they are home schooled.” The major impediments to drawing such a conclusion are:
1) Selection bias
Rudner’s sample of homeschoolers was not random. All of Rudner’s approximately 20,000 participants were recruited when they contracted with the evangelical Bob Jones University Press Testing and Evaluation Service, one of four major testing services for homeschoolers in 1998. However, only an estimated 16% of homeschoolers nationwide used such a testing service that year. It is likely that these homeschoolers were high achievers and had parents who were invested in their educations and expected them to do well on standardized tests. Unsurprisingly, Rudner’s participants scored around the 80th percentile—just as we would expect from a study designed only to test high achievers.
2) Non-representative sample
Rudner’s participants are not comparable to the population of school-age children in the United States, nor are they representative of all homeschoolers. Rudner’s sample of homeschoolers was disproportionately white (94%), Christian (94%), and young (only 11% of participants were in high school). They were also unequally distributed by state of residence. Most participants had married parents (97%) and large families (62% had families with three or more children), and their parents were highly educated (66% of their fathers and 57% of their mothers had at least a Bachelor’s degree) and had high incomes (only 8% of Rudner’s sample fell into the lowest income bracket, while 33% of homeschoolers fall into this category).
Rudner’s study tells us essentially nothing about homeschooled high schoolers, children of color, poor children, unschoolers, children with poorly educated parents, children being raised by single parents or by parents who both work, abused or educationally neglected children, or disabled or special needs children. The higher-than-average standardized test scores earned by Rudner’s highly privileged group of homeschoolers are only what we would expect from a study where nearly all disadvantaged children are excluded.
Rudner states very clearly in his introduction that “this study is not a comparison of home schools with public or private schools” [emphasis added], since homeschoolers are not representative of the general US population. This means that Rudner’s study does not show that homeschoolers score better than public schoolers. It merely assembles the scores of a particularly privileged group of homeschoolers and shows that they are capable of achieving high scores on standardized tests.
Unfortunately, this study’s findings have been continually misrepresented by HSLDA. Their online summary, entitled Home Schooling Works!, omits discussion of the study’s methodology and replaces Rudner’s conclusions with an op-ed by Michael Farris. In this piece, Farris claims that “It is clear that the average home school child performs significantly higher than the average public school child”—a finding which Rudner’s study explicitly does not show.
Farris further claims that “it would be contrary to the evidence to suggest that public school regulatory measures are justifiably imposed on home schoolers” because “it is only safe and fair to conclude that home education works well for those who are choosing this form of education.” But Farris has no evidence for this claim.
In the end, it doesn’t matter if Farris is deliberately deceiving the homeschooling parents who trust him for political gain, or if he is merely ignorant of what Rudner is actually saying. Either way, Farris is harming the cause of homeschooling by promoting it as a miracle cure for all families. Homeschooling alone does not produce better outcomes for children: supportive, involved families with high levels of resources produce better outcomes, no matter how their children are educated. Rudner’s evidence supports this hypothesis.