One of the sources most commonly cited to support claims that homeschoolers outperform public schoolers academically is Home Schooling Works!, a 1999 web summary of the findings of a study conducted by Lawrence M. Rudner in 1998. The study was designed by Michael Farris, the president of HSLDA, who hired Rudner (a reputable education researcher) to carry it out. Rudner’s findings are presented on HSLDA’s website in a highly deceptive summary (compiled by unidentified authors) which omits much discussion of the study’s methodology and Rudner’s conclusions, adding instead an op-ed by Michael Farris which purports to explain the study’s findings. Rudner’s actual paper, which was published in the peer-reviewed journal Education Policy Analysis Archives in 1999, is linked to by HSLDA and is available here.
Rudner’s published study does not claim that homeschoolers outperform public schoolers; in fact, Rudner clearly cautions against drawing this conclusion in the discussion of his results, which is omitted from HSLDA’s web version. Instead, what his study actually shows is that a small, highly privileged set of homeschoolers are capable of scoring highly on standardized tests. Their scores cannot be compared with public schoolers’ scores, nor are they representative of homeschoolers as a whole. In effect, the study answers the question “Is it possible for homeschooled children to score as well as other children on standardized tests?” with “Yes, it is possible”—Rudner makes no statement on how likely this is. (Compare a hypothetical research study which answered the question “Is it possible to become a millionaire?” with “Yes, it is possible”—would you then conclude that everyone is a millionaire?) The problems with using Rudner’s study to draw these conclusions have been pointed out by a number of critics from both within and outside the homeschooling movement.
In the sections that follow, I will first outline the major points made by Home Schooling Works! and those made in Rudner’s published article. Then I will provide a critical analysis of Rudner’s study, and finally I will summarize what Rudner’s results actually mean.Home Schooling Works! major points
Home Schooling Works! is an online narrative summary, divided into several pages, of Rudner’s 1999 study. It is posted on HSLDA’s website without attributions for its authors, except for the portion which contains an op-ed by Michael Farris. It is unknown to what degree Rudner participated in condensing his study’s findings for a web audience. This section will summarize the claims made by the authors of Home Schooling Works!.
The published version of Rudner’s study is entitled “Scholastic Achievement and Demographic Characteristics of Home School Students in 1998.” It appeared in the peer-reviewed online journal Education Policy Analysis Archives in 1999. This section summarizes the major claims made by Rudner.
Family income also had a significant effect (p<.01) on Rudner’s participants’ scores—students with richer parents scored better—and there was a significant interaction (p<.01) between family income and grade, indicating that a higher family income made more of a difference in participants’ academic achievement the older they got.
There was a significant correlation (p<.01) between expenditure per child and family income, indicating that families with higher incomes habitually spend more money per child, and that both of these variables influence homeschoolers’ academic achievement.
There are two main ideas to keep in mind as we analyze Rudner’s study: 1) Rudner’s participants are not representative of homeschoolers; and 2) Rudner’s participants are not representative of the national population of school-age children. There are also a few minor methodological problems with the study, which will be discussed at the end of this section.
Rudner states in his introduction that there were between 700,000 and 1.2 million homeschoolers in the US at the time his article was published, and he cites Lines’ (1998) claim that the numbers were growing. Lines (1998) uses state registration numbers to estimate the population of homeschoolers in 1995-1996. Supposing that the growth rate continued, there would have been approximately 909,000 American homeschoolers in the 1997-1998 school year, when Rudner collected his data. That means that Rudner’s 20,760 participants composed approximately 2.2% of the nation’s homeschoolers.
Going state-by-state, we see that in the majority of states (37 of 54, including territories), less than 5% of homeschoolers participated in Rudner’s study. Even in Virginia, home of HSLDA, only about 15% of the state’s homeschoolers participated. (Note that Rudner’s statistical analyses corrected for differences in number of respondents per state.)
Figure 1: Percentage of the state’s homeschoolers participating in Rudner’s study
Ordinarily, it does not pose a problem for survey research that only a small percentage of the population can be surveyed. However, this methodology relies on the random choice of participants. Rudner’s participants are definitively not random, a fact he takes pains to point out.
Research has also consistently shown that parents’ educational attainment affects their children’s academic success. Nationwide, slightly less than half (47%) of all homeschoolers’ parents had at least a Bachelor’s degree in 1999, while in Rudner’s sample, 61% had a BA or higher. Rudner’s sampled homeschoolers had much more highly educated parents than the average homeschooler.
Rudner (1999) states that “Home school parents are, by definition, heavily involved in their children’s education; the same, unfortunately, is not true of all public or private school parents.” In the transcript from the opening remarks of the press conference held by Rudner and Michael Farris (President of HSLDA) to announce the results of the study, Rudner and Farris elaborated on this idea.
Rudner: I think the real lesson from this report is that parent involvement really affects education. It’s consistent with all the literature. I’m viewing home schooling as the pinnacle of parent involvement.
Farris: Dr. Rudner, I certainly agree with that analysis. The parents who do choose to home school— we simply say their success is demonstrated, regardless of whether they’re a certified teacher or not, across all the various significant barriers that have been thrown at us legally over the years. Those really don’t serve as a criterion for determining who succeeds and who doesn’t succeed. All kinds of families with all kinds of backgrounds successfully home school their children. We think they should have the right.
While Rudner’s study does prove that “families with all kinds of backgrounds successfully homeschool their children”, as Farris claims, it does NOT prove that “home school parents are, by definition, heavily involved in their children’s education.” The many homeschoolers that were overlooked or discounted in Rudner’s study—high schoolers, unschoolers, children of color, poor children, children with poorly educated parents, children being raised by single parents or by parents who both work, abused or educationally neglected children, disabled or special needs children (who composed at least 8% of homeschoolers in 1999)—may not have heavily involved parents, or may have parents who are less successful at homeschooling. We essentially know nothing about these children, who could conceivably compose something like 85% of all homeschoolers. As Farris put it in the online Q&A that took place after the press conference (transcript here), his particularly successful form of homeschooling “takes two parents. [It] takes a commitment for one parent to leave a full time job…Racial minorities may not be able to make it when one parent foregoes an income [because] there are greater percentages of those from racial minorities whose family incomes fall below the median.”
If Rudner’s participants were not representative of homeschoolers, they were even less representative of the nation’s children as a whole.
These differences indicate that it is impossible to compare Rudner’s participants with national averages without controlling for all of these variables.
Finally, there are a few minor methodological issues with the study which do not greatly affect its results, but which nonetheless are important to consider.
Rudner’s findings indicate that homeschooled white elementary- and middle-school-age Christian children from rich, single-income families with married, college-educated parents who administer standardized tests to their children score considerably above the national average on standardized tests. This is not a very exciting result, since we might expect the same behavior from public- and private-schooled children who fulfill all these criteria. Furthermore, this population is approximately only 16% of the homeschooled population nationwide—the levels of academic achievement for the national population are completely unknown.
This study also indicates that homeschoolers tend to score best on reading and worst on math. Lifelong homeschoolers score better than children only homeschooled for a few years (although causation is not necessarily implied here). High family income/large amount of money spent per child on education, high parental education, and low amount of television viewing significantly increased the scores of Rudner’s participants. Gender, prepackaged curriculum, and a certified teacher parent did not make a significant impact on homeschoolers’ scores.
Michael Farris, who commissioned Dr. Rudner to conduct this study, offered the following conclusion during his press conference with Rudner:
“[W]hat we are saying, what we believe the Dr. Rudner study supports, is that home schooling works. And it works well enough for this to be a valid and viable option among those options that parents should have to choose from. That legally, we have satisfied the legitimate interest of any government in wanting to make sure that students have an opportunity to learn. And we think that this report clearly demonstrates that in a way that is significant, a way that is positive. We have upbeat, positive educational results. We think we should have the continued freedom to be able to home school our children.”
Farris is wrong in his conclusions. The Rudner study does not demonstrate that homeschooling is always effective and successful. It merely finds that homeschooling can be, particularly for rich children with highly-educated parents. The Rudner study does not draw ANY conclusions about the vast majority of homeschoolers, who are not represented in its sample. The “legitimate interest of [our] government in wanting to make sure [these] students have an opportunity to learn” has not been satisfied in any way.
Anonymous. 1999. “Home Schooling Works!” Home School Legal Defense Association. http://www.hslda.org/docs/study/rudner1999/
Anonymous. 2001, March 5. “State Laws.” Home School Legal Defense Association. Retrieved from the Internet Archive Wayback Machine. http://web.archive.org/web/20010305023159/http:/hslda.org/laws/default.asp
Anonymous. 2006, December 5. “Contact Us.” Bob Jones University Press Testing and Evaluation. Retrieved from the Internet Archive Wayback Machine. http://web.archive.org/web/20061205203013/http://www.bjupress.com/services/testing/brochures_catalogs/index.html
Anonymous. 2013, November 19. “How poverty impacts students’ test scores, in 4 graphs.” Huffington Post. http://www.huffingtonpost.com/2013/11/19/poverty-test-scores_n_4298345.html
Bielick, Stacey, Kathryn Chandler, and Stephen P. Broughman. 2001, July. Homeschooling in the United States: 1999 (NCES 2001-033). Statistical Analysis Report. National Household Education Surveys Program. Washington, DC: National Center for Education Statistics, U.S. Department of Education, Office of Educational Research and Development. http://nces.ed.gov/pubs2001/2001033.pdf
Bluedorn, Nathaniel. 2002. “Short List of Fallacies.” Fallacy Detective. Christian Logic. http://www.christianlogic.com/articles/read/short-list-of-fallacies/
Davis-Kean, Pamela E. 2005. “The influence of parent education and family income on child achievement: The indirect role of parental expectations and the home environment.” Journal of Family Psychology 19:2. 294-304. http://www.mikemcmahon.info/ParentEducationIncome.pdf
Dubow, Eric F., Paul Boxer, and L. Rowell Huesmann. 2009. “Long-term effects of parents’ education on children’s educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations.” Merrill Palmer Quarterly 55:3. 224–249. http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2853053/pdf/nihms140890.pdf
Farris, Michael and Lawrence Rudner. 1999. “Home Schooling Works—Pass it on!: Online press conference—March 23, 1999, 12:00pm EST: A transcript of the opening remarks by Michael Farris, Esq. & Lawrence M. Rudner, Ph.D.” Home School Legal Defense Association. http://www.hslda.org/docs/study/rudner1999/HSWorksAudioTranscript.pdf
Farris, Michael, Lawrence Rudner, and others. 1999. “Home Schooling Works—Pass it on!: Online press conference—March 23, 1999, 12:00pm EST: Transcript of chat session with media.” Home School Legal Defense Association. http://www.hslda.org/docs/study/rudner1999/ChatLog.pdf
Glanville, Jennifer L., David Sikkink, and Edwin I. Hernández. 2008. “Religious involvement and educational outcomes: The role of social capital and extracurricular participation.” The Sociological Quarterly 49. 105-137. http://onlinelibrary.wiley.com.ezproxy.lib.indiana.edu/doi/10.1111/j.1533-8525.2007.00108.x/pdf
Kaseman, Larry and Susan Kaseman. 1991, January-February. “Does homeschooling research help homeschooling.” Home Education Magazine. http://www.homeedmag.com/INF/FREE/free_rsrch.html
Kaseman, Larry and Susan Kaseman. 1999, July-August. “HSLDA Study: Embarrassing and dangerous.” Taking Charge. Home Education Magazine. http://homeedmag.com/HEM/164/ja_clmn_tch.php
Kunzman, Robert and Milton Gaither. 2013. “Homeschooling: A comprehensive survey of the research.” Other Education: The Journal of Educational Alternatives 2:1. 4-59. http://www.othereducation.stir.ac.uk/index.php/OE/article/view/10/55
Lines, Patricia M. 1999. “Homeschoolers: Estimating numbers and growth.” Washington, DC: National Institute on Student Achievement, Curriculum, and Assessment; Office of Educational Research and Improvement; US Department of Education. http://files.eric.ed.gov/fulltext/ED456167.pdf
Princiotta, Daniel, Stacey Bielick, Chris Chapman, and Carol Rohr. 2004, July. 1.1 million homeschooled students in the United States (NCES 2004-115). Issue Brief. Washington, DC: National Center for Education Statistics, U.S. Department of Education, Institute of Education Sciences. http://nces.ed.gov/pubs2004/2004115.pdf
Rampey, Bobby D., Gloria S. Dion, and Patricia L. Donahue. 2009. NAEP 2008 Trends in Academic Progress (NCES 2009–479). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington, D.C. http://nces.ed.gov/nationsreportcard/pdf/main2008/2009479.pdf
Ratcliffe, Caroline and Signe-Mary McKernan. 2012. “Child poverty and its lasting consequence.” Low-Income Working Families, Paper 21. The Urban Institute. http://www.urban.org/UploadedPDF/412659-Child-Poverty-and-Its-Lasting-Consequence-Paper.pdf
Reich, Rob. 2011, January 5. “Home Schooling.” Rob Reich. http://www.stanford.edu/group/reichresearch/cgi-bin/site/home-schooling/
Rudner, Lawrence M. 1999. “Scholastic Achievement and Demographic Characteristics of Home School Students in 1998.” Education Policy Analysis Archives 7:8.
Tintle, Nathan, Beth Chance, George Cobb, Allan Rossman, Soma Roy, Todd Swanson, and Jill VanderStoep. 2010. “Chapter 1: Introduction to statistical inference: One proportion.” Introduction to Statistical Investigations: Preliminary Edition. http://math.hope.edu/aasi/Chap1_11_29_10.pdf
Welner, Kariane Mari and Kevin G. Welner. 1999. “Contextualizing homeschooling data: A response to Rudner.” Education Policy Analysis Archives 7:13. http://epaa.asu.edu/ojs/article/view/548/671
Wuthnow, Robert. 2003. Arts and Religion Survey 1999 [codebook file]. Princeton, NJ: Cultural Policy and the Arts National Data Archive. http://www.cpanda.org/cpanda/studies/a00082#fileDscr
 I calculated the average number of registered homeschoolers per state in the 1995-1996 school year based on Lines’ state-reported registration data (the average was 7381.75) and used this number to estimate the number of homeschoolers in the 21 states for which Lines did not report data (either because it was not collected or was unavailable). I found that the estimated total number of homeschoolers in the US was 377,056. Lines (1998) hypothesizes that the nationwide homeschooling population may have tripled since the 1990-1991 school year; I used this figure to estimate the homeschooling population in 1997-1998 as 909,046 using the population function . Note that this may be an overestimate; NCES estimated the number of homeschoolers in the spring of 1999 to be about 850,000. However, I choose to use Lines’ (1998) figures as these were used and cited by Rudner.
 I found these percentages by dividing Rudner’s number of participants per state (1997-1998) by Lines’ total number of registered homeschoolers per state (1995-1996). In the 21 states where no registration data was available I used the national average number of homeschoolers per state. Note that these percentages would actually be much lower if the 1997-1998 population numbers were used.
 Children only homeschooled for a few years are more likely to be “pragmatic” homeschoolers, who may leave public school due to a disability or other special needs, a behavioral problem, bullying, belonging to a military family or one that requires travel, etc. These children may return to public school a few years later when the issue resolves.
Dr. Chelsea McCracken
Coalition for Responsible Home Education
661 Washington Street #563
Canton, MA 02021