Correcting the Record (Rudner 1999)

Click here to view this document as a pdf.

One of the sources most commonly cited to support claims that homeschoolers outperform public schoolers academically is Home Schooling Works!, a 1999 web summary of the findings of a study conducted by Lawrence M. Rudner in 1998. The study was designed by Michael Farris, the president of HSLDA, who hired Rudner (a reputable education researcher) to carry it out. Rudner’s findings are presented on HSLDA’s website in a highly deceptive summary (compiled by unidentified authors) which omits much discussion of the study’s methodology and Rudner’s conclusions, adding instead an op-ed by Michael Farris which purports to explain the study’s findings. Rudner’s actual paper, which was published in the peer-reviewed journal Education Policy Analysis Archives in 1999, is linked to by HSLDA and is available here.

Rudner’s published study does not claim that homeschoolers outperform public schoolers; in fact, Rudner clearly cautions against drawing this conclusion in the discussion of his results, which is omitted from HSLDA’s web version. Instead, what his study actually shows is that a small, highly privileged set of homeschoolers are capable of scoring highly on standardized tests. Their scores cannot be compared with public schoolers’ scores, nor are they representative of homeschoolers as a whole. In effect, the study answers the question “Is it possible for homeschooled children to score as well as other children on standardized tests?” with “Yes, it is possible”—Rudner makes no statement on how likely this is. (Compare a hypothetical research study which answered the question “Is it possible to become a millionaire?” with “Yes, it is possible”—would you then conclude that everyone is a millionaire?) The problems with using Rudner’s study to draw these conclusions have been pointed out by a number of critics from both within and outside the homeschooling movement.

In the sections that follow, I will first outline the major points made by Home Schooling Works! and those made in Rudner’s published article. Then I will provide a critical analysis of Rudner’s study, and finally I will summarize what Rudner’s results actually mean.

[expand title=”Home Schooling Works! major points”]

Home Schooling Works! is an online narrative summary, divided into several pages, of Rudner’s 1999 study. It is posted on HSLDA’s website without attributions for its authors, except for the portion which contains an op-ed by Michael Farris. It is unknown to what degree Rudner participated in condensing his study’s findings for a web audience. This section will summarize the claims made by the authors of Home Schooling Works!.

• The purpose of Rudner’s study was to answer the research questions “How can ordinary parents hope to do as well as the educational experts?” and “Who are these families? What drives them to choose such an unusual lifestyle?” More clearly stated, the main purpose of the study was to prove that it is possible for homeschoolers to score comparably to public schoolers on standardized tests, while secondary purposes included proving that regulations on homeschoolers are unnecessary and collecting demographic data about homeschoolers.
• The study was conducted in 1998 by Dr. Lawrence Rudner and commissioned by HSLDA. “The data were compiled from the achievement test scores of 20,760 students in 11,930 families, along with background questionnaires submitted by the families.”
• This study differed from previous studies in that the students involved all took the same tests (ITBS for grades K-8 and TAP for grades 9-12) and parents did not know children’s scores before agreeing to participate.
• The sample of homeschoolers used for the study was not representative of the movement; however, the authors of the web summary fallaciously attribute this weakness to the diversity of homeschooling. They claim that “[h]ome schooling thrives on individuality, and the home education movement daily becomes more diverse. These factors make it difficult to determine whether achievement test scores are truly the best demonstration of home school achievement, and whether the families involved in the Rudner study are an exact representation of the movement as a whole.” In fact, the participants in Rudner’s study were not at all representative of the movement as a whole, as will be explained below.
• “In every subject and at every grade level…, [Rudner’s participants] scored significantly higher than their public and private school counterparts.” The participating homeschoolers’ score percentiles in this study were, averaging across all grades, approximately 86th in reading, 77th in language, and 75th in math (compare to the national average, which is defined as the 50th percentile).
• 25.4% of Rudner’s participants were one grade or more above grade level for their age.
• Homeschooled participants in Rudner’s study started out ahead of their public-schooled peers in grade level and outpaced them in advancement, reaching approximately 4 grade levels ahead by 8th grade.
• Participants who were homeschooled K-12 scored higher than those who were only partially homeschooled.
• Participants’ scores differed significantly based on “amount of money spent on education, family income, parent education, and television viewing.” Nonetheless, even the lower-scoring groups under these variables scored higher than the national average.
• Gender and whether either parent had been certified to teach were not found to significantly affect participants’ scores.
• Summary of the demographic findings: 88% of participants’ parents had continued their education beyond high school, while only 50% of people nationwide did so. 24% of participants had at least one parent holding a teaching certificate. Participants’ median income was $52,000 while nationally families with children had a median income of $36,000. Around 62% of participant homeschooler families had three or more children, while around 80% of the general population had two or fewer. Around 97% of participant families had married parents (vs. 72% of families nationwide). 77% of participants’ mothers did not work, while this figure is 30% for US mothers in general. (Note: the figures for these comparisons are drawn from the online summary of Rudner’s findings and have not been independently verified.)
• The study undercounted high schoolers, who made up only 11.4% of the sample where nationwide they compose 30.3% of students. The author speculates on possible reasons that the sample was younger than the national average—e.g. because more high schoolers want to be in public school, or more homeschoolers graduate early, etc. He or she does not mention the possibility that the sample may have been skewed.
• Only 6% of the participants were racial/ethnic minorities, as opposed to 32.8% nationally. At least 60% of respondents identified with a conservative religious tradition (25.1% Independent Fundamental, 24.4% Baptist, 8.2% Independent Charismatic, 4.1% Assembly of God); the author identifies the religious affiliations of the testing service as a probable reason for this, but claims that it is not a problem because “there is no known reason to believe that secular home school students perform at a lower academic level than religious home schoolers.” In fact, the differences in academic performance between religious and secular homeschoolers are unknown, and some evidence suggests that religious homeschoolers may academically outperform secular homeschoolers.
• 1.6% of homeschooled 4th graders who participated in the study watch more than three hours of TV per day, while 40% of public schoolers do. The median expenditure per child for homeschooling families in the study was $400.
• This demographic data is followed by a section entitled “Why are homeschoolers succeeding?” written by Michael Farris. Though Farris does not directly claim that homeschooling is better than public schooling, he does his best to imply this idea by saying, “It is clear that the average home school child performs significantly higher than the average public school child… Without needing to criticize other forms of education, we can confidently assure policy makers that this system delivers solid academic results.” In fact, it is not clear from this study that the average homeschooler performs better than the average public schooler, as I will discuss below.
• Farris also encourages his audience to disregard more cautious interpretations of the data based on social science methodologies. “[Though we cannot say that] this study proves the superiority of home schooling over other educational choices,” Farris argues, “[the evidence for this finding] lies not in statistics, but rather in individual stories.” This anti-science view—which employs the logical fallacy that anecdotal experiences are representative of larger trends—is fairly common in the homeschooling community.
• Farris acknowledges that his sample is not representative of the nationwide population of children, noting that “Home school students typically come from families where income is relatively high, marriages are intact, and dedication to education is strong.” He also rightly observes that such demographic observations fail “to consider one factor that clearly contributes to home education’s success—parental motivation.” However, he then overreaches, incorrectly stating that “Because home schooling, by its very nature, is practiced only in families where parents are strongly motivated, it is undetermined what would happen if unmotivated parents were randomly assigned to home educate their children.” While it is true that the results of homeschooling by unmotivated parents are currently unknown, this is due to a lack of research, not due to a lack of unmotivated homeschooling parents on whom to do research. Unmotivated homeschooling parents did not participate in this study; therefore it is impossible to draw any conclusions about them at all.
• Since the study found no statistical relationship between participants’ scores and whether their parents were certified teachers, and since it found higher scores for participants who had been homeschooled for more years, Farris argues that regulations of homeschooling are unnecessary. As I will show, this is a faulty conclusion.
• Finally, at the very end of the summary, we find a section on methodology (entitled “About the study”). It describes how the participants were chosen: a questionnaire entitled “Voluntary Home School Demographic Survey” was sent to the 39,607 homeschoolers whose parents signed them up to take an achievement test with Bob Jones University Press Testing and Evaluation Service, an evangelical Christian testing service which caters to homeschoolers. BJUP-certified test administrators, often the children’s parents, administered the tests and submitted the completed tests and questionnaires to BJUP. “The tests were then bundled and sent to Riverside Publishers for machine scoring, and the questionnaires were bundled and sent to National Computer Systems for scanning.” Paired sets of test results and questionnaires were sent to Dr. Rudner to analyze. The total number of usable paired sets represented 20,760 students in 11,930 families—approximately 3000 respondents were excluded (presumably based on missing data, although the summary does not say).[/expand]

[expand title=”Rudner’s (1999) major points”]

The published version of Rudner’s study is entitled “Scholastic Achievement and Demographic Characteristics of Home School Students in 1998.” It appeared in the peer-reviewed online journal Education Policy Analysis Archives in 1999. This section summarizes the major claims made by Rudner.

• The purpose of this study is to describe “the academic achievement levels and some basic demographic characteristics of a large sample of students and their families;” that is, to answer the questions “Are the achievement levels of home school students comparable to those of public school students?” and “How does the home school population differ from the general United States population?” Rudner states very clearly in his introduction that “this study is not a comparison of home schools with public or private schools” [emphasis added], since homeschoolers are not representative of the general US population. This means that Rudner’s study does not show that homeschoolers score better than public schoolers. It merely assembles the scores of a particular group of homeschoolers, who are not representative of any larger population.
• Rudner’s methodology is described in detail. In Spring 1998, the parents of 39,607 home school students contracted with Bob Jones University Press Testing and Evaluation Service to have their children tested with the Iowa Tests of Basic Skills (for grades K-8) or the corresponding Tests of Achievement and Proficiency (for grades 9-12). An achievement test was administered to participants by a BJU-certified test administrator, who could be the child’s parent if he/she 1) held a Bachelor’s degree; or 2) held a teaching certificate; or 3) had been a full-time teacher in a conventional classroom.
• “[P]arents were asked to complete a questionnaire entitled “Voluntary Home School Demographic Survey.”” The background questionnaire was designed by HSLDA and does not appear to have been vetted through pilot studies or any other method. Where possible, it used the wording of questions from the US Census and other national surveys. Parents did not know their children’s scores before agreeing to participate in the study.
• “The achievement test and questionnaire results were combined to form the dataset used in [Rudner’s] analysis.” Achievement tests and questionnaires were computer-scanned. About 3000 questionnaires and 3000 test scores were dropped from the raw dataset due to mismatched identification numbers. A further 140 cases where respondents had not listed their state of residence were excluded. “A total of 20,760 students in 11,930 families provided useable questionnaires with corresponding achievement tests.” Their scores were compared to the standardized percentiles for the ITBS and TAP, which were developed from a sample of approximately 157,000 public and private school students.
• Study participants were not equally distributed by state—Ohio, Virginia, and Georgia each contributed more than 7% of the sample, while 24 states contributed less than 1% each. All subsequent analyses were weighted using the number of public schooled students per state, as the homeschooler numbers were not available.
• Rudner provides a fairly thorough breakdown of the demographics of his sample. They were fairly evenly split by gender and disproportionately young compared to students nationwide (11.4% of Rudner’s participants were in high school as compared with 30.3% of students nationwide). Rudner’s sample was 94% white and the largest racial/ethnic minority groups were American Indians (2.4%) and Asians (1.2%). In terms of religion, Rudner’s sample was overwhelmingly Christian (93.8%), with over 60% of participants’ mothers identifying with a conservative tradition (25.1% Independent Fundamental, 24.4% Baptist, 8.2% Independent Charismatic, and 5.4% Roman Catholic).
• Of the families in Rudner’s sample, 97.2% were headed by a married couple and 62.1% had 3 or more children. The participants’ parents were highly educated—66.2% of the fathers and 56.7% of the mothers had at least a Bachelor’s degree, and 23.6% of the students had at least one parent with a teaching certificate—but only 3.2% of the mothers worked full-time. Homeschooling families in Rudner’s sample had a median income of approximately $52,000 and the median expenditure per child was $400. Only 7.7% of participants were enrolled in a full-service curriculum, and the participants were infrequent users of technology: 65.3% of participants watched less than one hour of TV per day, and an average of 35.3% reported never using a computer.
• Rudner found that 24.5% of homeschoolers in his study were enrolled at least one grade level higher than their age would suggest. In his analysis, he compared the scores of all students who were enrolled in the same grade, an approach which produced more conservative results than if he had compared students who were all the same age. For example, he considered children whose age-mates were second-graders but whose parents listed their grade-level as fourth grade to be fourth-graders for the purposes of the study. If he had made the opposite choice, the average score for homeschoolers would be higher than it was.
• Scores on the ITBS/TAP were compared in terms of both national percentile (where the public school average was defined as 50%) and Developmental Standard Score (where a student’s score is correlated with his or her placement on a scale of grade level). For instance, Rudner found that the average homeschooled 8th grader in his study scored around the 80th percentile across all subject areas and had a DSS of 12.8; that is, he or she could read 8th grade material as well as the average public schooled 12th grader could read 8th grade material.
• Across all grades and all subject areas of the test, homeschoolers in Rudner’s sample scored between the 60th and the 90th percentiles. Their highest average score was 85.7% in reading and their lowest average score was 75.1% in math. Rudner’s participants also scored consistently higher than private schoolers, whose highest average score was 76.3% in reading and whose lowest average score was 65.4% in language.
• As shown below, homeschoolers in Rudner’s sample were consistently ahead of the national average in grade level (as measured by DSS) from grades 1-8. (DSS scores are not meaningful for high school.) Homeschooled first graders in Rudner’s sample are approximately one grade level ahead of their public schooled peers, and they outpace public schooled children in academic advancement: Rudner’s homeschooled 8th graders are approximately four grade levels ahead of their public schooled peers.

• Rudner then proceeds to analyze homeschooled participants’ scores with respect to various independent variables. For these analyses, he randomly chose one child from each household, and was thus able to only draw conclusions for grades 1-10; there were not enough participants in grades 11 and 12 to get accurate data.
• 47% of Rudner’s participants had been homeschooled for their entire lives. These participants scored significantly higher than those who had only been homeschooled for some grades (p<.01). There was also a significant interaction (p<.01) between grade and whether participants had been homeschooled for their entire lives; that is, as Rudner’s lifelong homeschoolers got older, they outpaced other homeschooled participants in academic achievement.

• Rudner’s participants whose parents spent more than $600 per child per year scored significantly better (p<.01) than participants whose parents spent less than $200. There was also a significant interaction (p<.01) between grade and expenditure per child; that is, greater expenditure per child made more of a difference for older children’s academic achievement.

Family income also had a significant effect (p<.01) on Rudner’s participants’ scores—students with richer parents scored better—and there was a significant interaction (p<.01) between family income and grade, indicating that a higher family income made more of a difference in participants’ academic achievement the older they got.

There was a significant correlation (p<.01) between expenditure per child and family income, indicating that families with higher incomes habitually spend more money per child, and that both of these variables influence homeschoolers’ academic achievement.

• Parental education level had a significant effect on participants’ scores. In Rudner’s sample, children of college graduates scored significantly higher (p<.01) than children whose parents were not college graduates. There was also a significant interaction (p<.01) between parental education and grade, meaning that having educated parents was more important for the academic achievement of older children. However, even Rudner’s participants whose parents did not have college degrees still scored better than the national average.

• The amount of time Rudner’s participants spent watching television had a significant negative effect (p<.01) on their scores, meaning students who watched more TV scored worse. There was also a significant interaction (p<.01) between time spent watching television and grade; that is, television was more detrimental to older homeschooled participants’ academic achievement.

• There was no significant difference between the scores of female and male homeschooled participants. There was no significant difference between the scores of those participants who were enrolled in a full-service curriculum and those who were not. There was no significant difference between homeschooled participants with at least one parent certified as a teacher and participants without (controlling for grade and parental education level).
• Rudner (1999) concludes with a summary of his results and a discussion of the limitations of his study. He places his findings in the context of other research on homeschooling; he cites three other surveys of homeschoolers which found similar demographic and achievement data.
• Rudner very clearly states that the sample of homeschoolers he surveyed was not representative of the national population of school-age children. “Home school students and their families are a select population,” he states—a population with high family incomes, high parental education levels, and a traditional family structure, often with one or more parents holding a teaching certificate. Furthermore, homeschoolers “are not a cross-section of the United States population. The act of home schooling distinguishes this group in terms of their exceptionally strong commitment to education and children…It should be noted that it was not possible within the parameters of this study to evaluate whether this sample is truly representative of the entire population of home school students.” (emphasis added) He goes on to add that “the reported achievement differences…cannot be attributed to the type of school a child attends. This study does not demonstrate that home schooling is superior to public or private schools…It does not indicate that children will perform better academically if they are home schooled.” (emphasis added) Any claims to the contrary by HSLDA are not supported by Rudner.[/expand]

[expand title=”Critical analysis”]

There are two main ideas to keep in mind as we analyze Rudner’s study: 1) Rudner’s participants are not representative of homeschoolers; and 2) Rudner’s participants are not representative of the national population of school-age children. There are also a few minor methodological problems with the study, which will be discussed at the end of this section.

1) Rudner’s participants are not representative of homeschoolers

Rudner states in his introduction that there were between 700,000 and 1.2 million homeschoolers in the US at the time his article was published, and he cites Lines’ (1998) claim that the numbers were growing. Lines (1998) uses state registration numbers to estimate the population of homeschoolers in 1995-1996. Supposing that the growth rate continued,[1] there would have been approximately 909,000 American homeschoolers in the 1997-1998 school year, when Rudner collected his data. That means that Rudner’s 20,760 participants composed approximately 2.2% of the nation’s homeschoolers.

Going state-by-state,[2] we see that in the majority of states (37 of 54, including territories), less than 5% of homeschoolers participated in Rudner’s study. Even in Virginia, home of HSLDA, only about 15% of the state’s homeschoolers participated. (Note that Rudner’s statistical analyses corrected for differences in number of respondents per state.)

Figure 1: Percentage of the state’s homeschoolers participating in Rudner’s study

Ordinarily, it does not pose a problem for survey research that only a small percentage of the population can be surveyed. However, this methodology relies on the random choice of participants. Rudner’s participants are definitively not random, a fact he takes pains to point out.

• The parents of Rudner’s 20,760 participants had all signed their children up, without any encouragement from the researcher, to take a standardized test from Bob Jones University Press Testing and Evaluation Service. The total number of students signed up to take the ITBS/TAP with BJUP that year was 39,607. That is, only 4% of all homeschoolers in the US had signed up to use this testing service. BJUP is, as Rudner states, “the largest and oldest of four organizations providing home school families access to standardized achievement tests.” If these other three organizations had as many customers as BJUP, that would still mean only 16% of homeschoolers nationwide used a major testing service to take a standardized test that year. It is likely that the homeschoolers who took standardized tests in 1999 were primarily those children that their parents expected to do well. The top-scoring 16% of homeschoolers in the nation would, by definition, score at or above the 84th percentile on standardized tests; unsurprisingly, Rudner’s homeschoolers score around the 80th percentile.
• Moreover, we might hypothesize that most of the families using the evangelical BJUP are Christian, and in fact Rudner’s participants were 93.8% Christian. However, only 38.4% of homeschoolers nationwide in 1999 were motivated by religious reasons. This statistic speaks to a strong oversampling of children from Christian families, which is relevant because religious involvement has been shown to be correlated with better academic outcomes. Furthermore, there are two main schools of thought in homeschooling: the “ideologues” homeschool for religious or moral reasons, while the “pedagogues” homeschool because they believe that it is the most effective educational technique. Homeschoolers whose parents have such differing views of homeschooling might have different levels of academic achievement. Only sampling the “ideologues”—the Christians—leaves out the many secular homeschoolers and unschoolers, who might differ in some way in their level of academic achievement.
• Rudner’s participants differed demographically from the national population of homeschoolers in terms of age, race/ethnicity, family structure, and socioeconomic class. However, Rudner was not aware of these differences since the NCES data report was not released until after his study was.
• Rudner overcounted elementary schoolers and severely undercounted high schoolers in his survey. In 1999, 44.3% of US homeschoolers were in elementary school (compare to Rudner’s 58%) and 31% of US homeschoolers were in high school (compare to Rudner’s 11%).
  Rudner’s explanation for why this might be is unsatisfying. He says, “Possible reasons for this lower participation for high school students may be the relative newness of the home school movement, early graduation from high school, and possibly a desire on the part of some home school parents to enroll their children in a traditional high school.” A variety of other possibilities present themselves: that BJUP markets themselves more to younger children; that parents of high schoolers prefer them to take other standardized tests (like the SAT or ACT); or that older children earn fewer accolades for performing above grade level. Especially due to the fact that Rudner found a number of significant interactions between test score and enrolled grade, the lack of sufficient high school-age participants in his study is problematic.

• Rudner’s sample of the homeschooling population was not representative in terms of race, as shown in the graphs below. Nationwide in 1999, 75% of homeschoolers were white, 10% were black, 9% were Hispanic, and 6% were “Other”. In Rudner’s sample, 94% of participants were white, .8% were black, .2% were Hispanic, and 5% were “Other”. This is relevant because, due to structural inequalities in education and other aspects of society, black and Latino children tend to score approximately 5% lower than white children on standardized tests. By excluding these populations from his sample, Rudner found artificially high scores for the homeschoolers he surveyed.

• Rudner’s sample was not representative of the homeschooling population in terms of family structure. 97.2% of Rudner’s participants had married parents; however, nationwide, about 20% of homeschoolers were being raised by a single parent or by nonparental guardians. Furthermore, in 76.9% of Rudner’s families, homeschooling mothers did not work for pay; nationwide, only 52% of homeschoolers lived in two-parent families where only one parent worked.
• Parental education and family income also differentiate Rudner’s sample from the national population of homeschoolers. Nationwide in 1999, 33.4% of homeschoolers were living in families with incomes below $25,000; Rudner’s sample—where homeschoolers in the lowest income bracket composed only 8.4%—dramatically undercounted them. Rudner also overcounted upper-middle class children, who composed 32.5% of his sample but only 17% of all homeschoolers nationwide. There is a great deal of evidence that poverty negatively impacts students’ scores on standardized tests.

Research has also consistently shown that parents’ educational attainment affects their children’s academic success. Nationwide, slightly less than half (47%) of all homeschoolers’ parents had at least a Bachelor’s degree in 1999, while in Rudner’s sample, 61% had a BA or higher. Rudner’s sampled homeschoolers had much more highly educated parents than the average homeschooler.

• There are most likely a number of other ways that Rudner’s sample was not representative of all homeschoolers—for instance, he probably overcounted lifelong homeschoolers and homeschoolers with teaching-certified parents—but this cannot be determined because nationwide statistics are not available for these variables.

Rudner (1999) states that “Home school parents are, by definition, heavily involved in their children’s education; the same, unfortunately, is not true of all public or private school parents.” In the transcript from the opening remarks of the press conference held by Rudner and Michael Farris (President of HSLDA) to announce the results of the study, Rudner and Farris elaborated on this idea.

Rudner: I think the real lesson from this report is that parent involvement really affects education. It’s consistent with all the literature. I’m viewing home schooling as the pinnacle of parent involvement.

Farris: Dr. Rudner, I certainly agree with that analysis. The parents who do choose to home school— we simply say their success is demonstrated, regardless of whether they’re a certified teacher or not, across all the various significant barriers that have been thrown at us legally over the years. Those really don’t serve as a criterion for determining who succeeds and who doesn’t succeed. All kinds of families with all kinds of backgrounds successfully home school their children. We think they should have the right.

While Rudner’s study does prove that “families with all kinds of backgrounds successfully homeschool their children”, as Farris claims, it does NOT prove that “home school parents are, by definition, heavily involved in their children’s education.” The many homeschoolers that were overlooked or discounted in Rudner’s study—high schoolers, unschoolers, children of color, poor children, children with poorly educated parents, children being raised by single parents or by parents who both work, abused or educationally neglected children, disabled or special needs children (who composed at least 8% of homeschoolers in 1999)—may not have heavily involved parents, or may have parents who are less successful at homeschooling. We essentially know nothing about these children, who could conceivably compose something like 85% of all homeschoolers. As Farris put it in the online Q&A that took place after the press conference (transcript here), his particularly successful form of homeschooling “takes two parents. [It] takes a commitment for one parent to leave a full time job…Racial minorities may not be able to make it when one parent foregoes an income [because] there are greater percentages of those from racial minorities whose family incomes fall below the median.”

2) Rudner’s participants are not representative of schoolchildren nationwide

If Rudner’s participants were not representative of homeschoolers, they were even less representative of the nation’s children as a whole.

• The homeschoolers in the sample were disproportionately young compared to students nationwide; 11.4% of Rudner’s participants were in high school as compared with 30.3% of students nationwide.
• Rudner’s sample was 94% white, where 67.2% of Americans were white in 1996.
• 97.2% of the families in Rudner’s sample were headed by a married couple, compared to 72% of families with a public-schooled child nationwide in 1997.
• Homeschoolers in Rudner’s study had larger families than schoolchildren nationwide; 79.6% of families with school-age children nationwide had 1 or 2 children, while 62.1% of the families Rudner surveyed had 3 or more children.
• In terms of religion, Rudner’s sample was overwhelmingly Christian (93.8%), with over 60% of participants’ mothers identifying with a conservative tradition (25.1% Independent Fundamental, 24.4% Baptist, 8.2% Independent Charismatic, and 5.4% Roman Catholic). One national survey found that the US population in 1999 was only 82% Christian, 7% other religions, and 11% no religion/unknown.
• The participants’ parents were more educated than the national average: 66.2% of the fathers and 56.7% of the mothers in Rudner’s sample had at least a Bachelor’s degree, while 50% of men and 51.4% of women nationwide in 1996 did not continue their education after high school.
• Homeschooling families in Rudner’s sample had a median income of approximately $52,000 while the median income for families with children in 1995 was $36,000.
• 65.3% of Rudner’s participants watched less than one hour of TV per day, while nationwide 74.9% of children watched 2 or more hours of TV per day.

These differences indicate that it is impossible to compare Rudner’s participants with national averages without controlling for all of these variables.

3) Minor methodological issues

Finally, there are a few minor methodological issues with the study which do not greatly affect its results, but which nonetheless are important to consider.

• Many of the administrators of the ITBS/TAP were the students’ own parents. Though these parents were required to submit signed affidavits stating that they had not interfered with their children’s test environment in any illicit way, still this situation was rife for abuse. However, Michael Farris stated in his press release Q&A that “The controls placed by the testing company preclude[d] parents from giving their own children the tests unless the parents were monitoring a larger group of students which included their own child.”
• Rudner states that “Bob Jones University Press Testing and Evaluation Service provides assessment services to home school students and private schools on a fee-for-service basis.” In 2006 the price for this test was $40; if we assume this was also the price in 1999, Rudner’s participants would have had to pay approximately $56 in 2013 dollars for the test (due to inflation). This comparatively high price may have placed such standardized testing out of the reach of impoverished families or those with many children.
• Rudner suggests that the overrepresentation of Ohio, Georgia, and Virginia may have been “due to the fact that these states require testing of home school students.” The best available data says that in 2001, standardized testing was mandatory for homeschooled students in Arkansas, Georgia, Hawaii, Minnesota, North Carolina, New Mexico, New York, Oregon, and South Dakota. It is unclear why these states were not also overrepresented.[/expand]

[expand title=”What the study actually says”]

Rudner’s findings indicate that homeschooled white elementary- and middle-school-age Christian children from rich, single-income families with married, college-educated parents who administer standardized tests to their children score considerably above the national average on standardized tests. This is not a very exciting result, since we might expect the same behavior from public- and private-schooled children who fulfill all these criteria. Furthermore, this population is approximately only 16% of the homeschooled population nationwide—the levels of academic achievement for the national population are completely unknown.

This study also indicates that homeschoolers tend to score best on reading and worst on math. Lifelong homeschoolers score better than children only homeschooled for a few years (although causation is not necessarily implied here[3]). High family income/large amount of money spent per child on education, high parental education, and low amount of television viewing significantly increased the scores of Rudner’s participants. Gender, prepackaged curriculum, and a certified teacher parent did not make a significant impact on homeschoolers’ scores.[/expand]

[expand title=”Conclusion”]

Michael Farris, who commissioned Dr. Rudner to conduct this study, offered the following conclusion during his press conference with Rudner:

“[W]hat we are saying, what we believe the Dr. Rudner study supports, is that home schooling works. And it works well enough for this to be a valid and viable option among those options that parents should have to choose from. That legally, we have satisfied the legitimate interest of any government in wanting to make sure that students have an opportunity to learn. And we think that this report clearly demonstrates that in a way that is significant, a way that is positive. We have upbeat, positive educational results. We think we should have the continued freedom to be able to home school our children.”

Farris is wrong in his conclusions. The Rudner study does not demonstrate that homeschooling is always effective and successful. It merely finds that homeschooling can be, particularly for rich children with highly-educated parents. The Rudner study does not draw ANY conclusions about the vast majority of homeschoolers, who are not represented in its sample. The “legitimate interest of [our] government in wanting to make sure [these] students have an opportunity to learn” has not been satisfied in any way.[/expand]

[expand title=”References”]

Anonymous. 1999. “Home Schooling Works!” Home School Legal Defense Association. http://www.hslda.org/docs/study/rudner1999/

Anonymous. 2001, March 5. “State Laws.” Home School Legal Defense Association. Retrieved from the Internet Archive Wayback Machine. http://web.archive.org/web/20010305023159/http:/hslda.org/laws/default.asp

Anonymous. 2006, December 5. “Contact Us.” Bob Jones University Press Testing and Evaluation. Retrieved from the Internet Archive Wayback Machine. http://web.archive.org/web/20061205203013/http://www.bjupress.com/services/testing/brochures_catalogs/index.html

Anonymous. 2013, November 19. “How poverty impacts students’ test scores, in 4 graphs.” Huffington Post. http://www.huffingtonpost.com/2013/11/19/poverty-test-scores_n_4298345.html

Bielick, Stacey, Kathryn Chandler, and Stephen P. Broughman. 2001, July. Homeschooling in the United States: 1999 (NCES 2001-033). Statistical Analysis Report. National Household Education Surveys Program. Washington, DC: National Center for Education Statistics, U.S. Department of Education, Office of Educational Research and Development. http://nces.ed.gov/pubs2001/2001033.pdf

Bluedorn, Nathaniel. 2002. “Short List of Fallacies.” Fallacy Detective. Christian Logic. http://www.christianlogic.com/articles/read/short-list-of-fallacies/

Davis-Kean, Pamela E. 2005. “The influence of parent education and family income on child achievement: The indirect role of parental expectations and the home environment.” Journal of Family Psychology 19:2. 294-304. http://www.mikemcmahon.info/ParentEducationIncome.pdf

Dubow, Eric F., Paul Boxer, and L. Rowell Huesmann. 2009. “Long-term effects of parents’ education on children’s educational and occupational success: Mediation by family interactions, child aggression, and teenage aspirations.” Merrill Palmer Quarterly 55:3. 224–249.  http://www.ncbi.nlm.nih.gov/pmc/articles/PMC2853053/pdf/nihms140890.pdf

Farris, Michael and Lawrence Rudner. 1999. “Home Schooling Works—Pass it on!: Online press conference—March 23, 1999, 12:00pm EST: A transcript of the opening remarks by Michael Farris, Esq. & Lawrence M. Rudner, Ph.D.” Home School Legal Defense Association. http://www.hslda.org/docs/study/rudner1999/HSWorksAudioTranscript.pdf

Farris, Michael, Lawrence Rudner, and others. 1999. “Home Schooling Works—Pass it on!: Online press conference—March 23, 1999, 12:00pm EST: Transcript of chat session with media.” Home School Legal Defense Association. http://www.hslda.org/docs/study/rudner1999/ChatLog.pdf

Glanville, Jennifer L., David Sikkink, and Edwin I. Hernández. 2008. “Religious involvement and educational outcomes: The role of social capital and extracurricular participation.” The Sociological Quarterly 49. 105-137. http://onlinelibrary.wiley.com.ezproxy.lib.indiana.edu/doi/10.1111/j.1533-8525.2007.00108.x/pdf

Kaseman, Larry and Susan Kaseman. 1991, January-February. “Does homeschooling research help homeschooling.” Home Education Magazine. http://www.homeedmag.com/INF/FREE/free_rsrch.html

Kaseman, Larry and Susan Kaseman. 1999, July-August. “HSLDA Study: Embarrassing and dangerous.” Taking Charge. Home Education Magazine. http://homeedmag.com/HEM/164/ja_clmn_tch.php

Kunzman, Robert and Milton Gaither. 2013. “Homeschooling: A comprehensive survey of the research.” Other Education: The Journal of Educational Alternatives 2:1. 4-59. http://www.othereducation.stir.ac.uk/index.php/OE/article/view/10/55

Lines, Patricia M. 1999. “Homeschoolers: Estimating numbers and growth.” Washington, DC: National Institute on Student Achievement, Curriculum, and Assessment; Office of Educational Research and Improvement; US Department of Education. http://files.eric.ed.gov/fulltext/ED456167.pdf

Princiotta, Daniel, Stacey Bielick, Chris Chapman, and Carol Rohr. 2004, July. 1.1 million homeschooled students in the United States (NCES 2004-115). Issue Brief. Washington, DC: National Center for Education Statistics, U.S. Department of Education, Institute of Education Sciences. http://nces.ed.gov/pubs2004/2004115.pdf

Rampey, Bobby D., Gloria S. Dion, and Patricia L. Donahue. 2009. NAEP 2008 Trends in Academic Progress (NCES 2009–479). National Center for Education Statistics, Institute of Education Sciences, U.S. Department of Education, Washington, D.C. http://nces.ed.gov/nationsreportcard/pdf/main2008/2009479.pdf

Ratcliffe, Caroline and Signe-Mary McKernan. 2012. “Child poverty and its lasting consequence.” Low-Income Working Families, Paper 21. The Urban Institute. http://www.urban.org/UploadedPDF/412659-Child-Poverty-and-Its-Lasting-Consequence-Paper.pdf

Reich, Rob. 2011, January 5. “Home Schooling.” Rob Reich. http://www.stanford.edu/group/reichresearch/cgi-bin/site/home-schooling/

Rudner, Lawrence M. 1999. “Scholastic Achievement and Demographic Characteristics of Home School Students in 1998.” Education Policy Analysis Archives 7:8.

Tintle, Nathan, Beth Chance, George Cobb, Allan Rossman, Soma Roy, Todd Swanson, and Jill VanderStoep. 2010. “Chapter 1: Introduction to statistical inference: One proportion.” Introduction to Statistical Investigations: Preliminary Edition. http://math.hope.edu/aasi/Chap1_11_29_10.pdf

Welner, Kariane Mari and Kevin G. Welner. 1999. “Contextualizing homeschooling data: A response to Rudner.” Education Policy Analysis Archives 7:13. http://epaa.asu.edu/ojs/article/view/548/671

Wuthnow, Robert. 2003. Arts and Religion Survey 1999 [codebook file]. Princeton, NJ: Cultural Policy and the Arts National Data Archive. http://www.cpanda.org/cpanda/studies/a00082#fileDscr[/expand]

[expand title=”Footnotes”]

Dr. Chelsea McCracken
Research Analyst
Coalition for Responsible Home Education
chelseamc@responsiblehomeschooling.org
661 Washington Street #563
Canton, MA 02021
(617) 765-7096