The Cardus Education Survey

The Cardus Education Survey, which was designed to study adult graduates of Christian schools in North America, was conducted between 2007 and 2012 by the Canadian Christian think tank Cardus. Reports on the study were published in 2011 (focusing on the United States) and in 2012 (focusing on Canada). Though the goals of the study were unrelated to homeschooling, best research practices required that incidental data also be collected on homeschool graduates.

The Cardus publications relied on random samples of homeschool graduates whose responses to various surveys were weighted based on the number of respondents and then weighted again for a number of demographic factors. As such, the Cardus survey is one of the only studies of a representative sample of homeschool graduates—and one of the only studies whose results can be applied to the larger population of all homeschoolers. This is discussed by Dr. Milton Gaither, a prominent homeschooling researcher, who provides critical analyses of the Cardus Education Survey on his former blog, Homeschooling Research Notes (Phase I and Phase II).

The major findings of the study relate only to religious homeschoolers (or, as defined in the study, homeschoolers whose mothers frequently attended religious services) in the US and Canada. The researchers found that homeschool graduates were less academically prepared for college and had less higher education than public school graduates; that they had a strict and legalistic moral outlook; and that they reported more feelings of helplessness and a lack of clarity about their life goals. In addition, American religious homeschool graduates reported more divorces and fewer children than public school graduates, as well as a lack of interest in politics and charitable giving; however, these characteristics were not shared by Canadian respondents.

In the sections that follow, I will first give the background of the study and outline the major points made by the reports on Phase I and Phase II. Then I will provide a critical analysis of the study, and finally I will summarize what its results actually mean.

[expand title=”Background of the study”]

Cardus is a Christian-affiliated think tank located in Ontario, Canada; its mission is to conduct independent socially conscious research on sociology, education, and economics. It also publishes four academic journals.

“THE CARDUS EDUCATION SURVEY originated with a symposium on the relationship between education and culture change, held on December 6-7, 2007. That symposium identified various research gaps regarding the state of K-12 Christian education in North America and the lack of reference benchmark data. Cardus developed a research proposal and secured funding from the [religious right-affiliated] Richard and Helen DeVos Foundation based in Grand Rapids, MI; the William Voortman Foundation based in Hamilton, ON [which primarily funds Protestant mission work]; and the Van Lunen Foundation based in Chicago, IL [which trains executive managers for Christian schools]. A research partnership was established with [the Catholic] University of Notre Dame which included an in-kind contribution to the project. The combined value of the funded and in-kind contributions to the project was $1,150,000.” [hyperlinks added]

The major research gap the symposium identified was a lack of knowledge of the outcomes of private Christian schooling in North America. A number of research teams pursued the project for several years; the results were published in two phases: Phase I (Pennings et al. 2011) and Phase II (Pennings et al. 2012a & 2012b).

The results of Phase I, which are available here or may be downloaded for $9.95 from Cardus’s website, were published in 2011 in a report entitled Do the motivations for private religious Catholic and Protestant schooling in North America align with graduate outcomes? A number of scholars were involved in the publication of this report: Ray Pennings, Senior Fellow and Director of Research at Cardus; Dr. John Seel, a Senior Fellow at Cardus; Dr. Deani Van Pelt, liaison with the quantitative research teams; Dr. David Sikkink, who directed the quantitative research; Dr. Kathryn Wiens, who penned the report; and Cardus editors Dan Postma and Kathryn De Ruijter.

Phase I “used a mixed-methods design to better understand the academic, spiritual, and cultural outcomes of Christian education in North America. Over a two-year period, five research teams concur­rently implemented research projects to answer the question of this project: to what extent are the motivations and outcomes of Christian schools aligned in academic, spiritual, and cultural domains?” The quantitative research team, led by Sikkink, surveyed a random sample of adult graduates (age 24-39) of private Christian schools in the United States, as well as a sample of approximately 150 school administrators from randomly selected Christian schools—424 schools in the US and 85 schools in Canada. There were also four qualitative research teams: (1) Dr. Harro Van Brummelen and Robert Koole studied Christian secondary schools in the US and Canada; (2) Dr. Patty LeBlanc and Dr. Patty Slaughter studied students at a Pentecostal university in the southeastern US; (3) Dr. Cara Stillings Candal and Dr. Charles L. Glenn studied race relations in urban secondary Christian schools; and (4) Dr. Jack Beckman, Dr. James Drexler, and Dr. Kevin J. Eames studied administrators of Christian schools. Articles resulting from these qualitative studies were published in the March 2012 issue of the Journal of School Choice.

The results of Phase II, which were made available as a free download on Cardus’s website (a data file for Phase II is also free to download), were published in 2012 in a report entitled A rising tide lifts all boats: Measuring non-government school effects in service of the Canadian public good. The scholars involved were: Project Leader Ray Pennings; Dr. David Sikkink, Head of Quantitative Study; Dr. Deani Van Pelt, who wrote the report; advisors Dr. Harro Von Brummelen and Dr. Amy Von Heyking; research assistant Shanna Corner; and Cardus editors Dan Postma and Kathryn De Ruijter. Phase II was conducted because of “the need for a comparative survey of a representative sampling of graduates from a variety of government and non-government schooling systems [in Canada].” The quantitative research team, led by Sikkink, surveyed a random sample of adult graduates (age 23-40) of non-governmental Christian schools in Canada.

For both phases of the survey, though researchers were primarily interested in studying graduates of private Christian schools, best research practices required that they also incidentally collect information on homeschoolers.[/expand]

[expand title=”Cardus Phase I major points (United States)”]

This section will summarize the methodology and findings of Pennings et al. (2011) as they relate to homeschooling. (As the main focus of the study was elsewhere, many of their findings are not relevant to homeschooling and as such will not be discussed.)

• The purpose of the study was to “provide evidence of the life directions of graduates of Catholic and Protestant Christian schools in the United States and Canada.” Questions “focused on educational and occupational attainment, civic and political engagement, spiritual formation, marriage and family as well as social psychological outcomes in the young adult years[,] experience of high school[, and] family background characteristics, including parent’s education, family structure while in high school, relationship with parents, etc.”
• The study was conducted on “a random sample of Americans provided by one of the most respected internet survey firms, Knowledge Networks (KN).” KN’s method of operation is to “recruit [US] households by randomly selecting residential addresses using a process called address-based sampling.” This sampling method allows KN to assemble a representative sample of US households that includes appropriate numbers of cell-phone-only households and other populations which are notoriously difficult to survey. KN regularly collects data about their selected households; this data and data from the Integrated Postsecondary Education Survey were available to the researchers in addition to their survey data. Though the total number of survey respondents at 1471 was smaller than many surveys of homeschoolers, it was randomly selected and representative of the US population. This means that its findings can be applied to a population beyond the people who participated in the study.
• The instrument used to conduct the study was a set of two web-based surveys. The first survey, entitled “Science of Generosity Survey,” was conducted in 2010 and took about 60 minutes to complete. Participants were paid $20 for completing the survey. The second survey included a large oversample of private school graduates and took approximately 30 minutes to complete. Participants indicated whether their school was categorized as “public, Catholic, conservative Protestant or “Christian school,” other Christian, non-Christian religious, nonreligious private, or home­school . . . . The home school category is split into home schoolers whose mother attended religious services regularly and those whose mother did not attend religious services regularly.” Of the 1471 respondents, about 500 had attended public school, 61 (4%) were religious homeschool graduates and 21 (1%) were nonreligious homeschool graduates. Nonreligious homeschool graduates were not included in the findings of Pennings et al. (2011), although the researchers have indicated that they intend to publish more on the topic.
• Responses to each survey question were weighted based on the number of participants from each schooling category. “The first regression model for each dependent variable, which included only the binary variables for high school type, included weights appropriate for comparing school sectors. Coefficients from the regression models are presented in the graphs in this report. Smaller coefficients or those with higher standard errors should be considered essentially identical to zero (this is reflected in the discussion of findings in this report). Given the small sample sizes of all but the Catholic sector, coefficients in the tables are considered significant if the p-value is less than .1.” Then, the data were further weighted based on “over 30 variables known to impact development, such as the closeness of one’s relationship to parents, religious service attendance, race, and educational attainment.” Both models are represented in the publication—in each graph, the responses to a single survey question are presented both 1) without demographic controls and 2) with demographic controls.  “In all charts the center line, marked zero, represents . . . graduates of public schools. Therefore, as the charts are interpreted, the second bar, the model including controls, attempts to isolate the specific effect of the school on the independent variable presented.”
• For example, Figure 3 shows a graph of responses to the question “I accept the authority of church leadership”, to which participants could respond on “a 7-point scale running from “completely disagree” to “completely agree”.” Here, the responses of public schoolers are represented by the horizontal line marked ‘0’, and the orange bars show that graduates of Protestant schools and religious homeschools agreed with the question more than public schoolers, while Catholic and nonreligious private schoolers agreed less than public schoolers. “Even after removing the effect of family background,”—that is, isolating the schooling effect, which is represented by the gray bars—“attending an evangelical Protestant school increases one’s expected acceptance of church authority by nearly half a point on this 7-point scale. That is roughly equiva­lent to the increase in acceptance of church authority when one had a mother who attended religious services once a week rather than a mother who only attended once or twice a year. ”
• “The scale for each chart varies depending on the values of the variable being predicted.” Many of the questions were based on a 7-point scale as in Figure 3, but a number of other scales were also used. Which scale was used for which survey question is not always apparent from the graphs used in Pennings et al. (2011). Furthermore, though Pennings et al. indicate that they performed significance testing, the results of these tests do not appear to be represented anywhere in the publication. It is therefore not possible to determine from the given information which differences between religious homeschoolers and public schoolers are representative of differences in the population, and which differences are merely statistical noise.

In the summary of findings that follows, I use the term “points” to refer to points on the scale used in the graph. It was not possible in most cases to determine the scale these points were based on.

• In terms of academics, religious homeschool graduates were .2 points more likely to say they were prepared for college than public schoolers. This opinion was belied by their lack of academic success: they attended universities whose 75th percentile SAT score was 20 points lower than public schoolers; they accumulated six months to a year less of education overall; they were 1.5 points more likely to attend an open admission university and 1.5 points less likely to attend a Research I or II university. Religious homeschool graduates were .5 points less likely to attend college and 1.75 points less likely to get an advanced degree than public schoolers.
• Religious homeschool graduates largely agreed with graduates of conservative Protestant high schools in terms of their strict and legalistic moral outlook. They rated themselves 3 points higher than public schoolers in their feeling that their high school had prepared them for a vibrant religious and spiritual life, and they were .5 points more likely to attend religious services. They were 1 point more likely to agree that morality should be based on absolute standards, and they gave several examples: they were 1.5 points more likely to say that the Bible is infallible; 1.5 points more likely to believe premarital sex is wrong; 1.5 points more likely to believe it is morally wrong to live together; and .8 points more likely to believe divorce is morally wrong. They were .4 points more likely to agree that one of the main problems in the US is a lack of respect for authority, although there was no appreciable difference between them and public schoolers in their response to the question “I accept the authority of church leadership.” They were .1 points less likely to say that they liked new and exciting experiences, even if they had to break the rules, and .25 points more likely to feel obliged to tithe than public schoolers.
• Like graduates of Protestant schools, religious homeschool graduates were not particularly political, rating themselves .1 points lower than public schoolers in terms of their interest in politics and public affairs. They were .3 points less likely to actively campaign for a political party, .4 points less likely to participate in a protest or demonstration, and .1 points less likely to know a community leader than public schoolers.
• One main area where religious homeschool graduates differed from graduates of Protestant schools was in their future outlook. Religious homeschool graduates were .6 points more likely than public schoolers to have feelings of unclarity regarding life goals or sense of direction. They were .4 points more likely to experience feelings of helplessness in dealing with life’s problems.
• Religious homeschool graduates also differed in terms of their emphasis on family from graduates of Protestant schools, “who are having more children and divorcing less frequently than their peers from [public] schools.” Though religious homeschool graduates rated themselves one point higher than public schoolers in terms of how well their high school prepared them for relationships, they had .1 fewer children and .5 more divorces than public schoolers. Their uncorrected age at first marriage was 1.5 years younger than public schoolers, although when demographic factors were corrected for this difference virtually disappeared. While religious homeschool graduates were .5 points more likely to pray with their spouse or partner; 2 points more likely to talk with them about God; and 1 point more likely to read the Bible with them than public schoolers, they were less likely to pray, talk about God, and read the Bible with their families (.25, .25, and .5 points less, respectively). They were also .5 points less likely to have a favorable view of their relationships with teachers.
• Another main area where graduates of religious homeschooling differed from graduates of conservative Protestant high schools was in their view of money and charitable giving. Pennings et al. (2011) concluded that “Protestant Christian school graduates are distinct in their giving habits…[They] seem to be more committed to their churches, volunteering more[,] giving more money to their congregations[, and] committing to mission trips in their adult lives.” However, religious homeschool graduates did not share this outlook. They did not differ appreciably from public schoolers in their response to the question “I would be unhappy with myself if I thought I was not a generous person.” Their total annual charitable donations were $200 less than public schoolers’—$750 less in terms of congregational donations and $50 less in charitable donations to religious causes outside their congregations. They spent .25 hours less than public schoolers volunteering in their congregations, and made .5 fewer mission trips, .1 fewer evangelism trips, and .4 fewer relief and development trips in adulthood than public schoolers. They were .3 points more likely than public schoolers to be seeking a job that pays well, were 2 points more likely to give political donations, and were .5 points more likely to know a CEO or elected official. They were .5 points more likely to be grateful for their income and possessions than public schoolers.[/expand]

[expand title=”Cardus Phase II major points (Canada)”]

This section will summarize the methodology and findings of Pennings et al (2012a & 2012b) as they relate to homeschooling. (As the main focus of the study was elsewhere, many of their findings are not relevant to homeschooling and as such will not be discussed.)

• This study was “based on a March 2012 survey of a representative sample of 24- to 39-year-old Canadian residents who had graduated from secondary schools.” It had much of the same methodology as Cardus Phase I.
• The total number of survey respondents was 2054, of which 845 had attended public schools, 41 (2%) were religious homeschool graduates and 17 (.83%) were non-religious homeschool graduates. When French-speaking respondents were eliminated from the dataset, there were 683 public school graduates and 34 religious homeschool graduates. The number of non-religious homeschool graduates was insufficient for analysis.
• Again, the data for each school group was weighted based on the sample size, then weighted again based on demographic variables including: “gender, race, and age, whether the respondent attended non-governmental elementary school…[t]he education, religious tradition, religious service attendance, and volunteering habits of the respondent’s mother and father…How close the respondent was to their mother and father and whether their father or mother pushed the respondent academically…The marital status and living arrangement when the respondent was in high school…[whether respondents were] raised by both biological parents [and] the number of years that respondents lived with each biological parent.”
• “Coefficients from the regression models are presented in the graphs in [Pennings et al. (2012a)]. Smaller coefficients or those with higher standard errors should be considered essentially identical to zero…Given the small sample sizes of all but the two Catholic sectors and the paucity of data on many of the smaller school sectors, coefficients in the regressions are considered significant if the p-value is less than 0.1.”
• In terms of academics, religious homeschool graduates in Canada had a generally positive view of their high school environment—they rated their high school experience in general .25 points higher than public school graduates did. Religious homeschool graduates approved of their school more than public schoolers did in terms of its ability to prepare them for a religious life (2.25 points higher), its handling of religious matters (1 point higher), its ability to prepare them to interact with society (.9 points higher), its administrators (.5 points higher), its close-knit community (.5 points higher), its social scene (.3 points higher), its students (.3 points higher), its teachers’ genuine care for students (.3 points higher), its ability to prepare them for relationships (.3 points higher), its students’ ability to get along with one another (.25 points higher), its educational quality (.2 points higher), and its ability to prepare them for academic success in university (.2 points higher). Religious homeschool graduates only rated their high schools lower than public schoolers did in their evaluation of teachers (.1 points lower) and of the available athletic opportunities (.2 points lower). Religious homeschool graduates also felt that their school environment sheltered them too much (.5 points higher than public schoolers). Public school graduates and religious homeschool graduates did not differ substantially in their responses to the statements ‘I enjoyed going to my high school’, ‘I’m proud to have graduated from my high school’, ‘My high school’s rules were too strict’, and ‘My high school prepared me for career success’.
• Despite this sunny attitude, Canadian homeschool graduates had somewhat mixed results in terms of their educational and career outcomes. They had .5 points less education overall than public school graduates, and the highest level of education they completed was .75 points lower. They indicated that they were 1.25 points more likely than public school graduates to discontinue their education after graduating from high school and 1.5 points more likely to discontinue after finishing college (Note that in Canada, college refers to an institution offering 1-3 year training programs which are career-oriented and do not terminate in a bachelor’s degree; while only universities grant bachelor’s, master’s, and doctoral degrees). However, for the comparatively fewer religious homeschool graduates who did attend university, they were .4 points less likely than public school graduates to stop at a master’s and 1.75 points more likely to complete a Ph.D.
• Religious homeschool graduates’ household income was .7 points lower than public school graduates’ before controlling for demographic variables; this was reduced to .1 points lower when family background was controlled. They were .75 points more likely to be employed part-time and .9 points more likely to be unemployed than public school graduates. They were, by contrast, .4 points more likely to hold a professional or managerial position than public school graduates, although this was not statistically significant.
• Furthermore, religious homeschool graduates reported a lack of clarity and satisfaction in their lives. They were .3 points more likely to feel helpless when dealing with problems and .15 points more likely to feel that their life lacked goals or direction. Though they exceeded public schoolers by less than .1 points in their response to the statement “I have so much in life to be thankful for”, they nonetheless reported lower levels of satisfaction with their friendships (.12 points lower), the place they lived (.5 points lower), their hobbies (.075 points lower), and their health (.4 points lower). They did not differ appreciably from public schoolers in their reported enjoyment of high school and their pride at graduating from their high school.
• Compared to American religious homeschool graduates, Canadians had more successful families, although they reported .4 points less satisfaction with family life than public school graduates. They had .3 more children, cohabitated at a rate 2 points lower than public schoolers, and were 3 points less likely to divorce. There was no appreciable difference from public school graduates in their age at first marriage or their likelihood of marrying. They were .75 points more likely to pray with their families, .5 points more likely to discuss politics with their families, and .4 points more likely to share their faith with family and friends. They read the Bible with their families .2 points more frequently than public schoolers, although they were .8 points less likely to eat a family meal together.
• As in the United States, Canadian religious homeschool graduates demonstrated a strict and legalistic moral outlook in terms of their personal lives and their views on theology and politics. In their personal lives, they were 1.5 points more likely to believe that living with a partner before marriage was immoral; 1.5 points more likely to view premarital sex as immoral; 1.25 points more likely to believe divorce was immoral; and .8 points more likely than public school graduates to agree that women should stay home from work to take care of the family. Religious homeschool graduates were 1.5 points more likely to believe that God or the Bible helped them to determine right and wrong; they were 1.5 points more likely to feel obligated to practice spiritual discipline; and they were .2 points less likely to enjoy new and exciting experiences if it required that they break the rules.
• From a theological point of view, religious homeschool graduates rated themselves 1.5 points more likely to believe that there is no way to salvation except through Jesus, 1.5 points more likely to believe that the Bible is an infallible guide for behavior, and 1.5 points less likely to believe that there are scientific or historical errors in the Bible. They rated themselves 1.25 points higher in their obligation to accept the authority of church leadership, .25 points higher in their belief in the personhood of God, .25 points higher in their belief that suffering is part of God’s plan, and .25 points higher in their belief that God created the world in six 24-hour days.
• In terms of politics, religious homeschool graduates scored 2 points higher on the belief that same-sex marriage is immoral and 1.75 points higher in their belief that abortion should always be illegal except to save the life of the woman. They were .9 points more likely to believe the dominant culture in Canada was hostile to their religious values. They were .65 points more likely to believe that higher standards of health care should be available to people who can afford it and .6 points less likely to feel that society should be more tolerant of non-Christian religions. They were .2 points more likely to feel obligated to participate in politics and .1 points less likely that national ID cards should be issued.
• In a few cases, religious homeschool graduates showed a less strict moral outlook: they scored 3 points higher than public school graduates in their belief that it should be legal to say offensive things about religion in public; they scored .1 points lower in their obligation to make sure that their coworkers behave ethically; and they scored .1 points lower in their belief in trickle-down economics (i.e. that when businesses make a lot of money, everyone benefits, including the poor).
• Pennings et al. (2012) found that, in contrast with their strict moral outlook, religious homeschool graduates in Canada had a more vibrant religious life than public school graduates, with 1.5 points more of them reporting that religion played an important role in their daily lives and .8 points more reporting a feeling of fulfillment from their spirituality. Religious homeschool graduates were 1.75 points more likely to read the Bible alone, 1.75 points more likely to regularly attend religious services, 1.5 points more likely to pray by themselves, and 1.5 points more likely to read religious literature than public school graduates. They were 1.25 points more likely to try to strengthen their relationship with God, 1 point more likely to experience deep communion with God, .75 points more likely to have had a religious conversion, .7 points more likely to find spiritual peace despite their problems, and .5 points more likely to attend a small group dedicated to spiritual discipleship. They reported that as teenagers, religious faith shaped how they lived (1 point more than public schoolers) and their family talked about God and religion (1 point more than public schoolers).
• In addition, religious homeschool graduates were less likely to have negative religious experiences than public school graduates: they were 1 point less likely to identify as non-religious, .4 points less likely to have religious doubts, and .1 points less likely to have had a deconversion experience. They also integrated religion into other aspects of their lives, serving on congregational committees 1.25 points more often than public schoolers and spending .25 more hours volunteering in their congregations. They were .6 points more likely to trust their congregation and had .5 points more confidence in organized religion than public schoolers. They were 1.25 points more likely to believe that God has called them to their line of work and 1 point more likely to seek a job that fulfilled their religious calling,
• Religious homeschool graduates in Canada reported that they were only mildly interested in politics and that they were distrustful of social institutions in general, though they were very involved at the local level. While they were .75 points more likely to participate in a political organization, .5 points more likely to know an elected official, .25 points more likely to campaign for a political party or candidate, and .15 points more likely to participate in a boycott; they were also .45 points less likely to donate to a political cause, .4 points less likely to participate in a political movement, .25 points less likely to volunteer for a political cause, .2 points less likely to participate in an environmental rights movement, and .2 points less likely to participate in a protest, march, or demonstration.
• They reported greater participation than public school graduates in a number of organizations, such as singing groups or choirs (.75 points greater), neighborhood or civic groups (.7 points greater), nonpartisan civic organizations (.7 points greater), and hobby, sports, or youth groups (.3 points greater); however, they were .5 points less likely to volunteer outside their congregations and reported .1 fewer total volunteer hours than public school graduates in groups such as arts or cultural organizations (.8 points fewer), health care organizations (.6 points fewer), high schools (.4 points fewer), neighborhood or civic groups (.1 points fewer), and organizations to help the poor and elderly (.1 points fewer). They were .3 points less likely to serve as a committee member of an organization outside their religious congregations and they were .35 points less likely to agree that the federal government should do more to solve social problems.
• Religious homeschool graduates in Canada reported greater trust and confidence than public school graduates did for some institutions, such as the prime minister (.9 points greater), the federal government (.5 points greater), banks and financial institutions (.45 points greater), the armed forces (.4 points greater), and federal parliament (.2 points greater). However, religious homeschool graduates were more distrustful than public school graduates of education (1.4 points less confident), the supreme court of Canada (1 point less), TV (1 point less), scientists (1 point less), the scientific community (.75 points less), the press and media (.6 points less), medicine (.3 points less), mass media (.2 points less), and major companies (.1 points less). They also had more personal distrust of public school teachers (1.5 points less trust), atheists (.6 points less), coworkers (.4 points less), neighbors (.2 points less), and strangers (.2 points less); and they were .6 points less likely to desire a job with a chance to make friends.
• Finally, religious homeschool graduates in Canada shared some financial values with Americans, such as a desire for a job that pays well (.35 points more than public school graduates), but were more willing to tithe (.75 points more) and to contribute money to charitable contributions (.25 points more) such as to their congregation (1.25 points more) or another religious organization (1.25 points more). On the other hand, they were .7 points less likely to donate to a secular organization.[/expand]

[expand title=”Critical analysis”]

The Cardus Education Survey is methodologically sound, making it one of the few surveys of homeschool graduates that may be applied to the population of homeschoolers. As the researchers used a random sample of homeschool graduates, weighted schooling sectors based on population, and corrected for demographic factors, their sample can be assumed to be representative.

However, it does have some limitations:

• The Cardus publications only address religious homeschoolers and do not draw any conclusions about non-religious homeschoolers.
• Pennings et al. (2011, 2012a, 2012b) only surveyed a small number of religious homeschool graduates. They explain that: “[w]hen evaluating the adequacy of sample sizes, it is…important to keep in mind that tests of statistical significance take into account the sample size. All else being equal, a small sample size makes it more difficult to uncover statistically significant differences between two groups…, but it doesn’t lead to biased results per se….Finding statistically significant results is more difficult with the religious home schooling sector. Some of the estimated differences between this group and public schoolers are very large, even though the small sample size makes it difficult to conclude that these differences are not due to sampling fluctuations.” That is, the researchers found few statistically significant findings in their survey of religious homeschool graduates because the sample size was so small. With a larger sample size, trends they observed may have been found to be statistically significant.
• The researchers distinguished between homeschool graduates whose mothers had frequently attended religious services and those whose mothers had not. This may not be the most effective way to distinguish between religious and secular homeschoolers, and it does not address pragmatic homeschoolers as a separate group.
• The researchers did not determine how long respondents were homeschooled or which grades they were homeschooled for. We do not know whether homeschoolers in umbrella schools, or those officially considered private schools by their state, were classified as homeschool graduates or as graduates of Protestant schools.
• Homeschool graduates surveyed for this study were between the ages of 24 and 29 (in the US) or 23 and 40 (in Canada). Their responses tell us what outcomes resulted from the homeschooling practices of 20 to 40 years ago. The homeschooling community today may be vastly different—we do not know.
• The instruments (that is, the surveys administered to the respondents) were not included in the publications. As such, it is difficult to tell what units are represented by the y-axis in each graph of a question’s results. In addition, the authors state that they performed significance testing and discuss a few of the results in the text of the publications; they describe the p-values as being represented on the graphs. However, it was not possible to determine the p-values with only this information. As such, many of the findings discussed in the previous sections may have been due to chance sampling fluctuations.
• As with any study, even those differences found to be statistically significant have a small probability—in this case, 10%—of not being representative of a difference in the population. This is why multiple replications of studies are important; however, few if any other studies have been conducted on a representative sample of homeschool graduates. More research is needed to determine how meaningful this study’s results are.[/expand]

[expand title=”What the study actually says”]

The major findings of the Cardus study are as follows:

• The study indicates fairly negative outcomes for religious homeschoolers in the United States. Though they were positive about their academic abilities, religious homeschool graduates were less likely than public school graduates to obtain quality higher education. They had a strict and legalistic moral outlook, a lack of interest in politics, and did not show a tendency for volunteerism or charitable giving. They reported a sense of helplessness and a lack of clarity about their lives. They married younger, divorced more, and had fewer children than public school graduates.
• The picture of religious homeschool graduates in Canada is somewhat less grim than that of the United States. They enjoyed their high school education and had more success in higher education and careers than American religious homeschool graduates, though compared to Canadian public school graduates their results were mixed. They reported low satisfaction with their lives, although their families were more successful than those of American religious homeschool graduates. Like Americans, Canadian religious homeschool graduates had a strict and legalistic moral outlook, although they reported vibrant religious lives. They also were found to have mixed interest in politics, high group involvement at the local level, and a deep distrust of social institutions.[/expand]

[expand title=”Conclusion”]

As one of the few studies to survey a random sample of homeschool graduates and control for demographic factors, the Cardus Education Survey provides a useful glimpse into the lives of religious homeschoolers in the US and Canada. Pennings et al. found that religious homeschool graduates in the United States were somewhat less successful than public school graduates in terms of academics, families, and outlook. The picture of religious homeschool graduates in Canada was somewhat more positive, although they were still outdone by public schoolers in many academic areas.[/expand]

[expand title=”References”]

Anonymous. 2013. “About Cardus.” Cardus. http://www.cardus.ca/organization/about/.

Anonymous. 2013. “KnowledgePanel.” Resources. Knowledge Networks. GfK Custom Research North America. http://www.knowledgenetworks.com/knpanel/.

Anonymous. 2013. “Open admissions.” Wikipedia. http://en.wikipedia.org/wiki/Open_admissions.

Anonymous. 2013. “Research I university.” Wikipedia. http://en.wikipedia.org/wiki/Research_I_university.

Anonymous. 2013. “Richard and Helen DeVos Foundation.” Sourcewatch. http://www.sourcewatch.org/index.php/Richard_and_Helen_DeVos_Foundation.

Anonymous. 2013. “Vision.” The Van Lunen Center: Executive Management in Christian Schools. http://www.calvin.edu/vanlunen/about/vision.html.

Anonymous. 2013. “What is the difference between college and university in Canada?” Settlement.org. Ontario Council of Agencies Serving Immigrants. http://www.settlement.org/sys/faqs_detail.asp?faq_id=4001136.

Anonymous. 2013. “William Voortman Fund.” Chimp Technology, Inc. https://chimp.net/charities/william-voortman-fund.

Gaither, Milton. 2011. “The Cardus Education Survey and homeschooling.” Homeschooling Research Notes. http://gaither.wordpress.com/2011/09/23/the-cardus-education-survey-and-homeschooling/.

LeBlanc, Patty and Patty Slaughter. 2012. “Growing thinking Christians: An investigation of the outcomes of Christian education.” Journal of School Choice: International Research and Reform 6:1. 62-81.

Merrifield, John (ed.). 2012. Journal of School Choice: International Research and Reform 6:1. Special Issue: Private Religious Protestant and Catholic Education in North America: Contributions and Concerns Special Section. http://www.tandfonline.com/toc/wjsc20/6/1#.UqZBb-KOcUE.

Pennings, Ray, David Sikkink, Kathryn L. Wiens, John Seel, and Deani A. Neven Van Pelt. 2011. Do the motivations for private religious Catholic and Protestant schooling in North America align with graduate outcomes? Cardus Education Survey: Phase I Report. Hamilton, ON: Cardus.

Pennings, Ray, David Sikkink, Deani Van Pelt, Harro Van Brummelen, Amy Von Heyking. 2012a. A rising tide lifts all boats: Measuring non-government school effects in service of the Canadian public good. Cardus Education Survey: Phase II Report. Hamilton, ON: Cardus.

Pennings, Ray, David Sikkink, Deani Van Pelt, Harro Van Brummelen, Amy Von Heyking. 2012b. Cardus Education Survey: Phase II Data Pack. Hamilton, ON: Cardus.

Slick, Matt. 2013. “How do we tell if God is a person?” About God. Christian Apologetics and Research Ministry. http://carm.org/is-god-a-person.

Smith, Christian, Patricia Snell Herzog, and Kraig Beyerlein. 2010. Methods Report and User’s Guide to the 2010 Science of Generosity Survey. Notre Dame, IN: University of Notre Dame.[/expand]

Dr. Chelsea McCracken
Research Analyst
Coalition for Responsible Home Education
chelseamc@responsiblehomeschooling.org
661 Washington Street #563
Canton, MA 02021