Overview: Three Responses to Mark Regnerus’s Study of Children of Parents In Same-Sex Relationships

For those of you returning from the weekend, you may have missed the release of a new study published in the July issue of the journal Social Science Research. That study claims to show “numerous, consistent differences, especially between children of women who have had a lesbian relationship and those with still-married (heterosexual) biological parents.” If true, these results would up-end some thirty years of established scientific research which had previously shown that gay and lesbian parents are, on the whole, just as good parents as their straight counterparts. The study received one-sided coverage in the Deseret News and the Washington Times.

If you haven’t already done so, I would recommend you read my analysis of Mark Regnerus’s study, “How Different are the Adult Children of Parents Who Have Same-Sex Relationships? Findings from the New Family Structures Study” (Social Science Review 41, no. 4 (July 2012): 752-770.)

Later yesterday morning, I learned that the July 2012 issue of Social Science Review also includes three commentaries on Regnerus’s paper and a rejoinder by Regnerus. That same issue also published a related paper by Loren Marks reviewing and identifying several weaknesses of the past thirty years of social science research about gay parents. I probably won’t be able to review Marks’s paper until later in the week. The three commentaries also remarked on Marks’s paper, but I will concentrate this overview on their observations about the Regnerus paper.

Commentary #1: Paul R. Amato, of Penn State’s Department of Sociology. Amato, like the other two commenters who we will review shortly, commended the Regnerus paper for its strongest feature: the use of a large, national probability sample. As I said yesterday, this is an extraordinarily rare achievement in social science research, and it is why Regnerus’s paper is so important. Amato reviewed Regnerus’s data and for the most part verified Regnerus’s finding that Regnerus’s sample of children of Lesbian Mothers (with all of the issues surrounding the construction of that sample I noted yesterday), did indeed depart from the Intact Biological Family sample.

Amato used a different statistical method to come to that conclusion, a method known as “effect size.” A very brief description is in order. Effect size is, according to this paper, “a simple way of quantifying the difference between two groups that has many advantages over the use of tests of statistical significance alone.” Traditional statistical tests, like those Rengnerus used, identify the statistical significance of two measures, meaning that they measure whether the similarity or difference between two measures can be explained by random chance — sort of like two numbers that fall within the same margin of error. If they can’t, then they are said to be statistically significant. Effect size, on the other hand, estimates the strength of an apparent relationship between two populations based on the standard deviation of results of the control population. This is important because if the control population has a high degree of variability, then even a large differences in the averages between the two groups might not mean anything since there would still be a large overlap.

I know that’s not enough of an explanation to make you (or me) an expert on statistical significance and effect size. But the important thing to keep in mind is that in social science and medical research, it is increasingly recognized that both calculations should be performed. Traditionally, only statistical significance is calculated (as Regnerus did in his paper) and many authors today still only rely on statistical significance tests to evaluate their data. But some journals are beginning to require effect size calculations alongside statistical significance measures, and statistical software packages are beginning to include effect size in their libraries. But calculating effect size has not yet become a standard standard practice.

So, getting back to Amato’s paper, he ran some effect size calculations and, as I said, he confirmed Regnerus’s finding that children growing up in lesbian households (as aggregated in Regnerus’s sample with all of its problems — and I want to keep reiterating that) differed from children growing up in intact biological families with “a moderately large effect size.” However:

The choice of comparison group makes a difference, however. Comparisons of offspring with lesbian mothers and offspring from heterosexual stepfamilies revealed a mean effect size of only .15. When children with divorced or continuously single mothers served as the comparison group, the mean effect size was only .19. I would describe these effect sizes as weak.

What is the most appropriate comparison group? This is a difficult question, given the heterogeneity of gay and lesbian families with children. Consider lesbian couples who have children through sperm donation, or gay couples who have children through surrogacy. Is it reasonable to compare these children with the children of continuously married heterosexual parents? Or should children in the heterosexual comparison group be limited to those born via sperm donation or surrogacy? What about lesbian mothers or gay fathers with children from former marriages or unions? Should these children be compared with those of heterosexual parents who are married, cohabiting, remarried, divorced, or never married? The fact that same-sex marriage is now allowed in several states adds another level of complexity to the problem. Perhaps in future studies, married same-sex parents should be matched with married heterosexual parents.

Which pretty much echoed my concerns yesterday.

Commentary #2: David J. Eggebeen of Penn State’s Department of Human Development and Family Studies. Eggebeen echoed many of Amato’s sentiments. First, he praised the sophistication of Regnerus’s raw data set, which represents a significant advance over previous studies. However:

Nevertheless these data are far from ideal. A larger sample does not translate into a sufficiently large sample. Given some of the complexity of the family lives detailed above, a sample of 163 young adults who report a lesbian mother and 73 who report a gay father are frustratingly inadequate for doing anything but broad comparisons across family characteristics.

Eggebeen also cautions against jumping to quick conclusions about what the data means:

Finding a significant number of negative correlates of well-being for children with gay or lesbian parents, even if they are derived from simple models, invites thinking about some possible mechanisms. It is hard to imagine explanations that point to the quality of parenting per se. Parents, regardless of sexual orientation, are equally motivated to provide the best care possible for their children. It is reasonable, however, to posit that gay and lesbian parents and their children face challenges that may make parenting more difficult.

He concludes that Regnerus’s study has the potential to address the shortcomings of previous studies, but with an important caveat:

The analyses in the Regnerus paper are provocative but far from conclusive. These very preliminary findings should not detract from the real importance of this paper, the description of a new data set that offers significant advantages. Whether the New Family Structures Study has the possibility of unsettling previously settled questions depends in equal parts on richness of the information collected, as well as the willingness of scholars to make use these data.

Regnerus’s larger data set and variable list will become publicly available in the fall so that other researchers to perform their own sets of calculations and comparisons.

Commentary #3: Cynthia Osborne, of the University of Texas LBJ School of Public Affairs. Osborne, like the others, also noted the heterogeneity (i.e. the large in-group differences) of the Lesbian Mother and Gay Fathers groups:

To increase the sample size of children who experienced a same-sex parent, Regnerus included respondents in either the LM or GF comparison groups if they reported that their parent ever had a same-sex relationship. Although this decision has a lot of merit, it makes comparisons across groups somewhat of a challenge. Because the LM group is comprised of young adults who experienced multiple family forms and transitions, it is impossible to isolate the effects of living with a lesbian mother from experiencing divorce, remarriage, or living with a single parent.

She also questioned the decision of what to use as an outcome variable verses a control variable. For the straight populations, the control variables included divorce parents, step-parents, adopted parents, single parents, etc. Among the outcome variables were questions about sexual molestation, forced sex, and whether the family received welfare. She wondered what the data might look like of those were used as control variables rather than outcome variables, since each of them can have a strong effect on childhood outcomes. But while there are all sorts of comparisons that might be made, assigning a cause for the outcome cannot be done based on the data at hand:

Importantly, one cannot clearly link having a lesbian mother (or gay father) with any of these outcomes. As stated earlier, the group is comprised of young adults who experienced multiple family structures, not only a same-sex parent household (indeed, some of the respondents never lived with the mother’s same-sex partner). It is quite possible, for example, that many or most of the negative outcomes result from the divorce of the young adult’s biological parents that preceded the mother’s same-sex relationship. …

…The concern for Regnerus is not Type II errors (saying something is NOT significant when it is), but the possible attribution of differences to living in a same-sex household rather than to experiencing multiple family structures in childhood, one of which happened to be a same-sex parenting relationship.

Osborne emphasized that researchers in this particular field — those who focus on children of same-sex parents — bear a particular responsibility for how their research is presented to the public:

The focus on children of same-sex parents seems, then, to be driven more by the sensitive political and social issues surrounding same-sex relationships than by evidence that this family structure is increasing rapidly or, for that matter, harmful to children. Because the topic is so politicized, scholars must pay even more careful attention to the presentation and interpretation of their findings. Although scholars are trained to use great care to disentangle the causal versus selection effects of family structure and child well-being, we understand that true causation can never be determined because we cannot randomly assign children to various family structures. Consumers of research on children of same-sex relationships, by contrast, may not always have the same training or be so careful in their interpretations. The results of scholarly studies are often scrutinized by pundits and legislators to support their pre-existing ideas of differences or ‘‘no differences’’ across groups.

…Regnerus (2012) finds substantial differences across groups and uses great care to note that his descriptive analysis does not imply causation and that the LM respondents may have lived in many different family structures. Still, the rigor of the study may lead some advocates to claim that growing up with a same-sex parent causes harm and should, therefore, be illegal.

I think we can bank on Osborne’s concern coming true. I’m willing to lay odds that we will see it coming true today.

Mark Regnerus’s Rejoinder. Regnerus’s response was brief. His first comment was to reiterate Amato’s and Osborne’s concerns about the political rammifications of the study:

I recognize, with Paul and Cynthia, that organizations may utilize these findings to press a political program. And I concur with them that that is not what data come prepared to do. Paul offers wise words of caution against it, as did I in the body of the text. Implying causation here—to parental sexual orientation or anything else, for that matter—is a bridge too far.

Regnerus acknowledged that “the sample size of respondents whose parents report a same-sex relationship is substantial but not large enough to explore some of the more fine-grained distinctions that may well be present.” He also said that he is already planning to use the detailed dataset in future studies, and invited other researchers to mine the data when it becomes publicly available in the fall. Finally, he ends with this paragraph defending his emphasis on biologically intact heterosexual families:

As each of the three explicitly or indirectly notes, family instability—whatever the sources—is often a top culprit in predicting dysfunction in the lives of children, and the data analyses in my article likewise point in this direction. In fact, the most significant story in this study is arguably not about the differences among young–adult children whose parents who have had same-sex relationships and those whose parents are married biological mothers and fathers, but between the latter and nearly everyone else. Contexts of instability—whether in gay or straight households—appear suboptimal for children’s healthy long-term development. While much is made in the scholarly literature about ‘‘resilient’’ youth—those who thrive despite the odds against them and in lieu of an optimal family context—resilience is, on average and perhaps by definition, not normal. Moreover, even resilient children would likely prefer to have engaging parents who are not simply in their lives but in their households. Adults of good will, and most family scholars, typically agree on this. Whether some relationship arrangements are more systematically prone to disorganization than others is an important and empirically-testable question.