Tools of the Trade

Many, many years ago, a co-worker decided to take up woodworking as a hobby. He moved his cars out of his suburban garage — there would no longer be any room for them there — and purchased some of the finest sets of tools and machinery available: band saw, jig saw, router, drill press, and a few others I couldn’t tell you the names of because I’m not a woodworker. He went top of the line, too (or so he assured me, and I believe him), making sure he had the best bells and whistles. That was just his machinery. His array of hand tools was equally impressive lined up on his shiny new cabinets where, in each drawer, he laid a piece of foam with cut-outs in the shape of his tools. This way, there would be a place for every tool and a tool in every place. It was, all in all, a weekend warrior’s dream.

And then he built his book cases and media center. They weren’t awful by any means, but they weren’t spectacular either. I had a hard time seeing how they were much better than what I could have made with my far humbler collection of mismatched and somewhat rusty tools laying around the house. Assuming I could find them.

My co-worker’s dazzling workshop and my mess of a garage come to mind as I read some Mark Regnerus’s postings defending — no, not defending, but bragging about — his purported study on gay and lesbian parenting I reviewed last week. I say purported because with most of the adult children studied spending much less than three years with their gay or lesbian parent, it seems to me to be a study that is not about gay parenting but about children of divorce, separation, parental infidelity, and a host of other sources of instability. Gay parents, it turns out, are barely in the picture at all.

Regnerus blithely ignores this criticism. Instead, he distinguishes his study by the shiny $785,000 worth of tools that he assembled to produce it. It’s often the first point he mentions when he talks about it, like in his defense posted on the Baptist-affiliated Baylor University web site:

By contrast, Regnerus relies on a large, random, and representative sample of more than 200 children raised by parents who have had same-sex relationships, comparing them to a random sample of more than 2,000 children raised in heterosexual families, to reach his conclusions. This is why sociology professor Paul Amato, chair of the Family section of the American Sociological Association and president of the National Council on Family Relations, wrote that the Regnerus study was “better situated than virtually all previous studies to detect differences between these [different family] groups in the population.”

National random samples are inordinately expensive. It is why they are so rare in social science research. It’s every social scientist’s wet dream to be able to assemble a garage like Regnerus’s, but for almost every one of them such a tool set is way out of their reach. So having a tool as shiny and impressive as a nationally random sample is a really big deal. I’d say that it would probably very difficult for most people to resist showing it off.

But what I see happening is that Regnerus wants us to be dazzled by his tools — and they are truly dazzling — and not look too closely at the book case he built with them. And if that book case looks a little off kilter, maybe its because he used his tools incorrectly. If you use a drill press when you really needed a router, you’re going get results that fall short of what you promised. If you ask adults between 18 and 39 if either of their parents ever had a same-sex relationship and leave it up to the responder to ponder whether a short tryst counts, then you are going to get a sample that doesn’t resemble anything like the kind of relationship that is comparable to any kind of marriage, gay or straight.

And as his study has little to say about anything close to resembling marriage, it also says almost nothing about parenting. In his study, only 57% said they had lived with their mother and her partner for at least four months before the age of 18, and only 23% reported living with their father and his partner for the same length of time. Only 23% of children of so-called “Lesbian Mothers” and 2% of children of so-called “gay fathers” — both loosely defined — reported living with their parents and their parents’ same-sex partners for three years or more. If this is any kind of a study about gay parenting, it is even more so a study about the fifteen-plus years these kids spent outside of that so-called relationship.

And so after constructed a sample that was guaranteed to look nothing like a stable family environment, and then compared it to a sample that was specifically constructed to be a stable as possible, he found differences. That happens when you compare apples to orangutangs. And what else would you expect? When you build an unlevel shelf, you shouldn’t be surprised to see all of the results sliding off to one end. Sure, he may have started with a nationally representative sample, but that doesn’t give him license to move populations around within his samples with the purpose of achieving his aims, or to make claims about what his sample represents when his sample wasn’t constructed to represent what he claims.

And these issues point to a much bigger problem with Regnerus’s study. He wanted to examine the impact of gay parenting on children, but his tools, as impressive and expensive as they were, turned out to be the wrong tools for the job. As powerful as they otherwise would have been, they lacked the critical features that he needed to do the job correctly. And so lacking those important features, he does what a lot of weekend hobbyists in a hurry end up doing: he kludges it and hopes nobody notices. Except they have, and those kludges have been the crux of the criticisms against his study.

His response so far has boiled down to this: don’t pay any attention to how he used his tools. Just look at how shiny and expensive they are. But as every craftsman will tell you, it’s not the tools that make the better product, but the care of the craftsman who uses them wisely.