Rating Romance: The Scientific Flaws of Online Dating Sites

C'mon. You knew that sooner rather than later someone with the kind of math skills you despised and resented in high school would do a scientific, statistically significant examination of the algorithms on which online dating is based.

And you suspected, could just feel, they would tell you it was all a sham. The quantitative web equivalent of those Three Card Monte street con artists.

But cheer up. The good news is that the findings are not totally dismissive. Yes, all the sites appear to exaggerate their claims - welcome to the business world - d'uh. But the overall effort is probably worth pursuing. Just like putting on some slick threads and going out to a bar on Friday night.

The problem is that all of the nuances science has identified as crucial to successful long term relationships are excluded from consideration by virtually every site. Literally. Every one. This is because those factors can be hard to quantify and then used to construct comparable data. It will eventually be possible to incorporate this kind of information in a useful format. But potential online daters probably aren't enticed by the use of the word 'eventually.'

In a frantic, attenuated, hit-or-miss world, humans, as a very social species, are looking for all the connectivity assistance they can get. Online dating is, on balance, a net positive. And it will get better. In the interim, tight jeans and slinky t-shirts R US. JL

Eli Finkel and Susan Sprecher report in Scientific American:

Every day, millions of single adults, worldwide, visit an online dating site. Many are lucky, finding life-long love or at least some exciting escapades. Others are not so lucky. The industry—eHarmony, Match, OkCupid, and a thousand other online dating sites—wants singles and the general public to believe that seeking a partner through their site is not just an alternative way to traditional venues for finding a partner, but a superior way.

Is it?
With our colleagues Paul Eastwick, Benjamin Karney, and Harry Reis, we recently published a book-length article in the journal Psychological Science in the Public Interest that examines this question and evaluates online dating from a scientific perspective. One of our conclusions is that the advent and popularity of online dating are terrific developments for singles, especially insofar as they allow singles to meet potential partners they otherwise wouldn’t have met. We also conclude, however, that online dating is not better than conventional offline dating in most respects, and that it is worse is some respects.

Beginning with online dating’s strengths: As the stigma of dating online has diminished over the past 15 years, increasing numbers of singles have met romantic partners online. Indeed, in the U.S., about 1 in 5 new relationships begins online. Of course, many of the people in these relationships would have met somebody offline, but some would still be single and searching. Indeed, the people who are most likely to benefit from online dating are precisely those who would find it difficult to meet others through more conventional methods, such as at work, through a hobby, or through a friend.

For example, online dating is especially helpful for people who have recently moved to a new city and lack an established friendship network, who possess a minority sexual orientation, or who are sufficiently committed to other activities, such as work or childrearing, that they can’t find the time to attend events with other singles.

It’s these strengths that make the online dating industry’s weaknesses so disappointing. We’ll focus on two of the major weaknesses here: the overdependence on profile browsing and the overheated emphasis on “matching algorithms.”

Ever since Match.com launched in 1995, the industry has been built around profile browsing. Singles browse profiles when considering whether to join a given site, when considering whom to contact on the site, when turning back to the site after a bad date, and so forth. Always, always, it’s the profile.

What’s the problem with that, you might ask? Sure, profile browsing is imperfect, but can’t singles get a pretty good sense of whether they’d be compatible with a potential partner based on that person’s profile? The answer is simple: No, they cannot.

A series of studies spearheaded by our co-author Paul Eastwick has shown that people lack insight regarding which characteristics in a potential partner will inspire or undermine their attraction to him or her. As such, singles think they’re making sensible decisions about who’s compatible with them when they’re browsing profiles, but they can’t get an accurate sense of their romantic compatibility until they’ve met the person face-to-face (or perhaps via webcam; the jury is still out on richer forms of computer-mediated communication). Consequently, it’s unlikely that singles will make better decisions if they browse profiles for 20 hours rather than 20 minutes.

The straightforward solution to this problem is for online dating sites to provide singles with the profiles of only a handful of potential partners rather than the hundreds or thousands of profiles that many sites provide. But how should dating sites limit the pool?

Here we arrive at the second major weakness of online dating: the available evidence suggests that the mathematical algorithms at matching sites are negligibly better than matching people at random (within basic demographic constraints, such as age, gender, and education). Ever since eHarmony.com, the first algorithm-based matching site, launched in 2000, sites such as Chemistry.com, PerfectMatch.com, GenePartner.com, and FindYourFaceMate.com have claimed that they have developed a sophisticated matching algorithm that can find singles a uniquely compatible mate.

These claims are not supported by any credible evidence. In our article, we extensively reviewed the procedures such sites use to build their algorithms, the (meager and unconvincing) evidence they have presented in support of their algorithm’s accuracy, and whether the principles underlying the algorithms are sensible. To be sure, the exact details of the algorithm cannot be evaluated because the dating sites have not yet allowed their claims to be vetted by the scientific community (eHarmony, for example, likes to talk about its “secret sauce”), but much information relevant to the algorithms is in the public domain, even if the algorithms themselves are not.

From a scientific perspective, there are two problems with matching sites’ claims. The first is that those very sites that tout their scientific bona fides have failed to provide a shred of evidence that would convince anybody with scientific training. The second is that the weight of the scientific evidence suggests that the principles underlying current mathematical matching algorithms—similarity and complementarity—cannot achieve any notable level of success in fostering long-term romantic compatibility.

It is not difficult to convince people unfamiliar with the scientific literature that a given person will, all else equal, be happier in a long-term relationship with a partner who is similar rather than dissimilar to them in terms of personality and values. Nor is it difficult to convince such people that opposites attract in certain crucial ways.

The problem is that relationship scientists have been investigating links between similarity, “complementarity” (opposite qualities), and marital well-being for the better part of a century, and little evidence supports the view that either of these principles—at least when assessed by characteristics that can be measured in surveys—predicts marital well-being. Indeed, a major meta-analytic review of the literature by Matthew Montoya and colleagues in 2008 demonstrates that the principles have virtually no impact on relationship quality. Similarly, a 23,000-person study by Portia Dyrenforth and colleagues in 2010 demonstrates that such principles account for approximately 0.5 percent of person-to-person differences in relationship well-being.

To be sure, relationship scientists have discovered a great deal about what makes some relationships more successful than others. For example, such scholars frequently videotape couples while the two partners discuss certain topics in their marriage, such as a recent conflict or important personal goals. Such scholars also frequently examine the impact of life circumstances, such as unemployment stress, infertility problems, a cancer diagnosis, or an attractive co-worker. Scientists can use such information about people’s interpersonal dynamics or their life circumstances to predict their long-term relationship well-being.

But algorithmic-matching sites exclude all such information from the algorithm because the only information those sites collect is based on individuals who have never encountered their potential partners (making it impossible to know how two possible partners interact) and who provide very little information relevant to their future life stresses (employment stability, drug abuse history, and the like).

So the question is this: Can online dating sites predict long-term relationship success based exclusively on information provided by individuals—without accounting for how two people interact or what their likely future life stressors will be? Well, if the question is whether such sites can determine which people are likely to be poor partners for almost anybody, then the answer is probably yes.

Indeed, it appears that eHarmony excludes certain people from their dating pool, leaving money on the table in the process, presumably because the algorithm concludes that such individuals are poor relationship material. Given the impressive state of research linking personality to relationship success, it is plausible that sites can develop an algorithm that successfully omits such individuals from the dating pool. As long as you’re not one of the omitted people, that is a worthwhile service.

But it is not the service that algorithmic-matching sites tend to tout about themselves. Rather, they claim that they can use their algorithm to find somebody uniquely compatible with you—more compatible with you than with other members of your sex. Based on the evidence available to date, there is no evidence in support of such claims and plenty of reason to be skeptical of them.

For millennia, people seeking to make a buck have claimed that they have unlocked the secrets of romantic compatibility, but none of them ever mustered compelling evidence in support of their claims. Unfortunately, that conclusion is equally true of algorithmic-matching sites.

Without doubt, in the months and years to come, the major sites and their advisors will generate reports that claim to provide evidence that the site-generated couples are happier and more stable than couples that met in another way. Maybe someday there will be a scientific report—with sufficient detail about a site’s algorithm-based matching and vetted through the best scientific peer process—that will provide scientific evidence that dating sites’ matching algorithms provide a superior way of finding a mate than simply selecting from a random pool of potential partners. For now, we can only conclude that finding a partner online is fundamentally different from meeting a partner in conventional offline venues, with some major advantages, but also some exasperating disadvantages.