A Blog by Jonathan Low

 

Oct 16, 2012

They're Just Not That Into You: Why Measuring Social User Engagement is Hard

Tracking traffic can become an obsession.

Figuring out how many people are following you or your enterprise online can seem like personal validation with a stock ticker attached. But as many social media platforms and the advertisers whose business they covet are discovering, a lot of traffic, as much as 70% by some estimates, is random.

So, while every web-based enterprise would love to think of themselves as the digital equivalent of rock stars, the reality is that the entire process is far more opportunistic than intentional. Which is worrisome to those whose budgets and job security are dependent on corporate systems engineered to take the guess-work out of returns on investment. And whether or not that is even a useful measure in this context, it remains the reigning business justification for success or failure.

The devilishly complicated sources and uses of content, context and use thwart efforts to do what companies do best: identify, measure, compare and manage. Especially when it comes to data.

But within this maze characterized by distraction and misinterpretation lies an opportunity. It requires the same mindset as jujitsu; dont fight the data, acknowledge its power and follow its direction. Those who do so can then use it more effectively than those who try to jam it into pre-assigned roles that may turn out to be more misleading than illuminating.

All of which is to say that there is not 'an' or 'the' answer. Yet. But there may be glimmers of understanding that over time will become the basis for greater assurance. JL

Mathew Ingram reports in GigaOm:
“This vast trove of social traffic is essentially invisible to most analytics programs. I call it DARK SOCIAL." If you’ve spent any time in a newsroom, traditional or otherwise, you know that publishers are obsessed with measuring where their Web traffic comes from. Whether it’s Google (GOOG) Analytics or Chartbeat, or comScore, or Omniture, or any one of a dozen other providers, tracking where readers come from is a crucial part of online media—mostly because publishers need to know which channels are worth focusing on, since there are so many to choose from. Is Twitter your biggest source? Then you should tweet more and optimize your content for Twitter. Is Facebook (FB) a big referrer of traffic? Then you need to be aware of changes to the newsfeed and how they affect you.

But what if your biggest source of traffic and readers is something you aren’t even really paying attention to, and something that is extremely hard to track in the same way as Google or Twitter or Facebook? That’s the reality of Web publishing today, according to Alexis Madrigal at the Atlantic—who writes about the influence of what he calls “dark social” on engagement and traffic patterns. While everyone is busy watching Twitter and Facebook because they are easy to track, Madrigal argues that most social traffic still comes from old-fashioned or difficult-to-track sources such as e-mail and chat messages:

It shows up variously in programs as ‘direct’ or ‘typed/bookmarked’ traffic, which implies to many site owners that you actually have a bookmark or typed www.theatlantic.com into your browser,” writes Madrigal. “But that’s not actually what’s happening a lot of the time. Most of the time, someone Gchatted someone a link, or it came in on a big e-mail distribution list, or your dad sent it to you.”

As evidence, Madrigal provides some data from Chartbeat, the Betaworks spin-off that focuses on real-time analytics for publishers, looking at everything from the amount of time readers spend on a page to how far down they got in an article before they decided to click away. Chartbeat, which we have written about a number of times, is one of the few analytics engines that tries to break down that “direct” category into sub-categories, such as e-mail, and what the service calls “direct social”—meaning everything from apps (for chat or other social features) to instant messaging.

For the Atlantic, the impact of this kind of direct social traffic outweighs any other kind of social network or service, such as Facebook, Twitter, or Reddit: According to data from Chartbeat, the magazine’s website gets almost 60 percent of its social traffic from these hard-to-track sources. Facebook is still a large referrer for the site, generating about 21 percent of the social traffic, and Twitter is also fairly large at 11 percent—but the “dark social” category is larger than all the other social services combined and has more than twice the impact that Facebook does.

Chartbeat’s numbers also showed that this direct-social traffic was a large contributor for other sites that the service tracks, according to Madrigal—and Chartbeat is used by some of the largest publishers in the media business, including ESPN and the New York Times. On average, the company’s stats showed that close to 70 percent of the social traffic to these sites came from e-mail, instant messaging, chat apps, and other sources. (As Madrigal points out, at many websites, including the Atlantic, social traffic far outweighs traffic that comes from search, and that gap is still growing.)

The one obvious conclusion to take away from all of this is that measuring user engagement and sources of traffic is probably a lot harder than most publishers think—and they likely already thought it was pretty hard. It’s bad enough that comScore and Compete and Nielsen (NLSN) and Google Analytics all provide different numbers, and it’s almost impossible to tell who is right (especially since all these sources often disagree with a publisher’s internal statistics). Now there is a huge source of traffic that is even harder to measure: E-mail is trackable in the aggregate, but how do you track instant messaging?

This problem is compounded by the shift to mobile content consumption, since chat apps and instant messaging and other direct communication methods are even more prevalent in the mobile world than on the desktop. Links are passed from social network to apps to chat to e-mail, and tracking them quite quickly becomes almost impossible. That’s part of the reason why almost all Web publishers get surprised by posts or stories that blow up traffic-wise days or weeks after they first appeared, with no obvious sign of how or why they hit that invisible tipping point.

So how are publishers and media companies supposed to deal with this problem? Madrigal’s solution is an appealing one, at least for those who create content—he says the only dependable way of generating real traffic and engagement is actually to write things that people care about or are interested in. In other words, the “content is king” approach. As he describes it:

“The only real way to optimize for social spread is in the nature of the content itself. There’s no way to game e-mail or people’s instant messages. There’s no power users you can contact. There’s no algorithms to understand.”

It’s not that getting people to share content via e-mail or chat app or instant messaging is that different from trying to get them to share it on Twitter or Facebook—the same general rules apply, in the sense that it has to be engaging and interesting and shareable, and all those other things we are supposed to be doing with our content. The difficult part is that it’s hard to track in the same way publishers watch Facebook “likes” or page followers, or Twitter retweets and other metrics. So it’s difficult to tell whether it’s working and why, or what you should do differently.

In a sense, what Madrigal is describing just reinforces the fact that much of what content companies do is more of an art than a science—even though social-media gurus and analytics providers would like to make it sound like something that can be quantified and measured from every aspect. And maybe that’s not such a bad thing, even if it does make our jobs harder.

0 comments:

Post a Comment