For many working in digital news outside of the top-tier enterprises, analyzing metrics is a do-it-yourself project. This is a brief primer — admittedly not an in-depth tutorial — on five pitfalls:
1. Focusing on the greatest hit. The tallest bar on a bar chart may be a valuable clue for a marketer who wants to know which product is hot. But what do news publishers learn by seeing their coverage of a celebrity death was off the charts? Of course everyone was interested, but it’s not like you can replicate the story by ordering up another Michael Jackson death. If you recognize that the biggest hit is an outlier, you’ll either need to ignore it or mine it deeper: Does the geographic origin or referral source of this wave of drive-by traffic provide clues to your potential audience that your regular coverage is not capturing? If you did some follow-up stories to the big hit, which ones worked and which segments of the original audience came back for more?2. Automating display. Tools that use real-time metrics to automate display can be tempting for a short-staffed website. What could go wrong if the most popular items are automatically given the most prominence? This is a recipe for more traffic, more revenue, and horrifying unintended consequences. A silly item could climb to the top of the homepage because readers are linking to it on Facebook in a discussion about whether it is offensive or inaccurate. Have you ever laughed at how Facebook decided your news feed or ad interests, or how YouTube’s “Recommended channel for you” missed the mark? Every data-driven formula works differently, but assuming you wield a lot less data than Facebook or YouTube, you’re bound to get some brand-tarnishing displays.
3. False equivalency. To create a user-friendly dashboard, some analytics providers rank content by number of clicks or video streams as though each content item is competing against all the others of that same type on the site. In fact, all the content on your site is competing with everything on the internet that is a click or two away. A story that typically would draw a relatively small base audience may be important to the site’s mission and its retention of loyal return visitors or growth of paid subscriptions. It’s important to understand what’s working in those stories to avoid wasting time and money on low-margin content. In other words, it doesn’t matter if the story about the puppy being saved beat the story about the dumbest cat ever for first place, but let’s find out why there is suddenly a town council story in the Top 25 if that never happens.
4. Misunderstanding engagement. Once you have gotten some insight from a particular metric or set of metrics, it’s tempting to give it too much weight going forward. Imagine that your community of readers has somehow gotten into the habit of sharing all your stories about the town council — because of its heroic action to save puppies earlier this year. That issue is over, and few are reading the stories now but they persist in sharing, burnishing their civic-minded, puppy-loving image among friends. That sounds far-fetched but a well-publicized Chartbeat study of 2 billion pageviews on 2,000 sites found that a majority of clicks don’t result in significant reading time, and those sharing stories on social media are not more likely to have read through them than the non-sharers.
5. Tuning out completely. Perhaps your digital metrics seem obvious or suspicious. They may be baffling because of coding errors on your site or interference from the bots that infest the web. You can easily find someone to tell you that the system you are using is useless or even inadequate — they will sell you a better one. But it would be a mistake to ignore or write off whatever data you have. The ability to have real-time audience measurement and quantification of readership and viewership is too valuable to give up.