Andrew Spittle

Greg Linch led a session in the morning at BCNI Philadelphia about how we can define new metrics for journalism. It was prompted by the similar post he wrote earlier this year.

To help we need to look at the metrics other industries are using to define success in their work. Greg’s idea stems from Alexis Madrigal’s notion that you have to define your own metrics. You can find influence from others but don’t just adopt their method.

We have methods for measuring how many people see our work and how long they spend on a site. But, we don’t have solid metrics for measuring the impact of journalism. As Greg said, “What does a pageview mean?” The metrics we currently have mean things. They tell us how many, how long, and other aspects of our work’s impact. But they are largely one-dimensional and don’t tell the whole story.

Current approaches

Altmetrics.org was mentioned as one different approach. As their site says:

Altmetrics expand our view of what impact looks like, but also of what’s making the impact.

Sabermetrics was the focus of the Moneyball movie and is a similar approach in sports to redefining the metrics of an industry. Things like VORP and OPS bring new light to the performance of athletes who may appear successful by traditional metrics.

Propensity score matching is yet another approach. Tries to answer the questions of, “What’s the possibility of X to happen based on Y?”

Greg also metioned the way the Cleveland Orchestra measures performance. They strive toward superior performance and measure it by standing ovations, ticket demand, and more.

Someone in the audience mentioned the way Gawker rotates staff. Nieman recently wrote a bit about this. For a different way of rotating staff check out how The Economist does it.

Ideas

Someone mentioned that one way to approach this is asking readers the question “Do you understand this issue better after reading this article?”

I brought up the way KISSmetrics approaches site stats from a user-specific perspective. For journalism that could be looking at how many people read the first story your publish on an issue and all of the followups. You could also measure how many jump from one report you’ve published to another one on a different topic.

Greg mentioned the net promoter score that many organizations use to measure the loyalty and reputation of a company or a product. One person in the room works at an organization which uses this on a per-brand basis. They separate the content creation from the content display. Each coverage team is given a net promoter score as well as each product or brand their content is flowing through.

The concern over not being able to measure impact in communities that are not online was also raised. Ultimately the way to fix this is to get out in the communities and talk with people about whether they understand issues and how they’d like to learn more.

Albert Sun said that part of why metrics are so confusing is because they are traditionally geared toward what advertisers want. The predominant approach is, “All metrics are good but we need to make money off of them.” The business side is failing, though, and what we need to do is create a new, better model. What we need is a way to measure the difference and change our journalism causes in the world.

If anyone has more to add feel free to leave your own notes as a comment.