Good metric-making aims for the concrete and sensory

If you want to come up with effective metrics, forget brainstorming. Drop the creativity. Done well, this is an analytical exercise, says Stacey Barr, and it should aim at deriving concrete, sensory effects to measure.

She’s “the performance measure specialist,” and she lives in Brisbane, Australia. Stephen Few, the BI industry’s leading critic of performance dashboards, refers to her when he’s asked about performance metrics.

I talked to her yesterday as part of my research for my next BI This Week story.

People in search of metrics jump too soon into measuring, Stacey says. One of the first things she does for clients is to insert one critical step: defining what effect they are looking for—and then describing that effect in “sensory specific language.”

She finds that people’s first attempts use “fluffy, vague language to describe those results.” Prime examples: “quality,” “efficient,” “effective,” “sustainable,” “enhanced.” You’ve heard them all before.

“It’s this habit we’ve gotten into of using words that mean seven different things to three different people,” she says. “We’ve got to use words that are more concrete.”

When people are all sitting in a room talking about goals and results, they have the same images in their heads.

“It makes it much easier to measure,” she says. Concreteness is the key.

I suppose that making metrics is like making a movie. No image or sound can tell what a character thinks, feels or intends to do. He has to show it somehow. This concreteness is also what Jon Franklin calls for in his book Writing for Story: Craft Secrets of Dramatic Nonfiction.

Stacey once worked with a local council (to Californians, a county board of supervisors) to raise public participation in meetings. They had been measuring participation with the number of meetings held. That is, more meetings automatically counted as more participation.

More vague words: what does engagement or participation look like?

She asked them, “If the community were more engaged, what would people be doing that they’re not doing now?” At first they said things like more people showing up, a high proportion speaking up, more ideas proposed, and so on.

Eventually they came up with two measures that worked together.

  • The number of “fresh faces,” those who’d never attended before. They had decided to give it a try.
  • The number of familiar faces, those who’d come to at least half of all recent meetings. They had decided that showing up was worthwhile.

It works.

Come to think of it, doesn’t the need for concrete, sensory effects sound like storytelling?

Filed Under: Uncategorized

One Response to Good metric-making aims for the concrete and sensory

Leave a reply

Other recent posts

Qlik finally set to leapfrog Tableau?

Who’s your rival? I carelessly asked a Qlik person at the company’s annual analyst reception Monday night in Miami if she hadn’t once worked for Tableau. Her revulsion was immediate. “No! Never!,” she said. We smiled. There was so much more to talk about. For one thing, how will private equity change things? Qlik wasn’t… Continue Reading

Five Tips for Better Data Stories

Originally published on September 22, 2015 in BI This Week, a TDWI publication. A “data story” sounds like such a great idea. You just mix data with storytelling and you’re done — except that most data storytellers get one thing wrong: they drown out the story with data. Such storytellers, I believe, assume that audiences… Continue Reading

Bohemian Grove a la BI

The Bohemian Grove of the BI industry convenes for the fifteenth time in just three weeks. Naturally, you ask the obvious question: Are you serious? The Grove? A summit? The answer begins with a fond recollection of the Grove. If you’ve never attended the Bohemian Grove yourself — I haven’t, though I live in the… Continue Reading

Favorite Star Trek, a data story

This story shows how elemental data stories really are. Humans come ready to tell and hear them, requiring no plug-ins at all. This young person can do a good job of it. There was a question, followed by data, then questions and answers, and and finally a conclusion. It’s all there. It’s elementary. Sure, this… Continue Reading