Good metric-making aims for the concrete and sensory

If you want to come up with effective metrics, forget brainstorming. Drop the creativity. Done well, this is an analytical exercise, says Stacey Barr, and it should aim at deriving concrete, sensory effects to measure.

She’s “the performance measure specialist,” and she lives in Brisbane, Australia. Stephen Few, the BI industry’s leading critic of performance dashboards, refers to her when he’s asked about performance metrics.

I talked to her yesterday as part of my research for my next BI This Week story.

People in search of metrics jump too soon into measuring, Stacey says. One of the first things she does for clients is to insert one critical step: defining what effect they are looking for—and then describing that effect in “sensory specific language.”

She finds that people’s first attempts use “fluffy, vague language to describe those results.” Prime examples: “quality,” “efficient,” “effective,” “sustainable,” “enhanced.” You’ve heard them all before.

“It’s this habit we’ve gotten into of using words that mean seven different things to three different people,” she says. “We’ve got to use words that are more concrete.”

When people are all sitting in a room talking about goals and results, they have the same images in their heads.

“It makes it much easier to measure,” she says. Concreteness is the key.

I suppose that making metrics is like making a movie. No image or sound can tell what a character thinks, feels or intends to do. He has to show it somehow. This concreteness is also what Jon Franklin calls for in his book Writing for Story: Craft Secrets of Dramatic Nonfiction.

Stacey once worked with a local council (to Californians, a county board of supervisors) to raise public participation in meetings. They had been measuring participation with the number of meetings held. That is, more meetings automatically counted as more participation.

More vague words: what does engagement or participation look like?

She asked them, “If the community were more engaged, what would people be doing that they’re not doing now?” At first they said things like more people showing up, a high proportion speaking up, more ideas proposed, and so on.

Eventually they came up with two measures that worked together.

  • The number of “fresh faces,” those who’d never attended before. They had decided to give it a try.
  • The number of familiar faces, those who’d come to at least half of all recent meetings. They had decided that showing up was worthwhile.

It works.

Come to think of it, doesn’t the need for concrete, sensory effects sound like storytelling?

Filed Under: Uncategorized

One Response to Good metric-making aims for the concrete and sensory

The data industry thrives on conversation. Please submit a comment.

Other recent posts

Bohemian Grove a la BI

The Bohemian Grove of the BI industry convenes for the fifteenth time in just three weeks. Naturally, you ask the obvious question: Are you serious? The Grove? A summit? The answer begins with a fond recollection of the Grove. If you’ve never attended the Bohemian Grove yourself — I haven’t, though I live in the… Continue Reading

Favorite Star Trek, a data story

This story shows how elemental data stories really are. Humans come ready to tell and hear them, requiring no plug-ins at all. This young person can do a good job of it. There was a question, followed by data, then questions and answers, and and finally a conclusion. It’s all there. It’s elementary. Sure, this… Continue Reading

Bad stories stop good data at the water cooler

We agree by now that data’s a good compass. One neglected question is tougher: Which map? Everyone’s known the kind of “grouchy guy” TDWI instructor Kellee M. Franklin, Ph.D tells about. This guy knew better than most of his co-workers about how their Washington, D.C. defense agency worked. And he was frustrated. Over the years,… Continue Reading