Six surprises on my way from data to analysis

Originally published on November 29, 2011 in BI This Week, a TDWI publication.

What do you do when you have a bunch of data on your hands and your skills fail to make sense of it? That’s where I was the other day.

As much as I’ve written about data analysts, I’m not one myself. When, in late September, I finally closed my survey of analysts, I needed a pro to help interpret my new data. I was like the amateur cook who could flip eggs and found a side of beef in his kitchen. That was one meal I wasn’t going to cook alone.

It’s not easy to find the right guide. What skilled analyst has the time? Who cares about this kind of data? Who has the patience to answer my beginner’s questions? It’s a problem I suspect that many business people face.

I was lucky. I knew someone: He’s the expert’s expert in the Tableau Software community, the one I’m most familiar with. He’s renowned among fellow analysts for his expertise —and his generosity. He’s on the Tableau forum every day answering questions from users of all levels.

His name is Joe Mako and he is mostly self taught. Soon after two Army tours in Iraq, he went to work at a large Internet service provider managing documents. His first taste of data analysis came when his boss asked him to figure out if certain customers were getting a free ride —and with that Joe found a career and a passion.

My transition to data analyst was serendipitous.

Surprise #1. Even as bare numbers, data’s thrilling if it’s yours. It must be like coming across cave paintings. Someone had left them there, full of meaning that’s left behind to be revealed.

Surprise #2. Data analysis is harder than marketing makes it look. It looks so easy when the experts fling dimensions and measures around, dragging them on and off of shelves to make bar charts, scatterplots, and heat maps appear and reshape and altogether dazzle. (TV cooks make their acts look easy, too.)

Surprise #3. Data analysis doesn’t begin with analysis. It begins with data. I told Joe that I wanted to see the whole process from the start, so he first walked me through “the boring part”: data preparation. My export of the data from the service provider, Survey Monkey, was far from Tableau-ready. Multiple-choice questions posed special problems with their double row of headers.

Surprise #4. Data analysts have their own style. Other analysts, Joe acknowledged, would not go to the same lengths of data preparation. In fact, he suggested, some are careless.

Joe, for example, takes measures I think other analysts would skip. For example, instead of going from Excel straight to Tableau, he took the data through a third tool, Lyza. There he made the data more easily show who did not respond to all questions.

Once data was imported into Tableau, Joe checked the data once more. “I’m confirming that I have every possible combination of response ID and question ID available,” he explained deliberately and calmly. “That’s how I get my high degree of flexibility with Tableau.”

Surprise #5. The analyst’s personal interest matters. Within the first few minutes, I came to appreciate Joe’s passionate interest in this data.

One question in my survey asked about the relative importance of statistics. I wanted to know whether beginners would rate its importance lower than experienced analysts. Some marketing seems to suggest that it’s optional. In the results, however, only a few rated it unimportant.

Still, Joe didn’t like to see even those few. He didn’t say anything for a few seconds after the stacked bar chart appeared. His cursor ceased the usual circular movement that preceded leaps into new views. “That throws me for a loop,” he said. “It just seems strange to me that someone would say that statistics is unimportant.”

“Why don’t we compare these guys with someone else?” I said, “or see what skills they do value.”

“Let’s see,” he said, enunciating with delicious anticipation. These respondents would now either save themselves or prove themselves idiots. His cursor circled busily again, and he flipped through a new succession of views and dialog boxes.

Surprise #6. Some data’s a dud. We found little about respondents who judged statistics of low importance. We couldn’t explain it except to assume that they had understood the question differently from others. I had written a faulty question.

The next questionnaire will be better. What may surprise me, though, is to find an expert in questionnaire design as generous with his time as Joe is with his data analysis.

Other recent posts

How to find a story in data: What a news reporter would do

Originally published on December 15, 2015 in BI This Week, a TDWI publication. A data analyst raised her hand in a class I taught on data storytelling and asked the question I hadn’t even thought about since journalism school: How do you “see” a story in a jumble of facts? It’s a novel problem for… Continue Reading

Bohemian Grove a la BI

The Bohemian Grove of the BI industry convenes for the fifteenth time in just three weeks. Naturally, you ask the obvious question: Are you serious? The Grove? A summit? The answer begins with a fond recollection of the Grove. If you’ve never attended the Bohemian Grove yourself — I haven’t, though I live in the… Continue Reading

Getting to less: executive-level storytelling at Schwab

A key part of telling a story with data, says a Schwab executive I talked to last week, is thinking like a business person. “You want to go for the high-level ‘aha!’” says John F. Carter, Schwab senior vice president of analytics and business insight, in San Francisco. Start the main conclusion, similar to news… Continue Reading

Data driven? No, you’re story driven, says German data scientist

Just when data seems to be approaching maturity as a touchstone in business, along comes a data scientist with doubts. Data is like the stone in stone soup, he says. It has not a single calorie of meaning. “The beef is in data fiction,” says Joerg Blumtritt, CEO of Munich-based Datarella and consultant to a… Continue Reading

Scenarios for a Qlik Sense data story

Real data storytellers know that data framed in a story is much more effective than data standing all alone by its naked self. They also know that storytelling is a giant that can lose much of its power when tied down to a vendor’s product. We can applaud Qlik for respecting these facts. We can… Continue Reading

Favorite Star Trek, a data story

This story shows how elemental data stories really are. Humans come ready to tell and hear them, requiring no plug-ins at all. This young person can do a good job of it. There was a question, followed by data, then questions and answers, and and finally a conclusion. It’s all there. It’s elementary. Sure, this… Continue Reading