Six surprises on my way from data to analysis

Originally published on November 29, 2011 in BI This Week, a TDWI publication.

What do you do when you have a bunch of data on your hands and your skills fail to make sense of it? That’s where I was the other day.

As much as I’ve written about data analysts, I’m not one myself. When, in late September, I finally closed my survey of analysts, I needed a pro to help interpret my new data. I was like the amateur cook who could flip eggs and found a side of beef in his kitchen. That was one meal I wasn’t going to cook alone.

It’s not easy to find the right guide. What skilled analyst has the time? Who cares about this kind of data? Who has the patience to answer my beginner’s questions? It’s a problem I suspect that many business people face.

I was lucky. I knew someone: He’s the expert’s expert in the Tableau Software community, the one I’m most familiar with. He’s renowned among fellow analysts for his expertise —and his generosity. He’s on the Tableau forum every day answering questions from users of all levels.

His name is Joe Mako and he is mostly self taught. Soon after two Army tours in Iraq, he went to work at a large Internet service provider managing documents. His first taste of data analysis came when his boss asked him to figure out if certain customers were getting a free ride —and with that Joe found a career and a passion.

My transition to data analyst was serendipitous.

Surprise #1. Even as bare numbers, data’s thrilling if it’s yours. It must be like coming across cave paintings. Someone had left them there, full of meaning that’s left behind to be revealed.

Surprise #2. Data analysis is harder than marketing makes it look. It looks so easy when the experts fling dimensions and measures around, dragging them on and off of shelves to make bar charts, scatterplots, and heat maps appear and reshape and altogether dazzle. (TV cooks make their acts look easy, too.)

Surprise #3. Data analysis doesn’t begin with analysis. It begins with data. I told Joe that I wanted to see the whole process from the start, so he first walked me through “the boring part”: data preparation. My export of the data from the service provider, Survey Monkey, was far from Tableau-ready. Multiple-choice questions posed special problems with their double row of headers.

Surprise #4. Data analysts have their own style. Other analysts, Joe acknowledged, would not go to the same lengths of data preparation. In fact, he suggested, some are careless.

Joe, for example, takes measures I think other analysts would skip. For example, instead of going from Excel straight to Tableau, he took the data through a third tool, Lyza. There he made the data more easily show who did not respond to all questions.

Once data was imported into Tableau, Joe checked the data once more. “I’m confirming that I have every possible combination of response ID and question ID available,” he explained deliberately and calmly. “That’s how I get my high degree of flexibility with Tableau.”

Surprise #5. The analyst’s personal interest matters. Within the first few minutes, I came to appreciate Joe’s passionate interest in this data.

One question in my survey asked about the relative importance of statistics. I wanted to know whether beginners would rate its importance lower than experienced analysts. Some marketing seems to suggest that it’s optional. In the results, however, only a few rated it unimportant.

Still, Joe didn’t like to see even those few. He didn’t say anything for a few seconds after the stacked bar chart appeared. His cursor ceased the usual circular movement that preceded leaps into new views. “That throws me for a loop,” he said. “It just seems strange to me that someone would say that statistics is unimportant.”

“Why don’t we compare these guys with someone else?” I said, “or see what skills they do value.”

“Let’s see,” he said, enunciating with delicious anticipation. These respondents would now either save themselves or prove themselves idiots. His cursor circled busily again, and he flipped through a new succession of views and dialog boxes.

Surprise #6. Some data’s a dud. We found little about respondents who judged statistics of low importance. We couldn’t explain it except to assume that they had understood the question differently from others. I had written a faulty question.

The next questionnaire will be better. What may surprise me, though, is to find an expert in questionnaire design as generous with his time as Joe is with his data analysis.

Leave a reply

Other recent posts

End of one-size-fits-all data stories

This appeared originally on the TDWI site in September behind a paywall. It’s still there, but today they’ve had the 90 days of exclusive use that I agreed to. Survey after survey reveals that about 80 percent of business users don’t use data analysis—despite all the marketing and “easy to use” tools. As if in… Continue Reading

Qlik finally set to leapfrog Tableau?

Who’s your rival? I carelessly asked a Qlik person at the company’s annual analyst reception Monday night in Miami if she hadn’t once worked for Tableau. Her revulsion was immediate. “No! Never!,” she said. We smiled. There was so much more to talk about. For one thing, how will private equity change things? Qlik wasn’t… Continue Reading

Five Tips for Better Data Stories

Originally published on September 22, 2015 in BI This Week, a TDWI publication. A “data story” sounds like such a great idea. You just mix data with storytelling and you’re done — except that most data storytellers get one thing wrong: they drown out the story with data. Such storytellers, I believe, assume that audiences… Continue Reading

Bohemian Grove a la BI

The Bohemian Grove of the BI industry convenes for the fifteenth time in just three weeks. Naturally, you ask the obvious question: Are you serious? The Grove? A summit? The answer begins with a fond recollection of the Grove. If you’ve never attended the Bohemian Grove yourself — I haven’t, though I live in the… Continue Reading