Feature lists miss the point

So many people who should know better seem to miss the point when they mention Tableau. Why? I asked BI veteran Stephen McDaniel for his thoughts — which he gave, but then went on to suggest an almost unheard of challenge: a data analysis face-off among vendors.

Consider this description by a BI analyst: “Tableau provides business analysts speed of thought visual analysis on data held in memory on their desktop machines.” All that’s fine, but it may as well have been about a whole bunch of other tools, too.

At the root of this fuzz, explained McDaniel, is that most analysts who concern themselves with tools don’t actually use the tools. They rely on demos , marketing, and hearsay.

Though much of McDaniel’s recent work has centered on Tableau — his second book is Rapid Graphs with Tableau Software and he gives training sessions around the country — he also has a long, credible trail back through BI and data mining. He was director of analytics at Netflix, and has worked with more than 50 companies in BI. His first book was SAS for Dummies.

“I love SAS,” he says. Still, he remembers his sister in law’s reaction to his book on SAS. She was not an analyst but a “people manager.” These are the ones, he says, who have hated BI because “it had been made into a priesthood.” When she had looked through the book, she said, “Oh, this is great” and put it down. But she read the Tableau book for a half hour and said, “You should come talk to some people I work with.” She had recognized what she could do with the tool.

McDaniel’s sister in law and many like her don’t care whether the data is “in memory,” they don’t see themselves as business analysts, they take “desktop” for granted, and they know “speed of thought” is just gloss.

The list of features really doesn’t matter. All that really matters is whether someone can do what needs to be done with the tool.

McDaniel imagines a throw down, a data analysis match. It would be open to any BI vendor. Each vendor would send their best people, and each team would receive a uniform set of data. Over some defined period, teams would analyze and then present the results to a panel of vendor-neutral judges.

The reward? Perhaps a signed copy of a Stephen McDaniel book, or maybe a beer, possibly both. But certainly, repute.

What do you think of the face-off idea? Please write a comment.

10 Responses to Feature lists miss the point

  1. Ted,

    Excellent idea – Stephen McDaniel’s ideas always are. Perhaps you and some cronies could document the process of getting to the “results,” with each of you assigned to one vendor. Count me in to help in some way.

    MANY BLESSINGS!
    Peace and All Good!
    Michael W Cristiani

  2. Thanks Ted, I appreciate the work on your site. You drive home a frequently overlooked point in the business intelligence world; time is of the essence for many problems people hope to solve with BI tools. While there are apps that possess some of the key features found in Tableau, I sincerely believe that none of them can achieve exceptional insights in short order.

    Feature lists are what third-party analysts and vendor evaluation scorecards often rely upon in determining “winners” in product bake-offs. While I agree that features do matter, I have learned by working with many customers that it matters even more if they enable rapid insights that can quickly answer a stream of relevant questions. Tableau has done an exceptional job of achieving this feat, all achieved via a relentless focus on the analytic workflow and user perception of information.

    I would love to see multiple BI vendors accept this challenge. A third party will select two “real-world” datasets, from finance and marketing. Three business questions will be asked per dataset with just four hours to create our best solutions. This effort of each BI vendor analyst would be recorded, allowing the world to see what is truly possible in short order with these tools.
    The fruits of our labor would be posted to your site. You would ask several marketing and finance subject matter experts to review the entries. Of course, none of the judges would work for a vendor involved in this contest. The results will also offer an opportunity for the world to view actual results to the same question created in the same time frame.

    The winner would garner more than a copy of my book; they will have proven the worth of their product in a fair, open competition. A great prize indeed!

  3. I am not sure that having a vendor send their specialist would really prove the deeper value of the tool. As you say, most of the BI landscapre created a “priesthood” that did not make sense for most users. I would rather see a face-off of regular users who are all starting from scratch. Can they take an unfamiliar tool to answer a series of questions about the data set.

  4. Great idea, and I’d jump on the challenge.

    One caveat, however: the “uniform” set of data needs to be real-world….messy data typing, heterogeneously aggregated, disjoint hierarchies, undocumented, mixture of normalized and denormalized, dispersed across multiple silos/applications, too big for in-memory, etc.

    Just a suggestion, since we’re talking about real-world usefulness.

  5. Good idea, agree with Morgan. Besides that a test with regular users would make it more realistic, it would eliminate the need of convincing the vendors to participate. This way a selected set of tools could be judged independently from the vendors.

  6. I agree with Morgan Bates that a face-off between users would be much more useful: can users actually use the software? Is it intuitive? Can they quickly configure the reports they need for their business? Or will the tools drive them back to the de-facto tool of choice: Excel?

  7. Rather than judge the tool and the data analytics ability of the BI vendor, I would rather see the same group of analysts (not affiliated with a vendor) take on each BI tool and produce the best result they could with the same data.

    Combine this with user submitted entries (from the same dataset) and then I think you could begin to form some great conclusions.

  8. Great idea, but I see a slight problem with suggestion from Alex – one of the biggest advantages of Tableau (and probably a few other tools) is its speed. If analysts have all the time in the world and take a few months to set up and configure the infrastructure, then some bulky, expensive and slow tools can do some amazing things. But is it a fair comparison?

  9. […] Feature lists miss the point So many people who should know better seem to miss… […]

Leave a reply

Other recent posts

Qlik finally set to leapfrog Tableau?

Who’s your rival? I carelessly asked a Qlik person at the company’s annual analyst reception Monday night in Miami if she hadn’t once worked for Tableau. Her revulsion was immediate. “No! Never!,” she said. We smiled. There was so much more to talk about. For one thing, how will private equity change things? Qlik wasn’t… Continue Reading

Five Tips for Better Data Stories

Originally published on September 22, 2015 in BI This Week, a TDWI publication. A “data story” sounds like such a great idea. You just mix data with storytelling and you’re done — except that most data storytellers get one thing wrong: they drown out the story with data. Such storytellers, I believe, assume that audiences… Continue Reading

Bohemian Grove a la BI

The Bohemian Grove of the BI industry convenes for the fifteenth time in just three weeks. Naturally, you ask the obvious question: Are you serious? The Grove? A summit? The answer begins with a fond recollection of the Grove. If you’ve never attended the Bohemian Grove yourself — I haven’t, though I live in the… Continue Reading

Favorite Star Trek, a data story

This story shows how elemental data stories really are. Humans come ready to tell and hear them, requiring no plug-ins at all. This young person can do a good job of it. There was a question, followed by data, then questions and answers, and and finally a conclusion. It’s all there. It’s elementary. Sure, this… Continue Reading