Month: May 2015

Google guys come to shake up BI with natural language

Kindergarten may have taught you all you need to know about life. But you may need to watch “Mr. Peabody and His Boy Sherman,” an animated, 1960s-era TV series for kids, to truly appreciate an interesting new natural-language product called ThoughtSpot.

ThoughtSpot’s natural-language querying represents a new stage of maturity for casual BI users — a step up even from visualization, which was a step up from rows and columns. ThoughtSpot users get data with natural language queries, otherwise known as questions.

ThoughtSpot opens with a Google-like interface. In a demo, a search for “total revenue last year” and visualized data appeared quickly with a simple dollar amount and, below it, bar charts with some detail. A followup query asked for a breakdown by age and gender, and a line chart showed the total revenue broken down that way. During what period? The query “last year quarterly” yielded yet more detail.

As in Google, queries type ahead based on past queries by others. And not just any others; the tool learns from the group. “Revenue in California,” for example, will show up first for one group while, say, “Revenue in Caledonia” will show up for another.

Inevitably, I suppose that many business users will try to do more analysis than they’re capable of. I can’t tell how deep ThoughtSpot will take them. But suppose they get into a problem and suppose expert help eventually arrives.

This is where I imagine that Mr. Peabody appears. In the TV cartoon, he was a brilliant, geeky dog, and Sherman was his pet boy. Sherman always had the interesting, pertinent questions. Mr. Peabody had enigmatic answers. “But Mr. Peabody,” Sherman often began, “why did they call it ThoughtSpot?” Mr. Peabody often gave a reply like this: “It’s elementary, my dear Sherman. It’s ‘thought’ for ‘thought’ and ‘spot’ for ‘spot.’ ThoughtSpot!” For me, they offer an allegory with data scientists and their business users.

“How did you get those numbers?,” the Mr. Peabody type might ask. To show him, the ThoughtSpot user hovers the cursor over a search term to reveal the identity of the table and column supplying that data. If that’s not good enough, a little window headed with “What am I looking at?” explains in a sentence.

The two ThoughtSpot representatives who conducted the demo for me proudly told about one such encounter in which the Peabody realized he’d been using incorrect data.

In the old TV cartoon, the dog is always smarter than the smartest human. Peabody and Sherman routinely travel back in time to render help at precisely the moment necessary to let such figures as Albert Einstein discover relativity or to align an apple to fall on Isaac Newton’s head.

If only His Boy Sherman could have sent the WABAC (wayback) Machine forward, instead of backward. Perhaps then he would have found a tool to let his innocent, unfiltered questions iterate toward insight with just natural language.

True, many of the world’s Shermans have already found that tools like Tableau and QlikSense let them iterate through questions and answers. But even such easy to use viz tools still don’t work for many without the patience or self-confidence to learn.

ThoughtSpot, say the two people who showed it to me, connects to just about any source of data, on the cloud or on premises, including Hadoop. It creates its own in-memory relational cache, though it doesn’t create its own aggregations. It makes an index of the data while retaining the original schema, joins, cardinality, etc.

If ThoughtSpot resembles Google, it should. That’s where four of the seven founders came from.

“We’re not a bunch of BI guys trying to bolt search onto BI,” said vice president of marketing Scott Holden. “We’re a bunch of search guys trying to reinvent BI.” They just might do it, even without a Mr. Peabody to swoop down from the future with his helpful paw. ThoughtSpot has that rumble and hiss of an invention about to break through the BI industry’s frontier. It doesn’t seem intended to replace heavy duty data analysis, at least not for now. But it does look like the easiest entry so far — lowering the ramp just enough for critical first steps, and maybe much more.

In fact, possibly the most important implication of the Google-like interface probably seemed too obvious to mention: Using it takes no training. Nearly anyone can do simple data analysis immediately.

Sherman might ask, “But Mr. Peabody, If I can do this, why do I need you?” I’ll bet that the smartest of the Mr. Peabodys would have thought they’d never see the day. Ah, liberation! They might be overjoyed to just fetch the data and to come when called. Natural balance will have been restored.

Lending a hand to would-be storytellers

Advocating data storytelling is easy. But actually making a data story out of data is something else. As much as storytelling could help to deliver meaning, says one consultant who works up close to the action, many data analysts just don’t even know where to start.


“If no one teaches you how to tell a story, how do you do it?”says DecisionViz president Lee Feinberg. How do you expect them to tell a story in words or visualizations? “Face it, most of them aren’t even good at writing or English.” What might be hard even for natural storytellers is making a set of reports into something meaningful.

In one project, an advertising agency with mounds of data had to convince a client that spending money with them was worthwhile, and to make recommendations on expenditures. The agency wanted to show media buy did well or did badly. What went up or went down? What’s it mean? Why did it happen? They know they need to do it, but they just don’t know how. He helps them through it.

“In their gut, they knew that what they should be doing was not reporting,” he says. But if they issue reports without having thought through to the final message, they actually spend more time in the end after reports are sent back and have to be done over.

He offers analysts a structure and a methodology. He shows them how to break findings into pieces.

Do organizations need roving storytellers? I imagined a thin old guy with long, graying hair and a wizened face wandering among the cubicles and speaking up at crucial meetings. Would such a specialist help? Feinberg, perhaps humoring me, said it might.

Learning from earthquake relief to design BI tools

Rescue dog. Humble, helpful.
Rescue dog. Humble, helpful.

You might say I’m crazy to see any connection between some big IT deployments and typical responses to big natural disasters — but that’s what I see. It fits a recurring theme across many disciplines of big interventions versus smaller, more humane and often more effective effort.

Yesterday on the PRI program The World, Associated Press reporter Jonathan Katz, author of The Big Truck That Went By: How the World Came to Save Haiti and Left Behind a Disaster, talked about what he saw in Haiti.

There was one mistake that was made over and over in Haiti: The people of Haiti were looked at as being bystanders, as obstacles or as security threats. It was thought that they needed to be put aside and that things needed to be done to them and for them — instead of with them or, even more importantly, following their lead. The decision to look at Haitians as obstacles in their own recovery led to creating these tent camps that ended up being an enormous focus of the relief and reconstruction effort for years to come.

The decision not to take seriously people’s needs during recovery. … They make the decision to pour in food instead of working with them to find out where they were getting their food before. The would-be helpers fail to ask, “How could we help you get food now? Is the problem that you have no income now to buy food? Were you growing food; could we help you grow it again?”

“How can we help?” shows a kind of humility that seems to underlie many of the tools I prefer, starting with QlikSense and Tableau.