Home » innovation

Category: innovation

Learning from earthquake relief to design BI tools

Rescue dog. Humble, helpful.
Rescue dog. Humble, helpful.

You might say I’m crazy to see any connection between some big IT deployments and typical responses to big natural disasters — but that’s what I see. It fits a recurring theme across many disciplines of big interventions versus smaller, more humane and often more effective effort.

Yesterday on the PRI program The World, Associated Press reporter Jonathan Katz, author of The Big Truck That Went By: How the World Came to Save Haiti and Left Behind a Disaster, talked about what he saw in Haiti.

There was one mistake that was made over and over in Haiti: The people of Haiti were looked at as being bystanders, as obstacles or as security threats. It was thought that they needed to be put aside and that things needed to be done to them and for them — instead of with them or, even more importantly, following their lead. The decision to look at Haitians as obstacles in their own recovery led to creating these tent camps that ended up being an enormous focus of the relief and reconstruction effort for years to come.

The decision not to take seriously people’s needs during recovery. … They make the decision to pour in food instead of working with them to find out where they were getting their food before. The would-be helpers fail to ask, “How could we help you get food now? Is the problem that you have no income now to buy food? Were you growing food; could we help you grow it again?”

“How can we help?” shows a kind of humility that seems to underlie many of the tools I prefer, starting with QlikSense and Tableau.

Data surfing with big data

The usual big-data story leaves out crucial bits. We hear about the “what” — big, huge data of all kinds. We hear about the “when” — now and coming soon. We hear about the “how” — Hadoop with helpers. But we almost never hear about the “who” and the “why.” Who’s bothering to analyze all this data, and why?

Read more

No dashboard, just an ironing board

As if to renounce one more convention of business, a Berkeley-area businessperson I know couldn’t find data, so he went out and got some of his own. He had to evaluate market areas.

For three days, he stood by an ironing board with a map on top in front of the Cheese Board in Berkeley’s Gourmet Ghetto. Each customer coming in or out got handed two stick-on dots, red to mark home and blue to mark work. His map quickly revealed patterns and commute routes.

It was easy. “Everybody’s curious,” said Terry Baird, who had made a career of cooperative/collective food retailing. He had sold advertising for 10 years, and later found himself appointed to unravel the bankruptcy of a once-huge food distributor, an experience he calls “my MBA.” He approached it all without training, only logic and nerve, such as with the ironing board. “It’s like running a three-card monte. A group gathers around you.”

In 1996, he was part of a newly formed collective inspired by the Cheese Board. The new collective trained there as they decided on the new store’s location.

Terry’s question: would any of the proposed locations for a new store overlap another store’s market area?

In three days, he had the answer. On weekdays, none overlapped. But on Saturdays, all did; that’s when people drove in from all over the Bay Area.

The best software might be the kind made without starch.

Put a mobile device on your dashboard

What if you could snap an iPad into your car’s dashboard to let the device listen to your car’s murmurs? Perhaps it could receive its maintenance-alert emails, the ones that let you know when it’s ready for an oil change or new seat covers. If you also kept your calendar on the iPad — and who wouldn’t? — your iPad could schedule a date with the mechanic.

Howard Dresner, the man who revived the term “business intelligence,” is now excited about mobile devices. This fall, he issued a study. And though I have been ambivalent — and who hasn’t been? — I like the science fiction-like joy of dreaming about the possibilities. We talked on the phone last week, and his enthusiasm was infectious.

Just think, your iPad (or, if you must, your Android device) knows where you car is parked at night, doesn’t it? It does if you turned on your GPS, and who hasn’t? Your maintenance could be run the way airlines do it — overnight, by unseen mechanics, wherever the airliner or car happens to be. So long, courtesy shuttles! Just ignore the sounds coming from your driveway at 4 a.m. It wouldn’t be who it used to be.

Look, Ma. No ETL

One of the first things you learn about in business intelligence is ETL. Raw data gets harvested, washed and served. But Sandy Steier hadn’t heard.

Sandy had been busy analyzing data. For years on Wall Street, he pored over mortgage-backed securities with a tool he and peers developed for themselves.

He only learned of ETL recently. He’d become acquainted with a data architect with whom he shared a bus ride every day to and from their offices in downtown Manhattan. “I had never really spoken to him before,” Sandy recalls. “He was in a different world even though we both dealt with data.”

Sandy described to him his rapidly maturing tool. As I imagine the scene, the calm data architect suddenly twisted himself on the cramped bus seat to face Sandy. “You don’t do ETL? You work with raw data??”

No, he didn’t do any ETL, Sandy explained. “We didn’t realize how important that was,” he recalled. “We had always just stuck the raw data into the database and then realized, ‘Hey, this data’s a mess.'” He instructed users to clean it themselves. “You get the data from the horse’s mouth. You’re the expert. We didn’t realize how powerful this was.”

In Sandy’s system, you don’t worry about database design. He and his partners not only didn’t worry about ETL, they wondered how data analysis could not be done their way — import first, clean later. “It makes good sense if you can get away with it.”

A crucial factor that lets the tool work as it does is speed. It allows the 1010Data engine to calculate and recalculate repeatedly. The summaries that cubes harbor for anticipated queries are no longer necessary. Parallel processing with a columnar database runs fast enough. In place of ETL, he uses what he now calls “ELTAR,” for extract, load, and transform as required.

A hurdle, he says, is conventional beliefs held by his sales prospects. In one phone call recently, he explained to a prospect that ETL was unnecessary. The man replied, “That’s not credible.” In fine sales form, Sandy said, “Then you’ll be impressed when I prove it to you.” The prospect replied more firmly, “You don’t understand. That’s not credible.”

Actually, the technology’s credibility doesn’t matter much. The company, 1010Data, offers reporting and analytics on the cloud — invisible to customers except for the results. Sandy says, “We could have monkeys writing on scratchpads.” To those willing to try, he offers to prove it with the prospect’s own data.

Their technology’s speed allows them to do the work of dozens with a team of a few people, he says, and to finish large data warehouse projects in weeks that would otherwise take months or years. If multiple customers use the same data, such as stock market data, the time required is even less.

All without ETL.