Big data comes of age

It’s too easy these days to be tired of big data, with all the defining and redefining, marketing, and Hadooping. I can’t help but think to myself, “Just shut up and do it!” Of course, some organizations have gone and done it. Now a new report from TDWI Research describes the common stages they go through on the way to making big data a permanent part of their toolset.

Research director Fern Halper and Sixth Sense Advisors CEO Krish Krishnan also offer a tool anyone can use to get a quick rating of any organization’s maturity in big data analytics. (See below.)

Halper and Krishnan’s maturity model can help answer an organization’s particular unanswerables: Do we have the right stuff? What problems should we prepare for? How do we evangelize the program? It can help guide structure and processes, define goals, shape vision, reassure, or warn. They also shoot down some false assumptions one might form from all the industry’s hoopla.

… Big data maturity is not simply about having some technology in place to deal with high volumes of data. Nor is it simply about using social media to analyze buzz about your brand. It is a journey that involves building an ecosystem that includes technologies, data management, analytics, governance, and organizational components.

Their five levels of maturity — six if you count the “Chasm” — start with the “nascent,” when you’re still sniffing out the possibilities. Then comes pre-adoption, early adoption, The Chasm, corporate adoption, and mature/visionary. Organizations being fundamentally social, progress can be uneven and confusing. Perhaps the infrastructure is in place but proficiency in data analysis lags.

Descriptions of each phase cover the whole field, from assembling the right stuff to helping big data find a warm spot the organization’s heart.

“The Chasm” sounds like the tough one. One might imagine the 1944 preparations for the Allied invasion of France. Funding, data “ownership,” skills, and governance have to be ready for full corporate adoption. But, wouldn’t you know it, it’s culture and politics that often stem big data.

The Chasm’s complexity shows in this section’s word count: more than 1000, which is several hundred words longer than the next longest sections, “early adoption” and “corporate adoption.” It’s also more than twice the length of the first and last sections, “nascent” and “mature/visionary.”

By the fourth stage and fifth stages, “corporate adoption” and “mature/visionary,” big data analytics has proven itself. In the “corporate adoption,” big data supports the organization. In “mature/visionary,” the organization finally supports big data. Big data finally gets tenure.

One more thing: The “TDWI Big Data Maturity Model Guide: Interpreting Your Assessment Score” is one of the industry’s better written pieces. Halper and Krishnan have managed to write clear and interesting explanations and to avoid the gray, fatigued phrases that substitute for actual thought in so many other pieces. In their 8500 words, they actually make the path to mature big data analytics bright and clear.

Assess your progress here.

4 Responses to Big data comes of age

The data industry thrives on conversation. Please submit a comment.

Other recent posts

Five Tips for Better Data Stories

Originally published on September 22, 2015 in BI This Week, a TDWI publication. A “data story” sounds like such a great idea. You just mix data with storytelling and you’re done — except that most data storytellers get one thing wrong: they drown out the story with data. Such storytellers, I believe, assume that audiences… Continue Reading

Bohemian Grove a la BI

The Bohemian Grove of the BI industry convenes for the fifteenth time in just three weeks. Naturally, you ask the obvious question: Are you serious? The Grove? A summit? The answer begins with a fond recollection of the Grove. If you’ve never attended the Bohemian Grove yourself — I haven’t, though I live in the… Continue Reading

Favorite Star Trek, a data story

This story shows how elemental data stories really are. Humans come ready to tell and hear them, requiring no plug-ins at all. This young person can do a good job of it. There was a question, followed by data, then questions and answers, and and finally a conclusion. It’s all there. It’s elementary. Sure, this… Continue Reading

Bad stories stop good data at the water cooler

We agree by now that data’s a good compass. One neglected question is tougher: Which map? Everyone’s known the kind of “grouchy guy” TDWI instructor Kellee M. Franklin, Ph.D tells about. This guy knew better than most of his co-workers about how their Washington, D.C. defense agency worked. And he was frustrated. Over the years,… Continue Reading