Home » BI industry » trends

Category: trends

A horse race in town: Chief Data Officer vs Chief Reliance Officer

Smart City, the movement, has a horse race underway. At stake: Who will sit closer to the mayor or city manager?

Chief Data Officer, the darling of the data crowd, was out of the gate first. Four lengths behind and gaining, though, is a dark horse. It’s Chief Resilience Officer, favorite of the humanists. He’s breathing hard and coming up fast. Whoever wins will subsume the other.

If Chief Data Officer wins, the city’s chief executive will feel the sway of CDO’s data-driven whispers. All other things being equal, decisions will rely on data analysis.

But if Chief Resilience Officer pulls off a surprise win, the chief executive will hear her slightly more humanist whispers. Data will be a factor, but so will empathy. Here’s how the group 100 Resilient Cities begins to explain resilience:

[Resilience is] the capacity of individuals, communities, institutions, businesses, and systems within a city to survive, adapt, and grow no matter what kinds of chronic stresses and acute shocks they experience.

I don’t see how that’s done without a big dose of data analysis. What modern process goes without it? The difference is scope: tending toward cold and narrow or toward broad and warm?

Each horse has a cheering section in the stands. Naturally, the data people — data analysts, data stewards, “data scientists,” data people of all kinds — root for the chief data officer. “Run, CDO, run!” To many of them, data is not merely a reflection of the world, it is the world itself.

Nearby them is a small crowd cheering for the chief resilience officer. “Run, CRO, run!” Data’s good, they agree, but there’s more to running a city or living in it.

Within business, the CDO, the data guy, would be the obvious winner. Data is tangible. It’s new and shiny, and it’s got that science-and-research allure of cold certainty.

But this is a city. Warmth and comfort matters, not just “truth” and cold facts. People — the voting populace — have lives to live there in that city.

Which crowd do I stand with? I’m always for the dark horse.

Free-the-data movement meets privacy

Back when data was little and simple, self-service analysis advocates started the chant, “Free the data!” IT stood in the way, they said. Fast forward to 2016: “democratized data” has become common, but so has public concern over privacy.

That nettlesome struggle drove a discussion that now stands as the data industry’s’ most important discussion of 2016. Around the conference table at last summer’s Pacific Northwest BI Summit — the annual, invitation-only confab held in Grants Pass, Oregon — two dozen data leaders pondered the issue for almost two hours. They concluded with an idea that broke open industry assumptions.

UK-based consultant Mike Ferguson told of a meeting held on continental Europe. He expected the usual usual stakeholders, such as from marketing and HR. But alongside them were two staff from the corporate counsel’s office. The lawyers made it clear that anything decided there would need their approval.

Their worry? Compliance with the European Union’s impending data privacy law, the General Data Protection Regulation. When it takes effect in 2018, privacy violations — including failure to erase individuals’ online presence on request — could amount to 4 percent of global revenue. Other regions will likely comply, eager to ensure continuing access to the EU market and to ensure access to EU data. Across the globe, the old democratization-versus-privacy is just about to grow some big, sharp teeth.

It poses a dilemma. Everyday business now requires ready access to data. Even compliance with new privacy regulations requires access even as the regulations seek to limit it.

At the problem’s root, says Ferguson, is data integration. Multiple platforms and tools have evolved to serve big data’s proliferating, specific workloads. Streaming data, Hadoop, the enterprise data warehouse, NoSQL and others chug away, each one possibly processing another platform’s data. And all that data keeps coming in faster and faster.

Data integration’s too expensive

“What I hear from clients,” said Ferguson who is managing director of Manchester-based Intelligent Business Strategies, “is that the cost of data integration is way too high.” Skills are spread across lots of tools, everything gets re-invented continually, metadata is fractured or lost entirely as it runs through multiple tools, and there’s just too much repetition all around. Data integration among platforms seems to become more complex all the time.

Self-service data integration is cheap. Many in IT like it. But, says Ferguson, it quickly results in “a kind of Wild West.” Data moves uncontrollably, with no one guarding the sources. Users apply countless tools for data prep, ETL, data integration, and other functions, and silos proliferate.

“There’s got to be better way,” said Ferguson. He suggests supplanting the “data lake” with what he calls a “data reservoir”: a governed[ replaced “organized”; obviously it’s organized. “Governed” comes from the following slide.] collection of raw, in-progress and trusted data that incorporates multiple stores and streams. The “reservoir” would define data once to run anywhere and supply info fast.

“The smart thing is to offer virtual views, Amazon-like,” said Ferguson. Instead of copying the data, it would be offered in virtualized form, ready to use but not copied. Data’s “Wild West” would be tamed with riding stables: Ride a trusted horse on a known trail.

Local policies could be applied as the data’s dispensed. Users with proper rights would see the data. But not those without rights would be told, “Sorry, Dude, you can’t see it. Wrong jurisdiction!” 

Urgency

Underscoring the urgency of controlling data, vice president of marketing at IBM Harriet Fryman told about a crashed drone on her roof and an unsettling tweet. The tweet read, “I think my drone is on your roof. Can I have it back?” Fryman went to her roof and, sure enough, there was a crashed drone. As the owner explained later, his drone was equipped to send one last photo home before it crashed. From that image, he matched the visible roofline with a Google Maps satellite view, and from there he followed a circuitous path to Fryman’s Twitter account.

Meanwhile, explained SAS vice president of best practices Jill Dyché, executives are fed up waiting for a solution to the problem Ferguson described. Dyché has observed “an utter lack of confidence” among executives in the ability of organizations to govern data.

Donald Farmer, principal at TreeHive Strategy, raised another problem. “It’s incredibly difficult to prove something’s been deleted,” especially when the data’s already been propagated. “How do you track it back?”

Solutions

The typically voluble group went quiet for a moment, attesting to the challenge.

A surprising suggestion came from Donald Farmer, principal at TreeHive Strategy: The solution may be organizational, he said, not technological. The risk of violating privacy laws could be minimized if companies isolated risk with spinoffs. The mother company would grow as far as it could with the current technology, governance, and practices. Then it would spin off a subsidiary that would own the risky data along with the liability. Eventual innovations would transfer homeward, abandoning the risk with the spinoff’s shell.

Merv Adrian, however, disagreed. “I don’t believe that for a minute,” he said. “They’ll find a way around it,” he said. Later, he wrote to me in email, “Companies don’t do spinouts lightly. It’s disruptive, complex and costly.” The incentive would have to be strong.

Farmer had a second, even more intriguing idea: “One of the myths is that we need more information,” said Farmer. If we think again about the data we use and why we use it, he explained, we might find just about the same value with bayesian noise added. Data can be slightly wrong, with enough noise inserted to prevent hacks, and still have equal benefit to business users.

That is, the data doesn’t have to be right, just slightly wrong — at first glance an outlandish idea. It invited quips, perhaps a natural response to an implicit admission that technology may not be the answer. But who can even hope that until-now unknown difficulties, founded on a new world of unheard of complexity and an aroused public, could be solved with technology alone? Farmer’s idea, or something like it, may prove itself yet.

These are hard problems,” observed Robert Eve, director of product management, data and analytics software at Cisco Systems, with one last quip before lunch. With a colloquialism denoting the need for deep thought and newfound finesse, he added, “At run time, you have to understand the kung fu.”

Data guided, not data driven

I like data same as the next guy. But I don’t like pronouncements like the one I heard at an industry event last year: “If it isn’t data, it doesn’t exist.”

Let’s get ahold of ourselves. Sure, data gives grounding. It’s a starting point, and it’s even a GPS. But there’s much more to any decision.

It used to be taboo to say that around the business intelligence industry. Almost 10 years ago, I repeated to a data warehouse expert what I had heard from a renegade inventor, that business users knew their own data. “No!” the expert protested. But things have changed. (See “BI consolidation first hand,” 2007.)

Just the other day, I was encouraged by the response from Qlik business analytics strategist James Richardson responded to my mention of “…it doesn’t exist”:

Read more

Qlik asks what a difference a device makes

Donald Farmer, Qlik Inc.
Donald Farmer, Qlik

When I first heard of Qlik’s research into use of mobile devices, I thought so what? It’s an engineering problem, I said. Just figure out how to make charts work on a laptop, a Mac, an smartphone, and a tablet and be done with it.

That was months ago, when Qlik released its study of mobility use. Then I started watching my own use of these things and finally decided Qlik may be onto something. There may be more to this report than meets the eye.

Read more

Tableau’s storytelling, conversation, and journalism

I imagine the Tableau marketers sitting down over the coming year’s menu of trends. “What, storytelling again?” one says as if dreading the taste of dim sum for the hundredth time.

Storytelling was a staple there at the trendy headquarters. The research department had not too long ago lured Robert Kosara away from an academic career. First thing, he and Jock Mackinlay published a whitepaper on the future of storytelling.

But no marketing stays fresh forever. Even before business people finish chewing on the last trend, new ones come along with the alacrity of T-shirt slogans. Fortunately, there were choices on the menu that would give new zing to “storytelling” — not with one new trend but with two.

“Ooh,” says another marketing exec pointing to the menu, “this looks good.”

Conversation is to trendy BI marketers what crickets and plant waters are to foodies. It’s also one of the tastiest components of storytelling. Throw a chart on the screen, and you want people to say something, ask you something, object, stand up and threaten to walk out if you don’t explain your data selection. This makes all the difference in meaning and effect.

Then comes the second trend. It’s salty, unpredictable, and challenging. It’s an old recipe made new: It’s journalism. Forget storytelling and its funny aftertaste. Journalism is the word for what data has always needed to reach the masses — yes, they exist — who in their hearts don’t give a god damn about data. All they want to know is what it means.

Of course, when Tableau says journalism they mean work by those who call themselves journalists. What business needs, though, are people who practice journalism no matter what they call it. I wish more data analysts would just take off the data hat and put on the journalist’s hat. Give the story — in journalism, every piece is called a “story,” with or without classic story structure — and present data only as they would other facts.

“Mmm,” that’s tasty,” the talented data analysts will say. “Let’s try that.” And the whole business world will thank them.