Tools and those who enable their misuse

To get a data architect I know worked up, just ask him about how customers end up buying the wrong tools.

How about sales people who push federation tools on those who actually need data warehouses?

“It all sounds extremely sexy,” says my source, who works for a major business intelligence vendor and whom I can’t identify. “You have a lot of people who exaggerate their ability to combine data to provide business solutions. … They don’t prototype, they don’t profile, they don’t actually think about the problem or do testing or even send some high school data analyst out with Excel to put something together that [the customer] might want. They don’t do that.”

Many sales people tout EII because that’s what they have to sell, he says. “The EII tools give you your data, warts and all,” he says. It’ll work fine as a data warehouse substitute “if the data’s pretty clean to start with, if it has a somewhat similar structure, if you can define the data you need, if the data’s relatively common across all the sources, and if there’s not much duplication.”

Even if the salesperson has a more appropriate tool than what the customer asks for, the customer may never hear about it. “‘Fine!,'” thinks the salesperson. “‘If you want to buy a hammer, that’s fine. If you want to buy a wrench, that’s fine. It’s not like I care. It’s just sales to me.'”

Just once, says my source, he’d like to hear one of these questions: “How long does it take for a novice to become OK at this task?” Or, “How long would it take for an expert to become proficient at these two things?” Or, “If I have a failure, what is your tool’s usual process for recovery, and what gives your tool more integrity than others?”

Mark Madsen, meanwhile, has been been thinking about similar problems but from a different perspective. He’s research director at the Third Nature consultancy and a keynote speaker at this month’s TDWI conference in Las Vegas.

One source of problems he sees is vendor marketing. “It’s all about ‘our tool does this’ or ‘has these features,'” he writes in email. “A lot of people don’t think about them that way. They think about them as ‘what this tool is for.'” People end up using an ETL tool for real-time synchronization, for example, or a federation tool in place of a data warehouse.

Even product documentation can lead users down dark paths. “All those docs that say what the features are help when you know what feature you want,” he writes. “When you’re trying to accomplish a task, you’re thinking in a different way.” A common result: convoluted solutions.

“I once did something in an ETL tool,” he writes, “and the product developer said, ‘That’s not how you do that.’ They had built around an improper conception of how users apply it.”

Design schools tell you that every user has a theory of how anything works, he writes, which determines their approach to it. Wrong theories explain why people push on doors that need to be pulled, for example. He says that this insight has made him change his approach to teaching his courses or showing clients.

“I’ve realized that I need to start with the ‘what this thing is for’ and move into what you do with it, and how it works.”

Mark may go into this more in his keynote at this month’s TDWI World Conference in Las Vegas. His long-running “Clues to the Future of Business Intelligence” — perhaps the “Cats” of tech presentations — has been one of the most interesting I’ve seen in any tech industry. I expect “Stop Paving the Cowpath” to be worthwhile.


  1. Good post Ted. I haven’t been reading your blog as much as I used to now that I’m focused on data integration as a cloud-based service at Informatica. This one struck me as very relevant to my new world, however, as we’ve been primarily selling integration applications to non-technical line of business “users” and SaaS administrators. As we’ve expanded our capabilities and strengthened the ties to our core on-premise products, it’s been really interesting to witness the politics of the cloud and the impact it is having on both the traditional software buying cycle and the role of enterprise IT in the process.

    The tools are definitely changing and “those who enable their misuse” is only going to get worse unless there are clear and open lines of communication / collaboration between LOB and IT. Yes, the responsibility of focusing on “what is this for” falls on the vendors, but the buyers are also going to need to align in order to prevent the proliferation of point solutions that don’t scale with the business, more data marts, fragmented data, and all of the traditional issues you’ve written so much about over the years.

  2. Dan Murray says:

    The vast majority of entities don’t need an architecture – they need data to be turned into information quickly. If there are 7.5 million entities in the USA then I say 7.4 million can’t afford or don’t need metadata that takes more than a few days to create.

Leave a Reply

Your email address will not be published. Required fields are marked *