CIA’s insights on the psychology of analysis

Imagine someone writing a book about data analysis without even mentioning software.

“To penetrate the heart and soul of the problem of improving analysis,” writes Richard J. Heuer Jr. in Psychology of Intelligence Analysis, “it is necessary to better understand, influence, and guide the mental processes of analysts themselves.” It’s the mind that does the heavy lifting.

Though Heuer writes for CIA-type analysts, similarities to business are clear. The biggest one is that it’s always the mind that does the heavy lifting.

The two kinds of data analysts share many other aspects of their work, such as their unconscious biases. For example, they all tend to give undue importance to vivid evidence over abstract evidence.

Both also endure the after-the-fact know-it-all, the type that announces, “That analysis told me nothing new,” after confirming evidence arises. Heuer describes an experiment that suggests such beliefs are usually delusional.

Analysts’ memories, too, undermine their work, such as when asked to remember what led them to their conclusions. Also, overseers tend to regard events as more foreseeable than they actually were.

Missing data, another problem, is “out of sight, out of mind.” Two groups of experienced auto mechanics were asked for a diagnosis of a car that wouldn’t start. They were offered a fault tree with several major branches — weak battery, starter-system defects, etc. — with several themes left out and summarized as “other causes.” One group’s fault tree included all major branches. But the second group’s fault tree had several branches missing — and selected the summarized “other causes” far less often than would be expected if the summarized reasons had been listed along with all the others.

Heuer offers a few solutions. The main one seems to be his “Analysis of Competing Hypotheses.” He outlines it in eight steps.

Heuer writes the way I’d expect a CIA veteran to write: formally. Even where he has a colorful tidbit — such as any of the load of stories mentioned in the footnotes — he opts for gray.

Such drab phrasing ignores the principle he gives in Chapter 10 under “the vividness criterion.” More vividness would have made a more readable book.

The impact of information on the human mind is only imperfectly related to its true value as evidence. Specifically, information that is vivid, concrete, and personal has a greater impact on our thinking than pallid, abstract information that may actually have substantially greater value as evidence.

If there’s one thing the book reminds the reader of over and over, it’s that analysts are only human.

On tools, Heuer lists only what he considers the basics: decomposition and externalization. Vendors will have to do what they can with those clues. Unlike so much software, though, Psychology of Intelligence Analysis is free.

2 comments

  1. Dan Murray says:

    Thought provoking post Ted. A real creative “Whack.”

    It’s not unusual for my ramblings in visual analysis to lead me down different paths. Keeping an open mind about the possible causes is what leads me from “question to insight.”

    Mr Heurer looks at the world through a different lens vs the typical business analyst. Some of his insights may prove valuable.

    I’m adding his book to me reading list. My aching Amazon account thanks you for the free link.

  2. “(…) information that is vivid, concrete, and personal has a greater impact on our thinking than pallid, abstract information that may actually have substantially greater value as evidence.”

    This also explains why screaming headlines get more attention than nuanced ones and why it is sometimes difficult to gain attention when real insights are being shared.

    Your article reminds us that being an analyst also requires a splash of art: the art of shouting “Fire!” without actually causing panic. In his 14 Chapter e-book, Heuer provides plenty of ideas about how to hone this art – including ways to help us question our own assumptions and conclusions.

    I particularly like his discussion about how information is assimilated and how we have to first evaluate what is known and assumed before we can open the door to new insight. Discerning knowledge from assumption is often more difficult than it sounds.

    Thank you for sharing!

Leave a Reply

Your email address will not be published. Required fields are marked *