Here is an excerpt from the transcript of a conversation during which Nobel laureate Daniel Kahneman and psychologist Gary Klein debate the power and perils of intuition for senior executives. It appeared in the McKinsey Quarterly, published by McKinsey & Company. To read the complete article, check out other resources, learn more about the firm, obtain subscription information, and register to receive email alerts, please click here.

To learn more about the McKinsey Quarterly, please click here.

* * *

For two scholars representing opposing schools of thought, Daniel Kahneman and Gary Klein find a surprising amount of common ground. Kahneman, a psychologist, won the Nobel Prize in economics in 2002 for prospect theory, which helps explain the sometimes counterintuitive choices people make under uncertainty. Klein, a senior scientist at MacroCognition, has focused on the power of intuition to support good decision making in high-pressure environments, such as firefighting and intensive-care units.

In a September 2009 American Psychology article titled “Conditions for intuitive expertise: A failure to disagree,” Kahneman and Klein debated the circumstances in which intuition would yield good decision making. In this interview with Olivier Sibony, a director in McKinsey’s Brussels office, and Dan Lovallo, a professor at the University of Sydney and an adviser to McKinsey, Kahneman and Klein explore the power and perils of intuition for senior executives.

In your recent American Psychology article, you asked a question that should be interesting to just about all executives: “Under what conditions are the intuitions of professionals worthy of trust?” What’s your answer? When can executives trust their guts?

Gary Klein: It depends on what you mean by “trust.” If you mean, “My gut feeling is telling me this; therefore I can act on it and I don’t have to worry,” we say you should never trust your gut. You need to take your gut feeling as an important data point, but then you have to consciously and deliberately evaluate it, to see if it makes sense in this context. You need strategies that help rule things out. That’s the opposite of saying, “This is what my gut is telling me; let me gather information to confirm it.”

Daniel Kahneman: There are some conditions where you have to trust your intuition. When you are under time pressure for a decision, you need to follow intuition. My general view, though, would be that you should not take your intuitions at face value. Overconfidence is a powerful source of illusions, primarily determined by the quality and coherence of the story that you can construct, not by its validity. If people can construct a simple and coherent story, they will feel confident regardless of how well grounded it is in reality.

Is intuition more reliable under certain conditions?

Klein: We identified two. First, there needs to be a certain structure to a situation, a certain predictability that allows you to have a basis for the intuition. If a situation is very, very turbulent, we say it has low validity, and there’s no basis for intuition. For example, you shouldn’t trust the judgments of stock brokers picking individual stocks. The second factor is whether decision makers have a chance to get feedback on their judgments, so that they can strengthen them and gain expertise. If those criteria aren’t met, then intuitions aren’t going to be trustworthy.

Most corporate decisions aren’t going to meet the test of high validity. But they’re going to be way above the low-validity situations that we worry about. Many business intuitions and expertise are going to be valuable; they are telling you something useful, and you want to take advantage of them.

Kahneman: This is an area of difference between Gary and me. I would be wary of experts’ intuition, except when they deal with something that they have dealt with a lot in the past. Surgeons, for example, do many operations of a given kind, and they learn what problems they’re going to encounter. But when problems are unique, or fairly unique, then I would be less trusting of intuition than Gary is. One of the problems with expertise is that people have it in some domains and not in others. So experts don’t know exactly where the boundaries of their expertise are.

Many executives would argue that major strategic decisions, such as market entry, M&A, or R&D investments, take place in environments where their experience counts—what you might call high-validity environments. Are they right?

Klein: None of those really involve high-validity environments, but there’s enough structure for executives to listen to their intuitions. I’d like to see a mental simulation that involves looking at ways each of the options could play out or imagining ways that they could go sour, as well as discovering why people are excited about them.

Kahneman: In strategic decisions, I’d be really concerned about overconfidence. There are often entire aspects of the problem that you can’t see—for example, am I ignoring what competitors might do? An executive might have a very strong intuition that a given product has promise, without considering the probability that a rival is already ahead in developing the same product. I’d add that the amount of success it takes for leaders to become overconfident isn’t terribly large. Some achieve a reputation for great successes when in fact all they have done is take chances that reasonable people wouldn’t take.

Klein: Danny and I are in agreement that by the time executives get to high levels, they are good at making others feel confident in their judgment, even if there’s no strong basis for the judgment (see sidebar, “Overconfidence in action?”).

So you would argue that selection processes for leaders tend to favor lucky risk takers rather than the wise?

Kahneman: No question — if there’s a bias, it’s in that direction. Beyond that, lucky risk takers use hindsight to reinforce their feeling that their gut is very wise. Hindsight also reinforces others’ trust in that individual’s gut. That’s one of the real dangers of leader selection in many organizations: leaders are selected for overconfidence. We associate leadership with decisiveness. That perception of leadership pushes people to make decisions fairly quickly, lest they be seen as dithering and indecisive.

Klein: I agree. Society’s epitome of credibility is John Wayne, who sizes up a situation and says, “Here’s what I’m going to do”—and you follow him. We both worry about leaders in complex situations who don’t have enough experience, who are just going with their intuition and not monitoring it, not thinking about it.

Kahneman: There’s a cost to not being John Wayne, since there really is a strong expectation that leaders will be decisive and act quickly. We deeply want to be led by people who know what they’re doing and who don’t have to think about it too much.

* * *

Here is a direct link to the complete article.

Daniel Kahneman is a Nobel laureate and a professor emeritus of psychology and public affairs at Princeton University’s Woodrow Wilson School. He is also a fellow at the Hebrew University of Jerusalem and a Gallup senior scientist. Gary Klein is a cognitive psychologist and senior scientist at MacroCognition. He is the author of Sources of Power: How People Make Decisions, The Power of Intuition, and Streetlights and Shadows: Searching for the Keys to Adaptive Decision Making.