It’s not just the dullards who fall victim. We all do, to varying degrees. Pi lots, for example, are susceptible to a host of perceptual illusions. They are trained to beware of them and to use their instruments to know that they’re getting things right.
A frightening example with a happy ending is China Airlines Flight 006 on a winter day in 1985. The Boeing 747 was 41,000 feet above the Pacifi c, almost ten hours into its eleven-hour fl ight from Taipei to LA, when engine number 4 lost power. The plane began to lose airspeed. Rather than taking manual control and descending below 30,000 feet to restart the engine, as prescribed in the fl ight book, the crew held at 41,000 with the autopi lot engaged and attempted a restart.
Meanwhile, loss of the outboard engine gave the plane asymmetrical thrust. The autopi lot tried to correct for this and keep the plane level, but as the plane continued to slow it also began to roll to the right. The captain was aware of the
Avoid Illusions of Knowing ê 107
deceleration, but not the extent to which the plane had entered a right bank; his System 1 clue would have been his vestibular refl ex— how the inner ear senses balance and spatial orientation—but because of the plane’s trajectory, he had the sensation of fl ying level. His System 2 clues would have been a glimpse at the horizon and his instruments. Correct procedure called for applying left rudder to help raise the right wing, but his System 2 focus was on the airspeed indicator and on the efforts of the fi rst offi cer and engineer to restart the engine.
As its bank increased, the plane descended through 37,000
feet into high clouds, which obscured the horizon. The captain switched off the autopi lot and pushed the nose down to get more speed, but the plane had already rolled beyond 45
degrees and now turned upside down and fell into an uncontrolled descent. The crew were confused by the situation. They understood the plane was behaving erratically but were unaware they had overturned and were in a dive. They could no longer discern thrust from engines 1– 3 and concluded those engines had quit as well. The plane’s dive was evident from their fl ight gauges, but the angle was so unlikely the crew decided the gauges had failed. At 11,000 feet they broke through the clouds, astonished to see that they were roaring toward earth. The captain and fi rst offi cer both pulled back hard on the stick, exerting enormous forces on the plane but managing to level off. Landing gear hung from the plane’s belly, and they’d lost one of their hydraulic systems, but all four engines came to life, and the captain was able to fl y on, diverting successfully to San Francisco. An inspection revealed just how severe their maneuver had been. Strains fi ve times the force of gravity had bent the plane’s wings permanently upward, broken two landing gear struts, and torn away two landing gear doors and large parts of the rear horizontal stabilizers.
Make It Stick ê 108
“Spatial disorientation” is the aeronautical term for a deadly combination of two elements: losing sight of the horizon and relying on human sensory perception that doesn’t jibe with reality but is so convincing that pi lots conclude their cockpit instruments have failed. As Kahneman says, System 1, the instinctual, refl exive system that detects danger and keeps us safe, can be very hard to overrule. Flight 006’s initial incident, the loss of an engine cruising at altitude, is not considered an emergency, but it quickly became one as a result of the captain’s actions. Rather than following prescribed procedure, and rather than fully engaging his System 2 analytic resources by monitoring all his instruments, he let himself become preoccupied with the engine restart and with a single fl ight indicator, airspeed. Then, when things spiraled out of control, he trusted his senses over his gauges, in effect trying to construct his own narrative of what was happening to the plane.
There’s a long list of illusions to which pi lots can fall prey (some with mordant names like “the leans,” “graveyard spin,”
and “the black hole approach”) and sites on the Internet where you can listen to the chilling last words of pi lots struggling and failing to understand and correct what’s gone wrong in the sky. Spatial disorientation was deemed the probable cause of the crash that killed Mel Carnahan, the governor of Missouri, while being fl own through a thunderstorm one night in October 2000, and the probable cause of the crash that killed John F. Kennedy Jr. and his wife and her sister off the shore of Martha’s Vineyard on a hazy night in July 1999. Fortunately, the China Airlines incident came to a good end, but the National Transportation Safety Board report of that incident reveals just how quickly training and professionalism can be hijacked by System 1 illusion, and therefore why we need to
Avoid Illusions of Knowing ê 109
cultivate a disciplined System 2, conscious analysis and reasoning, that always keeps one eye on the fl ight instruments.3
Illusions and Memory Distortions
The fi lmmaker Errol Morris, in a series of articles on illusion in the New York Times, quotes the social psychologist David Dunning on humans’ penchant for “motivated reasoning,” or, as Dunning put it, the “sheer genius people have at convincing themselves of congenial conclusions while denying the truth of incon ve nient ones.”4 (The British prime minister Benjamin Disraeli once said of a po liti cal opponent that his conscience was not his guide but his accomplice.) There are many ways that our System 1 and System 2 judgments can be led astray: perceptual illusions like those experienced by pi lots, faulty narrative, distortions of memory, failure to recognize when a new kind of problem requires a new kind of solution, and a variety of cognitive biases to which we’re prone. We describe a number of these hazards here, and then we offer mea sures you can take, akin to scanning the cockpit instruments, to help keep your thinking aligned with reality.
Our understanding of the world is shaped by a hunger for narrative that rises out of our discomfort with ambiguity and arbitrary events. When surprising things happen, we search for an explanation. The urge to resolve ambiguity can be surprisingly potent, even when the subject is inconsequential. In a study where participants thought they were being mea sured for reading comprehension and their ability to solve anagrams, they were exposed to the distraction of a background phone conversation. Some heard only one side of a conversation,
Make It Stick ê 110
and others heard both sides. The participants, not knowing that the distraction itself was the subject of the study, tried to ignore what they were hearing so as to stay focused on the reading and anagram solutions. The results showed that overhearing one side of a conversation proved more distracting than overhearing both sides, and the content of those partial conversations was better recalled later by the unintentional eavesdroppers. Why was this? Presumably, those overhearing half a conversation were strongly compelled to try to infer the missing half in a way that made for a complete narrative. As the authors point out, the study may help explain why we fi nd one- sided cell phone conversations in public spaces so intrusive, but it also reveals the ineluctable way we are drawn to imbue the events around us with rational explanations.
The discomfort with ambiguity and arbitrariness is equally powerful, or more so, in our need for a rational understanding of our own lives. We strive to fi t the events of our lives into a cohesive story that accounts for our circumstances, the things that befall us, and the choices we make. Each of us has a different narrative that has many threads woven into it from our shared culture and experience of being human, as well as many distinct threads that explain the singular events of one’s personal past. All these experiences infl uence what comes to mind in a current situation and the narrative through which you make sense of it: Why nobody in my family attended college until me. Why my father never made a fortune in business. Why I’d never want to work in a corporation, or, maybe, Why I would never want to work for myself. We gravitate to the narratives that best explain our emotions. In this way, narrative and memory become one. The memories we or ga nize meaningfully become those that are better remembered. Narrative provides not only meaning but also a
Avoid Illusions of Knowing ê 111
mental framework for imbuing future experiences and information with meaning, in effect shaping new memories to fi t our established constructs of the world and ourselves. No reader, when asked to account for the choices made under pressure by a novel’s protagonist, can keep her own life experience from shading her explanation of what must have been going on in the character’s interior world. The success of a magician or politician, like that of a novelist, relies on the seductive powers of narrative and on the audience’s willing suspension of disbelief. Nowhere is this more evident than in the national po liti cal debate, where like- minded people gather online, at community meetings, and in the media to fi nd common purpose and expand the story they feel best explains their sense of how the world works and how humans and politicians should behave.
You can see how quickly personal narrative is invoked to explain emotions when you read an article online whose author has argued a position on almost any subject— for example, an op- ed piece supporting the use of testing as a powerful tool for learning. Scan the comments posted by readers: some sing hallelujah while others can scarcely contain their umbrage, each invoking a personal story that supports or refutes the column’s main argument. The psychologists Larry Jacoby, Bob Bjork, and Colleen Kelley, summing up studies on illusions of comprehension, competence, and remembering, write that it is nearly impossible to avoid basing one’s judgments on subjective experience. Humans do not give greater credence to an objective record of a past event than to their subjective remembering of it, and we are surprisingly insensitive to the ways our par tic u lar construals of a situation are unique to ourselves. Thus the narrative of memory becomes central to our intuitions regarding the judgments we make and the actions we take.5
Make It Stick ê 112
It is a confounding paradox, then, that the changeable nature of our memory not only can skew our perceptions but also is essential to our ability to learn. As will be familiar to you by now, every time we call up a memory, we make the mind’s routes to that memory stronger, and this capacity to strengthen, expand, and modify memory is central to how we deepen our learning and broaden the connections to what we know and what we can do. Memory has some similarities to a Google search algorithm, in the sense that the more you connect what you learn to what you already know, and the more associations you make to a memory (for example, linking it with a visual image, a place, or a larger story), then the more mental cues you have through which to fi nd and retrieve the memory again later. This capacity expands our agency: our ability to take action and be effective in the world. At the same time, because memory is a shape- shifter, reconciling the competing demands of emotion, suggestions, and narrative, it serves you well to stay open to the fallibility of your certain-ties: even your most cherished memories may not represent events in the exact way they occurred.
Memory can be distorted in many ways. People interpret a story in light of their world knowledge, imposing order where none had been present so as to make a more logical story.
Memory is a reconstruction. We cannot remember every aspect of an event, so we remember those elements that have greatest emotional signifi cance for us, and we fi ll in the gaps with details of our own that are consistent with our narrative but may be wrong.
People remember things that were implied but not specifi -
cally stated. The literature is full of examples. In one, many people who read a paragraph about a troubled girl named Helen Keller later mistakenly recalled the phrase “deaf, dumb, and blind” as being in the text. This mistake was rarely made
Avoid Illusions of Knowing ê 113
by another group who read the same paragraph about a girl named Carol Harris.6
Imagination infl ation refers to the tendency of people who, when asked to imagine an event vividly, will sometimes begin to believe, when asked about it later, that the event actually occurred. Adults who were asked “Did you ever break a window with your hand?” were more likely on a later life inventory to report that they believed this event occurred during their lifetimes. It seems that asking the question led them to imagine the event, and the act of having imagined it had the effect, later, of making them more likely to think it had occurred (relative to another group who answered the question without having previously imagined it occurring).
Hypothetical events that are imagined vividly can seat themselves in the mind as fi rmly as memories of actual events. For instance, when it is suspected that a child is being sexually abused and he is interviewed and questioned about it, he may imagine experiences that the interviewer describes and then later come to “remember” them as having occurred.7 (Sadly, of course, many memories of childhood sexual abuse are absolutely true, usually ones reported soon after the occurrence.) Another type of memory illusion is one caused by suggestion, which may arise simply in the way a question is asked. In one example, people watched a video of a car running a stop sign at an intersection and colliding with another car passing through. Those who were later asked to judge the speed of the vehicles when they “contacted” each other gave an average estimate of thirty- two miles per hour. Those who were asked to judge the speed when the two vehicles “smashed” into each
Make It Stick ê 114
other estimated on average forty- one miles per hour. If the speed limit was thirty miles per hour, asking the question the second way rather than the fi rst could lead to the driver’s being charged with speeding. Of course, the legal system knows the danger of witnesses being asked “leading questions” (ones that encourage a par tic u lar answer), but such questions are diffi cult to avoid completely, because suggestibility can be very subtle. After all, in the case just discussed, the two cars did “smash together.”8
Some witnesses to crimes who are struggling to recall them are instructed to let their minds roam freely, to generate what-ever comes to mind, even if it is a guess. However, the act of guessing about possible events causes people to provide their own misinformation, which, if left uncorrected, they may later come to retrieve as memories. That is one reason why people who have been interviewed after being hypnotized are barred from testifying in court in almost all states and Canadian provinces. The hypnotic interview typically encourages people to let their thoughts roam freely and produce everything that comes to mind, in hopes that they will retrieve information that would not otherwise be produced. However, this pro cess causes them to produce much erroneous information, and studies have shown that when they are tested later, under instructions only to tell exactly what they remember of the actual events, their guesses made while under hypnosis cloud their memories about what truly happened. In par tic ular, they remember events they produced under hypnosis as actual experiences, even under conditions (in the laboratory) when it is known that the events in question did not occur.9
Interference from other events can distort memory. Suppose the police interview a witness shortly after a crime, showing
Avoid Illusions of Knowing ê 115
pictures of possible suspects. Time passes, but eventually the police nab a suspect, one whose picture had been viewed by the witness. If the witness is now asked to view a lineup, he may mistakenly remember one of the suspects whose photo he saw as having been present at the crime. A particularly vivid example of a related pro cess happened to the Australian psychologist Donald M. Thomson. A woman in Sydney was watching tele vi sion in midday when she heard a knock at the door. When she answered it, she was attacked, raped, and left unconscious. When she awoke and dialed the police, they came to her aid, got a description of her assailant, and launched a search. They spotted Donald Thomson walking down a Sydney street, and he matched the description. They arrested him on the spot. It turns out that Thomson had an airtight alibi— at the exact time of the rape, he was being interviewed on a live tele vi sion show. The police did not believe him and sneered when he was being interrogated. However, the story was true. The woman had been watching the show when she heard the knock on the door. The description she gave the police was apparently of the man she saw on tele vi sion, Donald Thomson, rather than the rapist. Her System 1 reaction—
quick but sometimes mistaken— provided the wrong description, probably due to her extreme emotional state.10
