Make It Stick ê 54
Researchers initially predicted that massed practice in identifying paint ers’ works (that is, studying many examples of one paint er’s works before moving on to study many examples of another’s works) would best help students learn the defi ning characteristics of each artist’s style. Massed practice of each artist’s works, one artist at a time, would better enable students to match artworks to artists later, compared to interleaved exposure to the works of different artists. The idea was that interleaving would be too hard and confusing; students would never be able to sort out the relevant dimensions. The researchers were wrong. The commonalities among one paint-er’s works that the students learned through massed practice proved less useful than the differences between the works of multiple paint ers that the students learned through interleaving. Interleaving enabled better discrimination and produced better scores on a later test that required matching the works with their paint ers. The interleaving group was also better able to match paint ers’ names correctly to new examples of their work that the group had never viewed during the learning phase. Despite these results, the students who participated in these experiments persisted in preferring massed practice, convinced that it served them better. Even after they took the test and could have realized from their own per for mance that interleaving was the better strategy for learning, they clung to their belief that the concentrated viewing of paintings by one artist was better. The myths of massed practice are hard to ex-orcise, even when you’re experiencing the evidence yourself.7
The power of interleaving practice to improve discrim-inability has been reaffi rmed in studies of people learning bird classifi cation. The challenge here is more complex than it might seem. One study addressed twenty different bird families (thrashers, swallows, wrens, fi nches, and so on). Within each family, students were presented with a dozen species
Mix Up Your Practice ê 55
(brown thrasher, curve-
billed thrasher, Bendire’s thrasher,
etc.). To identify a bird’s family, you consider a wide range of traits like size, plumage, behavior, location, beak shape, iris color, and so on. A problem in bird identifi cation is that members of a family share many traits in common but not all. For instance, many but not all thrashers have a long, slightly hooked beak. There are traits that are typical of a family but none that occur in all members of that family and can serve as unique identifi ers. Because rules for classifi cation can only rely on these characteristic traits rather than on defi ning traits (ones that hold for every member), bird classifi cation is a matter of learning concepts and making judgments, not simply memorizing features. Interleaved and variable practice proved more helpful than massed practice for learning the underlying concepts that unite and differentiate the species and families.
To paraphrase a conclusion from one of these studies, recall and recognition require “factual knowledge,” considered to be a lower level of learning than “conceptual knowledge.”
Conceptual knowledge requires an understanding of the interrelationships of the basic elements within a larger structure that enable them to function together. Conceptual knowledge is required for classifi cation. Following this logic, some people argue that practicing retrieval of facts and exemplars would fall short as a strategy for comprehending general characteristics that are required for higher levels of intellectual behavior.
The bird classifi cation studies suggest the opposite: strategies of learning that help students identify and discern complex prototypes (family resemblances) can help them grasp the kinds of contextual and functional differences that go beyond the acquisition of simple forms of knowledge and reach into the higher sphere of comprehension.8
Make It Stick ê 56
Improving Complex Mastery for
Medical Students
The distinction between straightforward knowledge of facts and deeper learning that permits fl exible use of the knowledge may be a little fuzzy, but it resonates with Douglas Larsen at Washington University Medical School in St. Louis, who says that the skills required for bird classifi cation are similar to those required of a doctor diagnosing what’s wrong with a patient. “The reason variety is important is it helps us see more nuances in the things that we can compare against,” he says. “That comes up a lot in medicine, in the sense that every patient visit is a test. There are many layers of explicit and implicit memory involved in the ability to discriminate between symptoms and their interrelationships.” Implicit memory is your automatic retrieval of past experience in interpreting a new one. For example, the patient comes in and gives you a story. As you listen, you’re consciously thinking through your mental library to see what fi ts, while also unconsciously polling your past experiences to help interpret what the patient is telling you. “Then you’re left with making a judgment call,” Larsen says.9
Larsen is a pediatric neurologist seeing patients in the university clinic and hospital. He’s a busy guy: in addition to practicing medicine, he supervises the work of physicians in training, he teaches, and as time permits, he conducts research into medical education, working in collaboration with cognitive psychologists. He’s drawing on all of these roles to redesign and strengthen the school’s training curriculum in pediatric neurology.
As you’d expect, the medical school employs a wide spectrum of instructional techniques. Besides classroom lectures
Mix Up Your Practice ê 57
and labs, students practice resuscitations and other procedures on high-tech mannequins in three simulation centers the school maintains. Each “patient” is hooked up to monitors, has a heartbeat, blood pressure, pupils that dilate and constrict, and the ability to listen and speak, thanks to a controller who observes and operates the mannequin from a back room. The school also makes use of “standardized patients,” actors who follow scripts and exhibit symptoms the students are required to diagnose. The center is set up like a regular medical clinic, and students must show profi ciency in all aspects of a patient encounter, from bedside manner, physical exam skills, and remembering to ask the full spectrum of pertinent questions to arriving at a diagnosis and treatment plan.
From studies of these teaching methods, Larsen has drawn some interesting conclusions. First— and this may seem self-evident: you do better on a test to demonstrate your competency at seeing patients in a clinic if your learning experience has involved seeing patients in a clinic. Simply reading about patients is not enough. However, on written fi nal exams, medical students who have examined patients and those who have learned via written tests do equally well. The reason is that in a written test the student is being given considerable structure and being asked for specifi c information. When examining the patient, you have to come up on your own with the right mental model and the steps to follow. Having practiced these steps on patients or simulated patients improves per for mance relative to just reading about how to do it. In other words, the kind of retrieval practice that proves most effective is one that refl ects what you’ll be doing with the knowledge later. It’s not just what you know, but how you practice what you know that determines how well the learning serves you later. As the sports adage goes, “practice like you play and you will play
Make It Stick ê 58
like you practice.” This conclusion lines up with other research into learning, and with some of the more sophisticated training practices in science and industry, including the increasingly broad use of simulators— not just for jet pi lots and medical students but for cops, towboat pi lots, and people in almost any fi eld you can name that requires mastery of complex knowledge and skills and where the stakes for getting it right are high. Book learning is not enough in these cases; actual hands-on practice is needed.
Second, while it is important for a medical student to build breadth by seeing a wide variety of patients manifesting different diseases, placing too much emphasis on variety runs the risk of underemphasizing repeated retrieval practice on the basics— on the typical way the disease presents itself in most patients.
“There’s a certain set of diseases that we want you to know very well,” Larsen says. “So we’re going to have you see these standardized patients again and again, and assess your perfor mance until you really have that down and can show us, ‘I really do that well.’ It’s not either/or, variety versus repetition.
We need to make sure we’re appropriately balanced, and also recognize that we sometimes fall into the trap of familiarity.
‘I’ve already seen a bunch of patients with this problem, I don’t need to keep seeing them.’ But really, repeated retrieval practice is crucial to long- term retention, and it’s a critical aspect of training.”
A third critical aspect is practical experience. For a doctor, seeing patients provides a natural cycle of spaced retrieval practice, interleaving, and variety. “So much of medicine is based on learning by experience, which is why, after the fi rst two years, we take students out of the classroom and start putting them into clinical settings. A huge question is, what is
Mix Up Your Practice ê 59
it about learning and experience that come together? We have lots of experiences we don’t learn from. What differentiates those that teach us something?”
One form of practice that helps us learn from experience, as the neurosurgeon Mike Ebersold recounted in Chapter 2, is refl ection. Some people are more given to the act of refl ection than others, so Doug Larsen has broadened his research to study how you might structure refl ection as an integral part of the training, helping students cultivate it as a habit. He is experimenting with requiring students to write daily or weekly summaries of what they did, how it worked, and what they might do differently next time to get better results. He specu-lates that daily refl ection, as a form of spaced retrieval practice, is probably just as critical in the real- world application of medicine as quizzing and testing are in building competencies in medical school.
What about the classroom lecture, or the typical in- service training conference that’s compressed over a couple of days?
Larsen fi gures his school’s interns spend 10 percent of their time sitting in conferences listening to lectures. It may be a talk on metabolic diseases, on different infectious diseases, or on different drugs. The speaker puts the PowerPoint slideshow up and starts going through it. Usually there’s lunch, and the docs eat, listen, and leave.
“In my mind, considering how much forgetting occurs, it’s very discouraging that we’re putting so many resources into an activity that, the way it is currently done, learning research tells us is so in effec tive. Medical students and residents go to these conferences and they have no repeated exposure what-soever to it. It’s just a matter of happenstance whether they end up fi nally seeing a patient in the future whose problem relates back to the conference topic. Otherwise, they don’t study the
Make It Stick ê 60
material, they are certainly not tested on the material, they just listen then they walk out.”
At a minimum, Larsen would like to see something done to interrupt the forgetting: give a quiz at the end of a conference and follow it with spaced retrieval practice. “Make quizzing a standard part of the culture and the curriculum. You just know every week you’re going to get in your email your ten questions that you need to work through.”
He asks, “How are we designing education and training systems that prevent or at least intervene in the amount of forgetting that goes on, and making sure they’re systematic throughout the school in support of what we’re trying to accomplish? As it stands now, medical resident programs are simply dictating: you have to have the curriculum, you have to have the conferences, and it ends there. They present these big conferences, they have all the faculty come through and give their talks. And in the end, what we actually accomplish is really kind of minimal.”10
These Principles Are Broadly Applicable College football might seem an incongruous place to look for a learning model, but a conversation with Coach Vince Dooley about the University of Georgia’s practice regime provides an intriguing case.
Dooley is authoritative on the subject. As head coach of Bulldogs football from 1964– 1988, he piled up an astonishing 201 wins with only 77 losses and 10 tied games, winning six conference titles and a national championship. He went on to serve as the university’s athletic director, where he built one of the most impressive athletics programs in the country.
We asked Coach Dooley how players go about mastering all the complexities of the game. His theories of coaching and
Mix Up Your Practice ê 61
