Make It Stick ê 54

Researchers initially predicted that massed practice in identifying paint ers’ works (that is, studying many examples of one paint er’s works before moving on to study many examples of another’s works) would best help students learn the defi ning characteristics of each artist’s style. Massed practice of each artist’s works, one artist at a time, would better enable students to match artworks to artists later, compared to interleaved exposure to the works of different artists. The idea was that interleaving would be too hard and confusing; students would never be able to sort out the relevant dimensions. The researchers were wrong. The commonalities among one paint-er’s works that the students learned through massed practice proved less useful than the differences between the works of multiple paint ers that the students learned through interleaving. Interleaving enabled better discrimination and produced better scores on a later test that required matching the works with their paint ers. The interleaving group was also better able to match paint ers’ names correctly to new examples of their work that the group had never viewed during the learning phase. Despite these results, the students who participated in these experiments persisted in preferring massed practice, convinced that it served them better. Even after they took the test and could have realized from their own per for mance that interleaving was the better strategy for learning, they clung to their belief that the concentrated viewing of paintings by one artist was better. The myths of massed practice are hard to ex-orcise, even when you’re experiencing the evidence yourself.7

The power of interleaving practice to improve discrim-inability has been reaffi rmed in studies of people learning bird classifi cation. The challenge here is more complex than it might seem. One study addressed twenty different bird families (thrashers, swallows, wrens, fi nches, and so on). Within each family, students were presented with a dozen species

Mix Up Your Practice ê 55

(brown thrasher, curve-

billed thrasher, Bendire’s thrasher,

etc.). To identify a bird’s family, you consider a wide range of traits like size, plumage, behavior, location, beak shape, iris color, and so on. A problem in bird identifi cation is that members of a family share many traits in common but not all. For instance, many but not all thrashers have a long, slightly hooked beak. There are traits that are typical of a family but none that occur in all members of that family and can serve as unique identifi ers. Because rules for classifi cation can only rely on these characteristic traits rather than on defi ning traits (ones that hold for every member), bird classifi cation is a matter of learning concepts and making judgments, not simply memorizing features. Interleaved and variable practice proved more helpful than massed practice for learning the underlying concepts that unite and differentiate the species and families.

To paraphrase a conclusion from one of these studies, recall and recognition require “factual knowledge,” considered to be a lower level of learning than “conceptual knowledge.”

Conceptual knowledge requires an understanding of the interrelationships of the basic elements within a larger structure that enable them to function together. Conceptual knowledge is required for classifi cation. Following this logic, some people argue that practicing retrieval of facts and exemplars would fall short as a strategy for comprehending general characteristics that are required for higher levels of intellectual behavior.

The bird classifi cation studies suggest the opposite: strategies of learning that help students identify and discern complex prototypes (family resemblances) can help them grasp the kinds of contextual and functional differences that go beyond the acquisition of simple forms of knowledge and reach into the higher sphere of comprehension.8

Make It Stick ê 56

Improving Complex Mastery for

Medical Students

The distinction between straightforward knowledge of facts and deeper learning that permits fl exible use of the knowledge may be a little fuzzy, but it resonates with Douglas Larsen at Washington University Medical School in St. Louis, who says that the skills required for bird classifi cation are similar to those required of a doctor diagnosing what’s wrong with a patient. “The reason variety is important is it helps us see more nuances in the things that we can compare against,” he says. “That comes up a lot in medicine, in the sense that every patient visit is a test. There are many layers of explicit and implicit memory involved in the ability to discriminate between symptoms and their interrelationships.” Implicit memory is your automatic retrieval of past experience in interpreting a new one. For example, the patient comes in and gives you a story. As you listen, you’re consciously thinking through your mental library to see what fi ts, while also unconsciously polling your past experiences to help interpret what the patient is telling you. “Then you’re left with making a judgment call,” Larsen says.9

Larsen is a pediatric neurologist seeing patients in the university clinic and hospital. He’s a busy guy: in addition to practicing medicine, he supervises the work of physicians in training, he teaches, and as time permits, he conducts research into medical education, working in collaboration with cognitive psychologists. He’s drawing on all of these roles to redesign and strengthen the school’s training curriculum in pediatric neurology.

As you’d expect, the medical school employs a wide spectrum of instructional techniques. Besides classroom lectures

Mix Up Your Practice ê 57

and labs, students practice resuscitations and other procedures on high-tech mannequins in three simulation centers the school maintains. Each “patient” is hooked up to monitors, has a heartbeat, blood pressure, pupils that dilate and constrict, and the ability to listen and speak, thanks to a controller who observes and operates the mannequin from a back room. The school also makes use of “standardized patients,” actors who follow scripts and exhibit symptoms the students are required to diagnose. The center is set up like a regular medical clinic, and students must show profi ciency in all aspects of a patient encounter, from bedside manner, physical exam skills, and remembering to ask the full spectrum of pertinent questions to arriving at a diagnosis and treatment plan.

From studies of these teaching methods, Larsen has drawn some interesting conclusions. First— and this may seem self-evident: you do better on a test to demonstrate your competency at seeing patients in a clinic if your learning experience has involved seeing patients in a clinic. Simply reading about patients is not enough. However, on written fi nal exams, medical students who have examined patients and those who have learned via written tests do equally well. The reason is that in a written test the student is being given considerable structure and being asked for specifi c information. When examining the patient, you have to come up on your own with the right mental model and the steps to follow. Having practiced these steps on patients or simulated patients improves per for mance relative to just reading about how to do it. In other words, the kind of retrieval practice that proves most effective is one that refl ects what you’ll be doing with the knowledge later. It’s not just what you know, but how you practice what you know that determines how well the learning serves you later. As the sports adage goes, “practice like you play and you will play

Make It Stick ê 58

like you practice.” This conclusion lines up with other research into learning, and with some of the more sophisticated training practices in science and industry, including the increasingly broad use of simulators— not just for jet pi lots and medical students but for cops, towboat pi lots, and people in almost any fi eld you can name that requires mastery of complex knowledge and skills and where the stakes for getting it right are high. Book learning is not enough in these cases; actual hands-on practice is needed.

Second, while it is important for a medical student to build breadth by seeing a wide variety of patients manifesting different diseases, placing too much emphasis on variety runs the risk of underemphasizing repeated retrieval practice on the basics— on the typical way the disease presents itself in most patients.

“There’s a certain set of diseases that we want you to know very well,” Larsen says. “So we’re going to have you see these standardized patients again and again, and assess your perfor mance until you really have that down and can show us, ‘I really do that well.’ It’s not either/or, variety versus repetition.

We need to make sure we’re appropriately balanced, and also recognize that we sometimes fall into the trap of familiarity.

‘I’ve already seen a bunch of patients with this problem, I don’t need to keep seeing them.’ But really, repeated retrieval practice is crucial to long- term retention, and it’s a critical aspect of training.”

A third critical aspect is practical experience. For a doctor, seeing patients provides a natural cycle of spaced retrieval practice, interleaving, and variety. “So much of medicine is based on learning by experience, which is why, after the fi rst two years, we take students out of the classroom and start putting them into clinical settings. A huge question is, what is

Mix Up Your Practice ê 59

it about learning and experience that come together? We have lots of experiences we don’t learn from. What differentiates those that teach us something?”

One form of practice that helps us learn from experience, as the neurosurgeon Mike Ebersold recounted in Chapter 2, is refl ection. Some people are more given to the act of refl ection than others, so Doug Larsen has broadened his research to study how you might structure refl ection as an integral part of the training, helping students cultivate it as a habit. He is experimenting with requiring students to write daily or weekly summaries of what they did, how it worked, and what they might do differently next time to get better results. He specu-lates that daily refl ection, as a form of spaced retrieval practice, is probably just as critical in the real- world application of medicine as quizzing and testing are in building competencies in medical school.

What about the classroom lecture, or the typical in- service training conference that’s compressed over a couple of days?

Larsen fi gures his school’s interns spend 10 percent of their time sitting in conferences listening to lectures. It may be a talk on metabolic diseases, on different infectious diseases, or on different drugs. The speaker puts the PowerPoint slideshow up and starts going through it. Usually there’s lunch, and the docs eat, listen, and leave.

“In my mind, considering how much forgetting occurs, it’s very discouraging that we’re putting so many resources into an activity that, the way it is currently done, learning research tells us is so in effec tive. Medical students and residents go to these conferences and they have no repeated exposure what-soever to it. It’s just a matter of happenstance whether they end up fi nally seeing a patient in the future whose problem relates back to the conference topic. Otherwise, they don’t study the

Make It Stick ê 60

material, they are certainly not tested on the material, they just listen then they walk out.”

At a minimum, Larsen would like to see something done to interrupt the forgetting: give a quiz at the end of a conference and follow it with spaced retrieval practice. “Make quizzing a standard part of the culture and the curriculum. You just know every week you’re going to get in your email your ten questions that you need to work through.”

He asks, “How are we designing education and training systems that prevent or at least intervene in the amount of forgetting that goes on, and making sure they’re systematic throughout the school in support of what we’re trying to accomplish? As it stands now, medical resident programs are simply dictating: you have to have the curriculum, you have to have the conferences, and it ends there. They present these big conferences, they have all the faculty come through and give their talks. And in the end, what we actually accomplish is really kind of minimal.”10

These Principles Are Broadly Applicable College football might seem an incongruous place to look for a learning model, but a conversation with Coach Vince Dooley about the University of Georgia’s practice regime provides an intriguing case.

Dooley is authoritative on the subject. As head coach of Bulldogs football from 1964– 1988, he piled up an astonishing 201 wins with only 77 losses and 10 tied games, winning six conference titles and a national championship. He went on to serve as the university’s athletic director, where he built one of the most impressive athletics programs in the country.

We asked Coach Dooley how players go about mastering all the complexities of the game. His theories of coaching and

Mix Up Your Practice ê 61

training revolve around the weekly cycle of one Saturday game to the next. In that short period there’s a lot to learn: studying the opposition’s type of game in the classroom, discussing offensive and defensive strategies for opposing it, taking the discussion onto the playing fi eld, breaking the strategies down to the movements of individual positions and trying them out, knitting the parts into a whole, and then repeating the moves until they run like clockwork.

While all this is going on, the players must also keep their fundamental skills in top form: blocking, tackling, catching the ball, bringing the ball in, carry ing the ball. Dooley believes that (1) you have to keep practicing the fundamentals from time to time, forever, so you keep them sharp, otherwise you’re cooked, but (2) you need to change it up in practice because too much repetition is boring. The position coaches work with players individually on specifi c skills and then on how they’re playing their positions during team practice.

What else? There’s practicing the kicking game. There’s the matter of each player’s mastery of the playbook. And there are the special plays from the team’s repertoire that often make the difference between winning and losing. In Dooley’s narrative, the special plays stand as exemplars of spaced learning: they’re practiced only on Thursdays, so there’s always a week between sessions, and the plays are run in a varied sequence.

With all this to be done, it’s not surprising that a critical aspect of the team’s success is a very specifi c daily and weekly schedule that interleaves the elements of individual and team practice. The start of every day’s practice is strictly focused on the fundamentals of each player’s position. Next, players practice in small groups, working on maneuvers involving several positions. These parts are gradually brought together and run

Make It Stick ê 62

as a team. Play is speeded up and slowed down, rehearsed mentally as well as physically. By midweek the team is running the plays in real time, full speed.

“You’re coming at it fast, and you’ve got to react fast,”

Dooley said. “But as you get closer to game time, you slow it down again. Now it’s a kind of rehearsal without physical contact. The play basically starts out the same each time, but then what the opponent does changes it. So you’ve got to be able to adjust to that. You start into the motion and say, ‘If they react like this, then this is what you would do.’ You practice adjustments. If you do it enough times in different situations, then you’re able to do it pretty well in what ever comes up on the fi eld.”11

How does a player get on top of his playbook? He takes it home and goes over the plays in his mind. He may walk through them. Everything in practice can’t be physically stren-uous, Dooley said, or you’d wear yourself out, “so if the play calls for you to step this way and then go the other way, you can rehearse that in your mind, maybe just lean your body as if to go that way. And then if something happens where you have to adjust, you can do that mentally. By reading the playbook, rehearsing it in your mind, maybe taking a step or two to walk through it, you simulate something happening. So that kind of rehearsal is added to what you get in the classroom and on the fi eld.”

The fi nal quarterback meetings are held on Saturday morning, reviewing the game plan and running through it mentally.