He scratched his head and changed the syllabus once again.
This time he announced that there would be nine quizzes during the semester, and he was explicit about when they would be. No surprises, and no midterm or fi nal exams, because he didn’t want to give up that much of his lecture time.
Despite fears that enrollments would plummet again, they actually increased by a handful. “Unlike the pop quizzes, which kids hate, these were all on the syllabus. If they missed one it was their own fault. It wasn’t because I surprised them or was being pernicious. They were comfortable with that.” Sobel took satisfaction in seeing attendance improve as well. “They would skip some classes on the days they didn’t have a quiz, particularly the spring semester, but they showed up for the quizzes.”
Like the course, the quizzes were cumulative, and the questions were similar to those on the exams he used to give, but the quality of the answers he was getting by midsemester was much better than he was accustomed to seeing on the midterms. Five years into this new format, he’s sold on it. “The quality of discussions in class has gone way up. I see that big a difference in their written work, just by going from three exams to nine quizzes.” By the end of the semester he has them writing paragraphs on the concepts covered in class, sometimes a full- page essay, and the quality is comparable to what he’s seeing in his upper division classes.
“Anybody can design this structure. But I also realize that, Oh, god, if I’d done this years ago I would have taught them that much more stuff. The interesting thing about adopting this strategy is I now recognize that as good a teacher as I
To Learn, Retrieve ê 39
might think I am, my teaching is only a component of their learning, and how I structure it has a lot to do with it, maybe even more.” Meanwhile, the course enrollment has grown to 185 and counting.
Exploring Nuances
Andy Sobel’s example is anecdotal and likely refl ects a variety of benefi cial infl uences, not least being the cumulative learning effects that accrue like compounded interest when course material is carried forward in a regime of quizzes across an entire semester. Nonetheless, his experience squares with empirical research designed to tease apart the effects and nuances of testing.
For example, in one experiment college students studied prose passages on various scientifi c topics like those taught in college and then either took an immediate recall test after the initial exposure or restudied the material. After a delay of two days, the students who took the initial test recalled more of the material than those who simply restudied it (68 v. 54 percent), and this advantage was sustained a week later (56 v. 42
percent). Another experiment found that after one week a study- only group showed the most forgetting of what they initially had been able to recall, forgetting 52 percent, compared to a repeated- testing group, who forgot only 10 percent.11
How does giving feedback on wrong answers to test questions affect learning? Studies show that giving feedback strengthens retention more than testing alone does, and, interestingly, some evidence shows that delaying the feedback briefl y produces better long- term learning than immediate feedback. This fi nding is counterintuitive but is consistent with researchers’
Make It Stick ê 40
discoveries about how we learn motor tasks, like making lay-ups or driving a golf ball toward a distant green. In motor learning, trial and error with delayed feedback is a more awkward but effective way of acquiring a skill than trial and correction through immediate feedback; immediate feedback is like the training wheels on a bicycle: the learner quickly comes to depend on the continued presence of the correction.
In the case of learning motor skills, one theory holds that when there’s immediate feedback it comes to be part of the task, so that later, in a real- world setting, its absence becomes a gap in the established pattern that disrupts per for mance.
Another idea holds that frequent interruptions for feedback make the learning sessions too variable, preventing establish-ment of a stabilized pattern of per for mance.12
In the classroom, delayed feedback also yields better long-term learning than immediate feedback does. In the case of the students studying prose passages on science topics, some were shown the passage again even while they were asked to answer questions about it, in effect providing them with continuous feedback during the test, analogous to an open- book exam. The other group took the test without the study material at hand and only afterward were given the passage and instructed to look over their responses. Of course, the open-book group performed best on the immediate test, but those who got corrective feedback after completing the test retained the learning better on a later test. Delayed feedback on written tests may help because it gives the student practice that’s spaced out in time; as discussed in the next chapter, spacing practice improves retention.13
Are some kinds of retrieval practice more effective for long-term learning than others? Tests that require the learner to
To Learn, Retrieve ê 41
supply the answer, like an essay or short- answer test, or simply practice with fl ashcards, appear to be more effective than simple recognition tests like multiple choice or true/false tests.
However, even multiple choice tests like those used at Columbia Middle School can yield strong benefi ts. While any kind of retrieval practice generally benefi ts learning, the implication seems to be that where more cognitive effort is required for retrieval, greater retention results. Retrieval practice has been studied extensively in recent years, and an analysis of these studies shows that even a single test in a class can produce a large improvement in fi nal exam scores, and gains in learning continue to increase as the number of tests increases.14
Whichever theories science eventually tells us are correct about how repeated retrieval strengthens memory, empirical research shows us that the testing effect is real— that the act of retrieving a memory changes the memory, making it easier to retrieve again later.
How widely is retrieval practice used as a study technique? In one survey, college students were largely unaware of its effectiveness. In another survey, only 11 percent of college students said they use this study strategy. Even when they did report testing themselves, they mostly said they did it to discover what they didn’t know, so they could study that material more.
That’s a perfectly valid use of testing, but few students realize that retrieval itself creates greater retention.15
Is repeated testing simply a way to expedite rote learning? In fact, research indicates that testing, compared to rereading, can facilitate better transfer of knowledge to new contexts and problems, and that it improves one’s ability to retain and
Make It Stick ê 42
retrieve material that is related but not tested. Further research is needed on this point, but it seems that retrieval practice can make information more accessible when it is needed in various contexts.
Do students resist testing as a tool for learning? Students do generally dislike the idea of tests, and it’s not hard to see why, in par tic u lar in the case of high- stakes tests like midterms and fi nals, where the score comes with signifi cant consequences.
Yet in all studies of testing that reported students’ attitudes, the students who were tested frequently rated their classes more favorably at the end of the semester than those tested less frequently. Those who were frequently tested reached the end of the semester on top of the material and did not need to cram for exams.
How does taking a test affect subsequent studying? After a test, students spend more time restudying the material they missed, and they learn more from it than do their peers who restudy the material without having been tested. Students whose study strategies emphasize rereading but not self- testing show over-confi dence in their mastery. Students who have been quizzed have a double advantage over those who have not: a more accurate sense of what they know and don’t know, and the strengthening of learning that accrues from retrieval practice.16
Are there any further, indirect benefi ts of regular, low- stakes classroom testing? Besides strengthening learning and retention, a regime of this kind of testing improves student attendance. It increases studying before class (because students
To Learn, Retrieve ê 43
know they’ll be quizzed), increases attentiveness during class if students are tested at the end of class, and enables students to better calibrate what they know and where they need to bone up. It’s an antidote to mistaking fl uency with the text, resulting from repeated readings, for mastery of the subject.
Frequent low-
stakes testing helps dial down test anxiety among students by diversifying the consequences over a much larger sample: no single test is a make- or- break event. And this kind of testing enables instructors to identify gaps in students’ understanding and adapt their instruction to fi ll them.
These benefi ts of low- stakes testing accrue whether instruction is delivered online or in the classroom.17
The Takeaway
Practice at retrieving new knowledge or skill from memory isa potent tool for learning and durable retention. This is true for anything the brain is asked to remember and call up again in the future— facts, complex concepts, problem- solving techniques, motor skills.
Effortful retrieval makes for stronger learning and retention. We’re easily seduced into believing that learning is better when it’s easier, but the research shows the opposite: when the mind has to work, learning sticks better. The greater the effort to retrieve learning, provided that you succeed, the more that learning is strengthened by retrieval. After an initial test, delaying subsequent retrieval practice is more potent for reinforcing retention than immediate practice, because delayed retrieval requires more effort.
Repeated retrieval not only makes memories more durable but produces knowledge that can be retrieved more readily, in more varied settings, and applied to a wider variety of problems.
Make It Stick ê 44
While cramming can produce better scores on an immediate exam, the advantage quickly fades because there is much greater forgetting after rereading than after retrieval practice.
The benefi ts of retrieval practice are long- term.
Simply including one test (retrieval practice) in a class yields a large improvement in fi nal exam scores, and gains continue to increase as the frequency of classroom testing increases.
Testing doesn’t need to be initiated by the instructor. Students can practice retrieval anywhere; no quizzes in the classroom are necessary. Think fl ashcards— the way second graders learn the multiplication tables can work just as well for learners at any age to quiz themselves on anatomy, mathematics, or law. Self- testing may be unappealing because it takes more effort than rereading, but as noted already, the greater the effort at retrieval, the more will be retained.
Students who take practice tests have a better grasp of their progress than those who simply reread the material. Similarly, such testing enables an instructor to spot gaps and miscon-ceptions and adapt instruction to correct them.
Giving students corrective feedback after tests keeps them from incorrectly retaining material they have misunderstood and produces better learning of the correct answers.
Students in classes that incorporate low-stakes quizzing
come to embrace the practice. Students who are tested frequently rate their classes more favorably.
What about Principal Roger Chamberlain’s initial concerns about practice quizzing at Columbia Middle School— that it might be nothing more than a glorifi ed path to rote learning?
