Notes to Pages 37–40 ê 261
9. Peter Brown interview of Roger Chamberlain, October 27, 2011, Columbia Middle School, Illinois. All quotes from Chamberlain are from this interview.
10. Peter Brown interview of Andrew Sobel, December 22, 2011, St. Louis, Missouri. All quotes from Sobel are from this interview.
11. The experiments described here are by H. L. Roediger & J. D. Karpicke, Test- enhanced learning: Taking memory tests improves long-term retention, Psychological Science 17
(2006), 249– 255. Experiments showing that recall of studied prose passages produced better 2-
day and one-
week
retention than did restudy of the passages. For an earlier study with the same outcome using word lists, see C. P.
Thompson, S. K. Wenger, & C. A. Bartling, How recall facilitates subsequent recall: A reappraisal. Journal of Experimental Psychology: Human Learning and Memory 4 (1978), 210– 221. This experiment showed that massing study was better than practicing retrieval on an immediate test but not a delayed test.
12. Many studies exist on the effects of feedback. One is A. C. Butler & H. L. Roediger, Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing.
Memory & Cognition 36 (2008), 604– 616. The experiments show that feedback strengthens the effects of testing alone, and that feedback may be more benefi cial when it’s slightly delayed. The authors also showed that that feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. For motor skills, a classic reference is A. W. Sal-moni, R. A. Schmidt, and C. B. Walter, Knowledge of results and motor learning: A review and critical reappraisal. Psychological Bulletin 95 (1984), 355– 386. The authors proposed the guidance hypothesis of feedback effects on motor learning: Frequent immediate feedback can be detrimental to long- term learning— even though it helps immediate performance—
because it provides a crutch during practice that is no longer present on a delayed test.
13. The open- book test study was P. K. Agarwal, J. D. Karpicke, S. H. K. Kang, H. L. Roediger, & K. B. McDermott, Examining
Notes to Pages 41–42 ê 262
the testing effect with open- and closed- book tests, Applied Cognitive Psychology 22 (2008), 861– 876.
14. Studies comparing the types of tests are S. H. Kang, K. B. McDermott, H. L. Roediger, Test format and corrective feedback modify the effect of testing on long- term retention. Eu ro pe an Journal of Cognitive Psychology 19 (2007), 528–
558, and
M. A. McDaniel, J. L. Anderson, M. H. Derbish, & N. Morri-sette, Testing the testing effect in the classroom. Eu ro pe an Journal of Cognitive Psychology 19 (2007), 494– 513. These parallel experiments, one conducted in the laboratory and one in a college course, showed that a short- answer quiz with feedback produced better gains on fi nal tests than a recognition quiz with feedback. The implication is that the testing effect is more robust when more effort is required for retrieval, as it typically is for short- answer questions than for multiple choice questions. However, some studies have shown that multiple choice tests, especially when given repeatedly, can have as much positive effect in the classroom as a short- answer test; see K. B. McDermott, P. K. Agarwal, L. D’Antonio, H. L. Roediger, & M.
A. McDaniel, Both multiple-
choice and short-
answer quizzes enhance later exam per for mance in middle and high school classes, Journal of Experimental Psychology: Applied (in press).
15. These studies examined students’ use of testing as a study strategy: J. D. Karpicke, A. C. Butler, & H. L. Roediger, III, Metacognitive strategies in student learning: Do students practice retrieval when they study on their own?, Memory 17
(2009), 471– 479, and N. Kornell & R. A. Bjork, The promise and perils of self regulated study, Psychonomic Bulletin & Review 14 (2007), 219– 224. These studies reported the surveys of college students’ use of retrieval practice as study technique.
16. Taking a test— even when one fails to correctly recall information on it— enhances learning from a new study episode. See K. M. Arnold & K. B. McDermott, Test- potentiated learning: Distinguishing between the direct and indirect effects of tests, Journal of Experimental Psychology: Learning, Memory and Cognition 39 (2013), 940– 945.
Notes to Pages 43–48 ê 263
17. This is a study of frequent low- stakes testing: F. C. Leeming, The exam- a-day procedure improves per for mance in psychology classes, Teaching of Psychology 29 (2002), 210– 212. The author found that in sections in which he gave students a short test at the start of every class the students attended class more often and felt that they studied more and learned more than students in classes with only four tests throughout the semester. Final test per for mance for the different sections (quiz a day or no quiz a day) confi rmed students’ impressions. Another interesting study conducted in a classroom is K. B. Lyle & N.
A. Crawford, Retrieving essential material at the end of lectures improves per for mance on statistics exams, Teaching of Psychology 38 (2011), 94– 97.
Two reviews of research on retrieval practice and testing appear in H. L. Roediger & J. D. Karpicke, The power of testing memory: Basic research and implications for educational practice, Perspectives on Psychological Science 1 (2006), 181– 210.
This paper represents a comprehensive review of laboratory and classroom studies over nearly one hundred years of research, showing that testing can be a powerful learning tool. A more recent review points to many benefi ts of frequent testing in addition to the direct benefi t from retrieval practice: H. L.
Roediger, M. A. Smith, & A. L. Putnam, Ten benefi ts of testing and their applications to educational practice, in J. Mestre & B. H. Ross (eds.), Psychology of Learning and Motivation (San Diego: Elsevier Academic Press, 2012). This chapter provides a summary of the host of potential benefi ts of using testing as a learning technique.
3. Mix Up Your Practice
1. The report of the beanbag study can be found in R. Kerr & B.
Booth, Specifi c and varied practice of motor skill, Perceptual and Motor Skills 46 (1978), 395– 401.
2. Many
well-
controlled experiments conducted with a variety of materials and training tasks provide solid evidence that massed practice (doing the same thing over and over repeatedly, a strategy often preferred by learners) is inferior to spacing and
Notes to Pages 48–53 ê 264
interleaving of practice for learning and retention. A review of the literature on the spacing effect in memory can be found in N. J. Cepeda, H. Pashler, E. Vul, J. T. Wixted, & D. Rohrer, Distributed practice in verbal recall tasks: A review and quantita-tive synthesis, Psychological Bulletin 132 (2006), 354– 380.
3. The surgery study is C-A. E. Moulton, A. Dubrowski, H. Mac-Rae, B. Graham, E. Grober, & R. Reznick, Teaching surgical skills: What kind of practice makes perfect?, Annals of Surgery 244 (2006), 400– 409. This study randomly assigned surgical residents to either a normal daylong intensive lesson on a surgical procedure or to an experimental lesson that spaced four short periods of instruction over several weeks. The fi ndings, showing better retention and application of the surgical techniques after spaced instruction, prompted the medical school to reexamine their standard instructional procedure of cramming instruction on a par tic u lar surgical technique into one intensive session.
4. The study showing the benefi t of interleaving in mathematics problems is D. Rohrer & K. Taylor, The shuffl ing of mathematics problems improves learning, Instructional Science 35
