Notes to Pages 12–20 ê 258

6. The study advice cited from the St. Louis Post- Dispatch is distributed by Newspapers in Education and can be seen online in “Testing 1, 2, 3! How to Study and Take Tests,” p14, at http:// nieonline .com /includes /hottopics /Testing %20Testing

%20123 .pdf, accessed November 2, 2013.

7. The studies showing the futility of mere repetition in recalling the details of what a penny looks like or where a fi re extinguisher is located in a building are in R. S. Nickerson & M. J.

Adams, Long term memory of a common object, Cognitive Psychology 11 (1979) , 287– 307, and A. D. Castel, M. Vendetti, & K. J. Holyoak, Inattentional blindness and the location of fi re extinguishers, Attention, Perception and Per for mance 74 (2012), 1391– 1396.

8. The experiment referred to by Tulving was reported in E. Tulving, Subjective or ga ni za tion and the effects of repetition in multi- trial free recall learning, Journal of Verbal Learning and Verbal Behavior 5 (1966), 193– 197.

9. The experiment on how rereading does not produce much benefi t in later retention is from A. A. Callender & M. A. McDaniel, The limited benefi ts of rereading educational texts, Contemporary Educational Psychology 34 (2009), 30– 41.

10. The survey showing that students prefer to reread as a study strategy is from Karpicke et al., Metacognitive strategies. Data were also taken from J. McCabe, Metacognitive awareness of learning strategies in undergraduates, Memory & Cognition 39 (2010), 462– 476.

11. Illusions of knowing will be a theme throughout this book. A general reference is Thomas Gilovich, How We Know What Isn’t So: The Fallibility of Human Reason in Everyday Life (New York: Free Press, 1991).

12. R. J. Sternberg, E. L. Grigorenko, & L. Zhang, Styles of learning and thinking matter in instruction and assessment, Perspectives on Psychological Science 3 (2008), 486– 506.

13. The project at Columbia Middle School is reported in M. A.

McDaniel, P. K. Agarwal, B. J. Huelser, K. B. McDermott, & H. L. Roediger (2011). Test- enhanced learning in a middle school science classroom: The effects of quiz frequency and placement. Journal of Educational Psychology, 103, 399– 414.

Notes to Pages 20–32 ê 259

14. The concept of testing as a learning tool is described in detail in Chapter 2. A general reference on material in this chapter (and other educational applications of cognitive psychology to education) is M. A. McDaniel & A. A. Callender, Cognition, memory, and education, in H. L. Roediger, Cognitive Psychology of Memory, vol. 2 of Learning and Memory: A Comprehensive Reference (Oxford: Elsevier, 2008), pp.

819– 844.

2. To Learn, Retrieve

1. Peter Brown interview of Michael Ebersold, December 31, 2011, Wabasha, MN. All quotes from Ebersold are from this interview.

2. The early work on forgetting curves was published by Her-mann Ebbinghaus in 1885 in a book translated into En glish as On Memory in 1913. The most recent version is H. Ebbinghaus, Memory: A contribution to experimental psychology (New York: Dover, 1964). Ebbinghaus is often viewed as the

“father” of the scientifi c study of memory.

3. The quotes from Aristotle and Bacon are from H. L. Roediger

& J. D. Karpicke, The power of testing memory: Basic research and implications for educational practice, Perspectives on Psychological Science 1 (2006), 181– 210.

4. Benedict Carey, “Forget what you know about good study habits,” New York Times, September 7, 2010. The study reported in this article was H. L. Roediger & J. D. Karpicke, Test- enhanced learning: Taking memory tests improves long-term retention, Psychological Science 17 (2006), 249– 255.

5. A. I. Gates, Recitation as a factor in memorizing, Archives of Psychology 6 (1917) and H. F. Spitzer, Studies in retention, Journal of Educational Psychology 30 (1939), 641– 656. These two large- scale studies with children in elementary and middle school were among the fi rst to document that taking a test or reciting material appearing in didactic texts improved retention for that material.

6. The study involving repeated testing versus repeated studying was E. Tulving, The effects of pre sen ta tion and recall of material

Notes to Pages 33–35 ê 260

in free- recall learning, Journal of Verbal Learning and Verbal Behavior 6 (1967), 175– 184. The study involving amounts of forgetting being reduced from testing is M. A. Wheeler & H. L.

Roediger, Disparate effects of repeated testing: Reconciling Ballard’s (1913) and Bartlett’s (1932) results, Psychological Science 3 (1992), 240– 245.

7. The positive effects of generation appear in L. L. Jacoby, On interpreting the effects of repetition: Solving a problem versus remembering a solution, Journal of Verbal Learning and Verbal Behavior 17 (1978), 649– 667. This laboratory experiment demonstrated that generation of target information does not have to be exceptionally challenging in order for generation to produce better retention relative to reviewing information to be learned.

8. Two papers describing the research at Columbia Middle School are H. L. Roediger, P. K. Agarwal, M. A. McDaniel, & K. McDermott, Test- enhanced learning in the classroom: Long- term improvements from quizzing, Journal of Experimental Psychology: Applied 17 (2011), 382– 395, and M. A. McDaniel, P. K.

Agarwal, B. J. Huelser, K. B. McDermott, & H. L. Roediger, Test- enhanced learning in a middle school science classroom: The effects of quiz frequency and placement, Journal of Educational Psychology 103 (2011), 399– 414. These companion papers were the fi rst to report well- controlled experiments on the benefi ts of quizzing for middle school students’ per formances on classroom exams in social studies and science. The fi ndings demonstrated that quizzing produced a signifi cant improvement relative to no- quizzing or directed review of target concepts on unit exams and on cumulative semester and end- of- year exams. In addition, in some cases a single well-placed review quiz produced benefi ts on the exams that were as robust as several repeated quizzes. For an interesting view of this project by one of the lead researchers, the fi rst teacher and the fi rst principal involved, see P. K. Agarwal, P. M. Bain, & R. W. Chamberlain, The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist. Educational Psychology Review 24 (2012), 437– 448.

Notes to Pages 37–40 ê 261

9. Peter Brown interview of Roger Chamberlain, October 27, 2011, Columbia Middle School, Illinois. All quotes from Chamberlain are from this interview.

10. Peter Brown interview of Andrew Sobel, December 22, 2011, St. Louis, Missouri. All quotes from Sobel are from this interview.

11. The experiments described here are by H. L. Roediger & J. D. Karpicke, Test- enhanced learning: Taking memory tests improves long-term retention, Psychological Science 17

(2006), 249– 255. Experiments showing that recall of studied prose passages produced better 2-

day and one-

week

retention than did restudy of the passages. For an earlier study with the same outcome using word lists, see C. P.

Thompson, S. K. Wenger, & C. A. Bartling, How recall facilitates subsequent recall: A reappraisal. Journal of Experimental Psychology: Human Learning and Memory 4 (1978), 210– 221. This experiment showed that massing study was better than practicing retrieval on an immediate test but not a delayed test.

12. Many studies exist on the effects of feedback. One is A. C. Butler & H. L. Roediger, Feedback enhances the positive effects and reduces the negative effects of multiple-choice testing.

Memory & Cognition 36 (2008), 604– 616. The experiments show that feedback strengthens the effects of testing alone, and that feedback may be more benefi cial when it’s slightly delayed. The authors also showed that that feedback enhances the positive effects and reduces the negative effects of multiple-choice testing. For motor skills, a classic reference is A. W. Sal-moni, R. A. Schmidt, and C. B. Walter, Knowledge of results and motor learning: A review and critical reappraisal. Psychological Bulletin 95 (1984), 355– 386. The authors proposed the guidance hypothesis of feedback effects on motor learning: Frequent immediate feedback can be detrimental to long- term learning— even though it helps immediate performance—

because it provides a crutch during practice that is no longer present on a delayed test.

13. The open- book test study was P. K. Agarwal, J. D. Karpicke, S. H. K. Kang, H. L. Roediger, & K. B. McDermott, Examining

Notes to Pages 41–42 ê 262

the testing effect with open- and closed- book tests, Applied Cognitive Psychology 22 (2008), 861– 876.

14. Studies comparing the types of tests are S. H. Kang, K. B. McDermott, H. L. Roediger, Test format and corrective feedback modify the effect of testing on long- term retention. Eu ro pe an Journal of Cognitive Psychology 19 (2007), 528–

558, and

M. A. McDaniel, J. L. Anderson, M. H. Derbish, & N. Morri-sette, Testing the testing effect in the classroom. Eu ro pe an Journal of Cognitive Psychology 19 (2007), 494– 513. These parallel experiments, one conducted in the laboratory and one in a college course, showed that a short- answer quiz with feedback produced better gains on fi nal tests than a recognition quiz with feedback. The implication is that the testing effect is more robust when more effort is required for retrieval, as it typically is for short- answer questions than for multiple choice questions. However, some studies have shown that multiple choice tests, especially when given repeatedly, can have as much positive effect in the classroom as a short- answer test; see K. B. McDermott, P. K. Agarwal, L. D’Antonio, H. L. Roediger, & M.

A. McDaniel, Both multiple-

choice and short-