Forgetting doesnt work like you think. Benjamin Keep, PhD, JD 2022-06-01 | Everyone forgets things. We forget what we read, what we study, what we learn. But where do our memories go, after we've forgotten them? And why did we forget in the first place?00:00 Introduction00:30 Memories do battle with each other1:55 A second way that memories interfere with one anotherSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferences:A great discussion of forgetting (especially theories of cue competition) comes from:Anderson, M. C., & Neely, J. H. (1996). Interference and inhibition in memory retrieval. In Memory (pp. 237-313). Academic Press. http://www.memorycontrol.net/an1996.pdfThe "drinking helps you remember" idea has a relatively long history.Here's an earlier replication of some of the first work:Mueller, C. W., Lisman, S. A., & Spear, N. E. (1983). Alcohol enhancement of human memory: Tests of consolidation and interference hypotheses. Psychopharmacology, 80(3), 226-230. https://www.academia.edu/download/46474229/bf0043615820160614-5333-a2occx.pdfAnd here's a more recent piece:Moulton, P. L., Petros, T. V., Apostal, K. J., Park II, R. V., Ronning, E. A., King, B. M., & Penland, J. G. (2005). Alcohol-induced impairment and enhancement of memory: A test of the interference theory. Physiology & behavior, 85(3), 240-245. pubag.nal.usda.gov/download/46671/pdfThe same is true, by the way of other drugs that impair memory formation. They tend to facilitate memory for events and information that happened prior to impairment.
Online Learning Can Be Awesome. Heres How. Benjamin Keep, PhD, JD 2024-07-31 | There are a lot of wonderful learning resources on the internet. How should we go about using them?GiveInternet.org Donation Link: giveinternet.org/BenjaminKeep00:00 Introduction00:45 Finding the right learning resources2:04 Getting practice4:02 Finding a learning community5:08 Planning your practice5:54 What happens next?8:18 A worthy causeFor updates on my course and website, click here: https://forms.gle/9TEUw7bj3YTs7B4H6To sign up to my (infrequent) newsletter, click here: http://www.benjaminkeep.com.Edited by Presage: pre-sage.comResources featured in the video:Kalid Azad's Better Explained: betterexplained.comPaul's Online Math Notes: https://tutorial.math.lamar.edu/Susan Rigetti's guide to learning physics: susanrigetti.com/physicsAndy Guitar: andyguitar.co.ukSimplyPiano: hellosimply.com/simply-pianoTenenbaum, M., & Pollard, H. (1985). Ordinary differential equations: an elementary textbook for students of mathematics, engineering, and the sciences. Courier Corporation. archive.org/details/TenenbaumAndPollardOrdinaryDifferentialEquationsYoung, H. D., Freedman, R. A., Ford, A. L., Sears, F. W., & Zemansky, M. W. (2014). Sears and Zemansky's University Physics with Modern Physics.
Why Cant People Solve These Simple Problems? Benjamin Keep, PhD, JD 2024-07-10 | The cognitive reflection test was meant to measure one thing: the tendency for someone to stop and think before giving a quick answer. The simplicity of the test made it a widely used measure for critical thinking. But some clever experiments illustrate that the CRT is not so simple after all.00:00 Introduction00:19 The CRT problems1:20 The structure of CRT problems3:04 Some hints about what's going wrong4:53 The structure of insight problems6:27 Explanation Number One7:57 Explanation Number Two8:57 Explanation Number ThreeI'm launching a membership site. If you want in, go here: https://forms.gle/JoMA1Xa5Nnnwm8VU9Recommended reading: benjaminkeep.com/recommended-readingSign up to my free (infrequently published) newsletter here: benjaminkeep.comREFERENCESThis video is mostly based off of Patel, N., Baker, S. G., & Scherer, L. D. (2019). Evaluating the cognitive reflection test as a measure of intuition/reflection, numeracy, and insight problem solving, and the implications for understanding real-world judgments and beliefs. Journal of Experimental Psychology: General, 148(12), 2129–2153. doi.org/10.1037/xge0000592This is the (classic) paper that introduced the CRT as a measure of reflection. Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives, 19, 25– 42. http://dx.doi.org/10.1257 089533005775196732. It also includes correlations between the math section of the SAT and the CRT (.46) vs the verbal section of the SAT and the CRT (.24).On the CRT’s correlation to tests measuring numeracy being about .5, see Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring Risk Literacy: The Berlin Numeracy Test. Judgment and Decision Making, 7(1), 25–47. doi.org/10.1017/S1930297500001819On the associations between reflection/analytical thinking and various beliefs, see Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015). Everyday Consequences of Analytic Thinking. Current Directions in Psychological Science, 24(6), 425-432. doi.org/10.1177/0963721415604610This mentions some of the earlier results correlating CRT performance to rational thinking, but also extends the CRT to a seven-question test. The four added questions are probably even more reliant on numerical ability than the first three. Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & cognition, 39(7), 1275-1289.
Everyone misses this problem solving step Benjamin Keep, PhD, JD 2024-06-26 | There's lots of good advice on problem solving. But the details matter.00:00 Introduction1:08 The problem solving process1:54 The important sentence3:02 Skipping the step vs doing the thingBTW, if you're learning physics on your own, I think Susan Rigetti's recommendations are indispensable. susanrigetti.com/physicsThe "2000" problems video I reference is here: youtu.be/_5MQUMtRPmMReferences:The physics book in question is: Young, H. D., Freedman, R. A., & Ford, A. L. (20). Sears and Zemansky's University Physics with Modern Physics. 15th Edition. Pearson education. Although I think any of the earlier editions (14th, 13th, international, etc.) are good. Again, credit to Susan Rigetti for this recommendation.The papers I referenced on conceptual knowledge and procedural skill:Kim, E., & Pak, S. J. (2002). Students do not overcome conceptual difficulties after solving 1000 traditional problems. American Journal of Physics, 70(7), 759-765. (available at http://sdsu-physics.org/sdsu_per/articles/ProblemS_ConUnderst.pdf)Byun, T., & Lee, G. (2014). Why students still can't solve physics problems after solving over 2000 problems. American Journal of Physics, 82(9), 906-913. (available at aapt.scitation.org/doi/abs/10.1119/1.4881606)
Learning scientist learns how to cook Benjamin Keep, PhD, JD 2024-06-12 | A lot of the research on learning focuses on learning in the classroom. But what about all the other stuff we need to learn? Here are some thoughts on learning how to cook.00:00 Introduction0:29 Cognitive load theory3:17 Retrieval practice4:17 Skill analysis6:12 Deliberate practice8:25 Leveraging prior knowledgeEdited by Presage: https://pre-sage.com.Sign up to my newsletter here: http://www.benjaminkeep.com.
What makes something memorable? Benjamin Keep, PhD, JD 2024-05-29 | This is my "everything I know about encoding" video. Well, not quite everything. But best I could do in 45 minutes.00:00 Introduction00:46 What can we encode?1:46 Two issues in encoding research4:43 Depth of processing9:17 The generation effect13:31 Imagery17:28 Memory palaces and “time palaces”20:10 The drawing effect23:36 Distinctiveness27:50 Value31:42 Integration37:17 Transfer appropriate processing43:32 The complete list of themesTo learn more about the membership site, click here: https://forms.gle/9TEUw7bj3YTs7B4H6Edited by Presage: https://pre-sage.com.References:Perhaps this goes without saying, but each of the issues that I mention are considerably more nuanced than how I've presented them here. But hopefully the video gives you the basic ideas so that you can dive in further, if you want.Depth of processing: Craik, F. I., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of experimental Psychology: general, 104(3), 268.Generation: Bertsch, S., Pesta, B. J., Wiscott, R., & McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & cognition, 35, 201-210.Imagery: Leopold, C., & Mayer, R. E. (2015). An imagination effect in learning from scientific text. Journal of Educational Psychology, 107(1), 47–63. doi.org/10.1037/a0037142"Time palaces" and memory palaces Bouffard, N., Stokes, J., Kramer, H. J., & Ekstrom, A. D. (2018). Temporal encoding strategies result in boosts to final free recall performance comparable to spatial ones. Memory & Cognition, 46(1), 17–31. doi.org/10.3758/s13421-017-0742-zValue and encoding: Hennessee, J. P., Patterson, T. K., Castel, A. D., & Knowlton, B. J. (2019). Forget me not: Encoding processes in value-directed remembering. Journal of Memory and Language, 106, 29–39. doi.org/10.1016/j.jml.2019.02.001Drawing: Fernandes, M. A., Wammes, J. D., & Meade, M. E. (2018). The Surprisingly Powerful Influence of Drawing on Memory. Current Directions in Psychological Science, 27(5), 302–308. doi.org/10.1177/0963721418755385Wammes, J. D., Meade, M. E., & Fernandes, M. A. (2016). The drawing effect: Evidence for reliable and robust memory benefits in free recall. Quarterly Journal of Experimental Psychology, 69(9), 1752–1776. doi.org/10.1080/17470218.2015.1094494Acting things out: Engelkamp, J., & Zimmer, H. D. (1997). Sensory factors in memory for subject-performed tasks. Acta Psychologica, 96(1-2), 43-60.Production and distinctiveness: Ozubko, J. D., & MacLeod, C. M. (2010). The production effect in memory: evidence that distinctiveness underlies the benefit. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(6), 1543.Integration and self-explanations: Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182.Renkl, A. (2002). Worked-out examples: Instructional explanations support learning by self-explanations. Learning and instruction, 12(5), 529-556.Integration and video games/passages:Arena, D. A., & Schwartz, D. L. (2014). Experience and explanation: Using videogames to prepare students for formal instruction in statistics. Journal of Science Education and Technology, 23, 538-548.Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–5223.Transfer appropriate processing: Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16(5), 519–533. doi.org/10.1016/S0022-5371(77)80016-9
The Essentials of Problem Solving Benjamin Keep, PhD, JD 2024-01-17 | An introduction to the psychology of problem solving. Featured problems: the towers of Hanoi, the Chinese ring puzzle, the Wason 4-card selection task, the candle problem, Roman matchstick problems, and toothpick shape problems.00:00 A quick note00:47 The problem state space and the towers of Hanoi4:45 Problems of representation and the Chinese ring puzzle6:42 Context and variations of the Wason 4-card selection task9:42 Introduction to insight problems: the candle problem11:05 Differences between insight and incremental problems12:15 Barriers to insight: Roman matchstick problems17:30 Insight problems: too big of a distinction?19:08 Well-structured and ill-structured problems21:11 Representation and argument23:34 Becoming a better problem solver: toothpick problems26:45 Domain-specific knowledge and strategy change30:55 What transfers across problem-solving domains?Sign up to my newsletter here: http://www.benjaminkeep.com.ACKNOWLEDGEMENTSThe towers of Hanoi footage comes from Math Playground's version of the towers.The Chinese ring puzzle footage comes from Youtuber @Jojikiba, here youtube.com/watch?v=nydIl2VO12IThe candle problem visuals: youtube.com/watch?v=PEvThYxtnFQ. If you made it or you know who did, please let me know. The link above seems to be the oldest and most complete version.Paul Zeitz photo: imo-official.org/advisory.aspx?year=2019Math and STEP problems: imo-official.org/problems/IMO2022SL.pdf and usmle.org/sites/default/files/2021-10/Step_1_Sample_Items.pdf, respectively.Physics problems: Badeau, R., White, D. R., Ibrahim, B., Ding, L., & Heckler, A. F. (2017). What works with worked examples: Extending self-explanation and analogical comparison to synthesis problems. Physical Review Physics Education Research, 13(2), 020112. doi.org/10.1103/PhysRevPhysEducRes.13.020112Visit www.eternagame.org for more info on Eterna.The Chess explanation clip: @BetterChessTraining.The Starcraft 2 clip: an @ESChamp video, here: youtube.com/watch?v=4okBbHaCQ98The go examples come from Sensei’s library: senseis.xmp.net/?CornersThenSidesThenCenterStarcraft 2 map: news.blizzard.com/en-us/starcraft2/23223597/new-ladder-maps-for-2019-season-4Parity example: artofproblemsolving.com/wiki/index.php?title=ParityThe free body diagram is from a Math and Science video: youtube.com/watch?v=mI11vHsRnC8The map of Easter Island: en.wikipedia.org/wiki/File:Easter_Island_map-en.svgThe cashflow statement: en.wikipedia.org/wiki/Cash_flow_statement#cite_note-21, but is originally from Epstein, Barry J.; Eva K. Jermakowicz (2007). Interpretation and Application of International Financial Reporting Standards. John Wiley & Sons. pp. 91–97. ISBN 978-0-471-79823-1.REFERENCESSimon, H. A., & Newell, A. (1971). Human problem solving: The state of the theory in 1970. American psychologist, 26(2), 145.The tower of Hanoi state space was adapted from Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive science, 18(1), 87-122.The Chinese Ring puzzle state space (for 5 rings) is adapted from Kotovsky, K. & Simon, H. A. (1990). What Makes Some Problems Really hard: Explorations in the Problem Space of Difficulty. Cognitive Psychology, 22(2), 143–183.The “underage drinking” example of the Wason 4-card task comes from this classic:Griggs, R. A., & Cox, J. R. (1982). The elusive thematic‐materials effect in Wason's selection task. British journal of psychology, 73(3), 407-420.A short meta-analysis on the Wason 4-card task: Ragni, M., Kola, I., & Johnson-Laird, P. (2017). The Wason Selection task: A Meta-Analysis. In CogSci. researchgate.net/profile/Ilir-Kola/publication/322682384_The_Wason_Selection_Task_A_Meta-Analysis/links/5a688e6f4585156abdffd541/The-Wason-Selection-Task-A-Meta-Analysis.pdfOn the insight experience: Webb, M. E., Little, D. R., & Cropper, S. J. (2016). Insight Is Not in the Problem: Investigating Insight in Problem Solving across Task Types. Frontiers in Psychology, 7. frontiersin.org/articles/10.3389/fpsyg.2016.01424The Matchstick Roman numeral problems and the discussion of constraints and chunks come from: Öllinger, M., Jones, G., & Knoblich, G. (2008). Investigating the Effect of Mental Set on Insight Problem Solving. Experimental Psychology, 55(4), 269–282. doi.org/10.1027/1618-3169.55.4.269On strategic change, see: Siegler, R. S. (2002). Microgenetic studies of self-explanation. Microdevelopment: Transition processes in development and learning, 31, 58.On the importance of representations in ill-structured domains, see: Chandrasekharan, S., & Nersessian, N. J. (2011). Building cognition: the construction of external representations for discovery. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 33, No. 33).
Learning Foreign Language Vocabulary - The Fundamentals Benjamin Keep, PhD, JD 2023-12-07 | Some thoughts on learning foreign language vocabulary.The “Chinese is Hard” podcast: open.spotify.com/show/6DdGLH8neJoAkggOZu4VrNWilliam Hart’s Youtube channel: youtube.com/@will3267Danyo Pang’s Youtube channel: youtube.com/@thedanyopang00:00 That's a lot of words00:24 Different kinds of vocabulary words2:06 Word mapping between languages3:16 Vocabulary goes beyond words6:25 A story about prior knowledge9:22 Learning words by example and in context10:31 Flashcards12:22 Comprehensible input13:50 Pushed outputSign up to my email newsletter, here: http://www.benjaminkeep.comAcknowledgements:Definitions of “chew” comes from dictionary.com; definitions for “hard,” “take for granted,” and "repetition" come from Merriam-Webster, slightly adapted.References:The observations in this video are a mish-mash of my own personal experience and my reading of the literature on language learning. The best survey of that literature that I know of is How Languages are Learned by Patsy Lightbrown and Nina Spada. The 4th edition is here: bookshop.org/a/91541/9780194541268 (bookshop links are affiliate links, so if you purchase through this link, I do get a small commission; I would try to find this from your local library, though).It’s written with foreign language teachers in mind (not necessarily foreign language students), but hits on many of the key ideas, mentions a lot of great studies that I won’t enumerate here, and offers a relatively balanced portrait of the research on language learning.The 10,000 word estimate I mentioned at the beginning of the video is low and I wish I would have been more accurate. A lot depends on how you’re defining a word and how you define what “knowing” means. This is a solid paper on estimating the vocabulary of native American English speakers: Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). How Many Words Do We Know? Practical Estimates of Vocabulary Size Dependent on Word Definition, the Degree of Language Input and the Participant’s Age. Frontiers in Psychology, 7. frontiersin.org/articles/10.3389/fpsyg.2016.01116.The upshot is that they estimate about ~11,000 word families and 42,000 lemmas on average. A lemma is a root word that you can derive into other things (baked, bake, baking would not be three separate words - just one lemma). Word families are larger groups of related words with similar forms and related meanings.Comprehensible Input is one of the main ideas in Stephen Krashen’s Monitor Theory. This is the classic piece that lays it out: Krashen, S. (1981). Second language acquisition. Second Language Learning, 3(7), 19-39. https://www.academia.edu/download/35238869/second_languge_acquisition_and_learning.pdfThe idea of pushed output comes from Merrill Swain and her studies of proficiency in immersive classrooms. She's got a lot to say on it over the years, but here's a representative example: Swain, M. (1997). Collaborative dialogue: Its contribution to second language learning. riull.ull.es/xmlui/bitstream/handle/915/30603/RCEI_34_%281997%29_07.pdf?sequence=1There’s much more to say about comprehensible input and pushed output, and I’ve got plans to release videos on those two ideas at least.
The Skill of Learning from Lectures Benjamin Keep, PhD, JD 2023-11-15 | Some viewers asked for a video on learning from lectures. So I finally made one.Link to hear news on my membership site: https://forms.gle/2XzhvnGptxeYc28n8Sign up to my free newsletter, Avoiding Folly, here: http://www.benjaminkeep.com00:00 Starting with a question00:53 Lectures and learning2:10 On the popularity of the lecture format2:56 Preparing for a lecture4:38 Making the most of reading assignments6:33 The goals of preparation: cognitive load7:37 Create relevant prior knowledge8:10 Get relevant practical experience8:29 Begin the organizational process9:06 Going to the lecture10:54 Good attention11:47 Attention drops during lecture12:29 Dealing with online lectures13:37 Reviewing after the lecture14:42 Creative ways of reviewing16:20 Fundamentals are key17:26 Frameworks to help you review19:50 Review vs doing homework21:38 A little goes a long wayACKNOWLEDGEMENTSThe scrolling text when I was talking about cognitive load comes from an excerpt from an old Vanity Fair magazine, written by P.G. Wodehouse, available here: vanityfair.com/culture/2014/10/physical-culture-peril-pg-wodehouse-best-of-early-vanity-fair.The green screen Youtube video (talking about stopping at 10 minutes, 30 minutes) came from here: youtube.com/watch?v=q-7LExVpU4gI used an example lecture from Scott Page’s Model Thinking on Coursera, found here. The specific excerpt, was taken from the first lecture on aggregation. coursera.org/lecture/model-thinking/aggregation-UWXGF. I definitely recommend Scott Page’s work, especially if you’re interested in social science and social dynamics.The "taking a test" example came from an MIT course on differential equations: https://ocw.mit.edu/courses/18-03-differential-equations-spring-2010/resources/mit18_03s10_ex1/.REFERENCESOn attention during lectures. The basic take-away is that passive learning during lectures is similar to a vigilance task (asking people to monitor for low frequency signals from a background of noise - radar monitoring for instance). 10-30 minutes in, steep attention decrements.Young, M. S., Robinson, S., & Alberts, P. (2009). Students pay attention!: Combating the vigilance decrement to improve learning during lectures. Active Learning in Higher Education, 10(1), 41–55. doi.org/10.1177/1469787408100194If you are a teacher looking to improve student learning during lectures, this piece from almost 40 years ago still holds up. If you’re a science teacher at the undergraduate level, I recommend visiting the Carl Wieman Science Education Initiative.Gibbs, G., Habeshaw, S., & Habeshaw, T. (1987). Improving Student Learning During Lectures. Medical Teacher, 9(1), 11–20. doi.org/10.3109/01421598709028976This is a more modern piece covering some of the same ground:Cerbin, W. (2018). Improving student learning from lectures. Scholarship of Teaching and Learning in Psychology, 4(3), 151–163. doi.org/10.1037/stl0000113
40 Hours to Learn Mental Math | The riveting conclusion? Benjamin Keep, PhD, JD 2023-10-26 | It can't be true! I finally published the update to my "learning mental math in 40 hours video"! How many eons have passed? First part is here: youtu.be/uCi0_tjOiyQ00:00 Last time...00:40 Is tracking my hours helpful?01:08 Is the diagnostic test a good measure?01:55 Speed vs accuracy03:22 Interleaved vs deliberate practice04:54 The importance of seeing progress05:50 Midway diagnostic results06:15 Practice and performance gains07:08 Skill integration09:33 What the test measures10:12 Are your mistakes changing?11:15 What I’m focusing on next12:21 The major system14:17 28 hours in14:39 The development of intuition16:29 35 hours in18:38 The return of the major system19:45 Lessons for learning something on your ownSign up to my email newsletter on reasoning, social science, and decision-making: http://www.benjaminkeep.comInterested in courses on learning and studying? Sign up to be notified when my site launches: https://forms.gle/xzi1ovwkzCHKvmP58The math course I was working from was: thegreatcourses.com/courses/secrets-of-mental-math, but Arthur Benjamin also has a book teaching the same things: bookshop.org/a/91541/9780307338402. The bookshop link is an affiliate link so I get a buck or something if you buy it through this link (thanks, if you do!).Some thoughts on interleaved practice: youtu.be/AWTYfzxBwPgSome thoughts on deliberate practice: youtu.be/WbUOY9ioIqw (tackling some myths about deliberate practice); youtu.be/3SZDj5TEhU0 (on criticisms of the deliberate practice model)A quick intro to the novice to expert transition: youtu.be/qwDzjQsM3yQ
How to learn from a book (maybe) | note-taking, visualizations, spacing | history example Benjamin Keep, PhD, JD 2023-10-04 | Does anyone really know how to learn from a book? Here is a record of my first serious attempt to understand and remember what I was reading for the long term, along with various justifications (and rationalizations!) for what I did. 0:00 What I'm trying to do0:50 What does it mean to learn from a book?2:27 Writing down themes4:27 The back-and-forth6:01 The function of notes7:01 Taking paths through the material7:36 Why do my visualizations suck?8:53 Getting precise10:20 Why these (bad) visualizations?11:14 Spacing things out12:20 What I would do differently next timeIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter here: http://www.benjaminkeep.com.REFERENCES:The book is China: A History by John Keay.My practices here come from a variety of sources, but here's some representative work.This piece is about teaching students higher-level reading strategies to aid deep comprehension (constructing inferences and explanations, self-monitoring, etc.). A useful read that I largely agree with: McNamara, D. S. (2010). Strategies to read and learn: Overcoming learning by consumption. Medical Education, 44(4), 340–346. doi.org/10.1111/j.1365-2923.2009.03550.xOn free recall and retrieval:Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021). Retrieval Practice Consistently Benefits Student Learning: A Systematic Review of Applied Research in Schools and Classrooms. Educational Psychology Review, 33(4), 1409–1453. doi.org/10.1007/s10648-021-09595-9Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. doi.org/10.1016/j.tics.2010.09.003Karpicke, J. D., & Roediger, H. L. (2008). The Critical Importance of Retrieval for Learning. Science, 319(5865), 966–968. doi.org/10.1126/science.1152408On the value of "spacing things out" AKA long inter-study intervals, see:Latimier, A., Peyre, H., & Ramus, F. (2021). A Meta-Analytic Review of the Benefit of Spacing out Retrieval Practice Episodes on Retention. Educational Psychology Review, 33(3), 959–987. doi.org/10.1007/s10648-020-09572-8Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi.org/10.1037/0033-2909.132.3.354Acknowledgements:Image of Liu Bang (founding emperor of the Han Dynasty) comes from: en.wikipedia.org/wiki/File:Liu-bang.jpg. It’s actually a Ming-era depictionImage of Xiang Yu (Liu Bang’s rival) comes from: en.wikipedia.org/wiki/File:Xiang_Yu.jpgXuanzang image: en.wikipedia.org/wiki/File:Xuanzang_w.jpgXuanzang’s route: en.wikipedia.org/wiki/File:Xuanzang_route.jpgZhang Qian’s route: en.wikipedia.org/wiki/File:Han_Expansion.pngZheng He’s ships: en.wikipedia.org/wiki/File:ZhengHeShips.gifZheng He’s route: en.wikipedia.org/wiki/File:Voyages_of_Zheng_He.png (author: SY)Topographic map: en-us.topographic-map.com/map-pxv3q/China
Youtubers tested the generation effect. Did it work? Benjamin Keep, PhD, JD 2023-09-20 | Did Youtubers replicate the generation effect? At first, it might not seem like it, but let's take a closer look.0:00 This video is a follow-up video0:39 Was there an effect?3:12 An attempt to visualize the effect4:36 Sources of variation5:13 Encoding strategies and non-native speakers7:30 Why what I just did wasn't really legit8:14 Variation among experimental designs10:16 A common source of error12:12 Oh no, it's an ad...To be notified when my membership site launches, go here: https://forms.gle/9Q8aDCrSPEkpx8Zn8Sign up to my email newsletter here: http://www.benjaminkeep.comREFERENCESThe first one is the meta-analysis that I took screenshots of for the video. The second one is more recent and delves deeper into the theories that explain the effect.Bertsch, S., Pesta, B. J., Wiscott, R., & McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & Cognition, 35(2), 201–210. doi.org/10.3758/BF03193441McCurdy, M. P., Viechtbauer, W., Sklenar, A. M., Frankenstein, A. N., & Leshikar, E. D. (2020). Theories of the generation effect and the impact of generation constraint: A meta-analytic review. Psychonomic Bulletin & Review, 27(6), 1139–1165. doi.org/10.3758/s13423-020-01762-3
The Experiment That Teaches People How To Learn Benjamin Keep, PhD, JD 2023-09-06 | Can you teach yourself to learn more effectively? Memory researcher Elizabeth Bjork thinks so. Participate in a short experiment that illustrates an important link between encoding and retrieval.0:00 An introduction to Bjork's experiments0:34 Let's try a little experiment.4:55 Figuring out your score.5:32 What's the expected result?5:58 How did Dr. Bjork use this to help people learn?7:05 The surprising result.7:44 What's really driving the learning in the second round?9:49 The larger point.The follow-up video is here: youtu.be/7N7tIM7sIxUIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter here: http://www.benjaminkeep.com.REFERENCESThis video and the experiments I describe in it was based on a series of experiments led by Elizabeth Bjork, renowned memory researcher. The first one is the most readable summary of the research; the other three are a bit dense, but give lots of details about how the experiments were run.Bjork, E. L., de Winstanley, P. A., & Storm, B. C. (2007). Learning how to learn: Can experiencing the outcome of different encoding strategies enhance subsequent encoding? Psychonomic Bulletin & Review, 14(2), 207–211. doi.org/10.3758/BF03194053A. de Winstanley, P., & Ligon Bjork, E. (1997). Processing Instructions and the Generation Effect: A Test of the Multifactor Transfer-appropriate Processing Theory. Memory, 5(3), 401–422. doi.org/10.1080/741941392DeWinstanley, P. A., & Bjork, E. L. (2004). Processing strategies and the generation effect: Implications for making a better reader. Memory & Cognition, 32(6), 945–955. doi.org/10.3758/BF03196872Bjork, E. L., & Storm, B. C. (2011). Retrieval experience as a modifier of future encoding: Another test effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1113–1124. doi.org/10.1037/a0023549This is the one where they illustrate that students don’t have to experience the benefits of encoding to improve their encoding strategy. Rather, it seems like just knowing the format of the test and having some experience with different encoding strategies is enough (separately and likely additively together) to lead to superior encoding.Storm, B. C., Hickman, M. L., & Bjork, E. L. (2016). Improving encoding strategies as a function of test knowledge and experience. Memory & Cognition, 44(4), 660–670. doi.org/10.3758/s13421-016-0588-9The word lists used in the video were generated from Friendly, M. & Dubins, M. (2019), Paivio et al. Word List Generator, Online application, http://euclid.psych.yorku.ca/shiny/Paivio, Accessed: 2023-06-23
Active Control vs Passive Control in Randomized Controlled Trials | Whats the Difference? Benjamin Keep, PhD, JD 2023-06-21 | One of the critical decisions that researchers make when designing a controlled trial is the choice of the control: what should be the control group? The answer depends on what we're interested in testing. The distinction I make here is an important one to keep in mind.0:00 An under-appreciated distinction.0:34 What is a passive control group?1:36 What is an active control group?If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter here: http://www.benjaminkeep.com.REFERENCESA good explanation of the difference between passive control groups and active control groups begins on page 116, here: Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016). Do “Brain-Training” Programs Work? Psychological Science in the Public Interest, 17(3), 103–186. doi.org/10.1177/1529100616661983
Two Reasons NOT to Copy Memory Athletes Benjamin Keep, PhD, JD 2023-06-14 | Memory athletes use special techniques to perform amazing feats of memory. Are these techniques appropriate for classroom materials?0:00 Memory athletes can do some amazing things0:34 Techniques memory athletes use1:59 The argument against using mnemonics - Reason #14:13 The argument against using mnemonics - Reason #25:50 The argument in favor of using mnemonics8:23 Incorporating mnemonic techniques into your studying8:41 The case of single-digit multiplication9:18 Chinese emperors and Chinese geography10:40 The periodic table11:25 Deciding whether to use mnemonic techniques12:30 Memory emergenciesIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8For more on mnemonic techniques, check out the Art of Memory: artofmemory.comSign up to my email newsletter, Avoiding Folly, here: http://www.benjaminkeep.comREFERENCESThe piece below reports something like the consensus among cognitive psychologists: that mnemonics are pretty good for remembering serial items of unrelated information for short periods of time, but have unproven relevance in the classroom.Putnam, A. L. (2015). Mnemonics in education: Current research and applications. Translational Issues in Psychological Science, 1(2), 130–139. doi.org/10.1037/tps0000023This is a piece from a psychologist who taught her students the method of loci (AKA memory palace) that showed some modestly positive results, at least on list learning and general attitude toward the method.McCabe, J. A. (2015). Location, Location, Location! Demonstrating the Mnemonic Benefit of the Method of Loci. Teaching of Psychology, 42(2), 169–173. doi.org/10.1177/0098628315573143For a novel application of the method of loci, here is a piece exploring how it might be leveraged for emotional regulation.Werner-Seidler, A., & Dalgleish, T. (2016). The Method of Loci Improves Longer-Term Retention of Self-Affirming Memories and Facilitates Access to Mood-Repairing Memories in Recurrent Depression. Clinical Psychological Science, 4(6), 1065–1072. doi.org/10.1177/2167702615626693ACKNOWLEDGEMENTSFootage from a card memorization event is from: youtube.com/watch?v=OApSOU7NIAwThe periodic table is from the NIH website - amazing! pubchem.ncbi.nlm.nih.gov/periodic-tableThe map of China rivers comes from Antonioga, here: en.wikipedia.org/wiki/File:China_rivers.svgImage of the Loess landscape, in China, comes from Till Niermann, here: en.wikipedia.org/wiki/File:Loess_landscape_china.jpgThe image of Gansu province (camel photo) comes from Sigismund von Dobschütz, here: en.wikipedia.org/wiki/File:Mondsichelsee-06.JPGThe map of traditional geographical and cultural regions in China comes from SY, here: en.wikipedia.org/wiki/File:China_Traditional_Divisions.pngImages related to evolutionary processes come from: Elimbis at Wikimedia commons (commons.wikimedia.org/wiki/File:Mutation_and_selection_diagram.svg) Esurnir at the English Wikipedia (Allele frequency, en.wikipedia.org/wiki/File:Allele-frequency.png); and speciation here (en.wikipedia.org/wiki/File:Speciation_modes_edit.svg)Image of the Tang Dynasty comes from Seasonsinthesun at Wikimedia Commons (commons.wikimedia.org/wiki/File:Tang_Protectorates.png). Image of the Roman Empire comes from Paulusburg at Wikimedia Commons (commons.wikimedia.org/wiki/File:Mapa_Imperio_Occidental_a%C3%B1o_418.svg)
Teach Your Kid Basic Math Benjamin Keep, PhD, JD 2023-05-17 | Teaching your kid basic math is a little more complicated than it seems. Here I go over some important learning goals and what I've tried with my own son to try to achieve them.0:00 So... you want to teach your kid math?0:15 Four categories of learning goals1:35 Some important conceptual learning goals3:14 Some important procedural learning goals4:36 What does your kid associate math with?5:09 Becoming a problem solver5:48 Setting up the time and the environment6:25 How to use examples and abstractions7:13 Developing more sophisticated strategies8:28 Letting them solve the problem9:07 Developing a skeptical and thorough problem solver9:42 Flip the asking script10:32 Modeling how to respond to mistakes11:41 "Secrets"12:18 Interesting ways of incorporating number lines12:55 Scaffolding estimation skillsIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my newsletter, Avoiding Folly, here: www.benjaminkeep.comREFERENCES:For readable summaries of the research in these areas, check out these two excellent posts by Gwen Dewar: parentingscience.com/preschool-board-game-math (preschool math games) and parentingscience.com/preschool-math-lessons (general advice). Parenting Science is just an excellent resource, generally.On concreteness fading and the power of using both concrete and abstract representations together, see the two references below:Fyfe, E. R., & Nathan, M. J. (2019). Making “concreteness fading” more concrete as a theory of instruction for promoting transfer. Educational Review, 71(4), 403–422. doi.org/10.1080/00131911.2018.1424116Moreno, R., Ozogul, G., & Reisslein, M. (2011). Teaching with concrete and abstract visual representations: Effects on students' problem solving, problem representations, and learning perceptions. Journal of educational psychology, 103(1), 32.On the relationship between conceptual and procedural knowledge and the role that different kinds of examples can play in developing both, see: Rittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101(3), 529. https://dash.harvard.edu/bitstream/handle/1/4889483/CC3b_Final.pdfOn the use of number lines for learning: Lin, C. H. (2022). Developing mental number line games to improve young children’s number knowledge and basic arithmetic skills. Journal of Experimental Child Psychology, 222, 105479. sciencedirect.com/science/article/abs/pii/S0022096522001084
Study Less, Study Smart(er) - Extending Marty Lobdells Study Advice Benjamin Keep, PhD, JD 2023-05-10 | Marty Lobdell's video, "Study Less, Study Smart" remains excellent advice for college students on how to study. Here are answers to some of the most common questions that come up when you follow his advice.0:00 A brief introduction0:23 What kinds of study breaks should you take?1:59 Where should you study?3:34 What kinds of notes should you take?4:38 Deep processing, shallow processing, and transfer-appropriate processing7:11 How should you structure study groups?10:17 Ways to study that avoid confusion between recognition and recall11:49 The skill of reading textbooksIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter here: http://www.benjaminkeep.comRELATED VIDEOSMarty Lobdell's Study Less, Study Smart is here: youtube.com/watch?v=IlU-zDU6aQ0For alternatives to highlighting and flashcards, see:How to do free recall - youtu.be/FatnXnlwAc4How to Use Free Recall to Learn More Effectively - youtu.be/evIuDvQDDSYA simple study tip: stop highlighting - youtu.be/_g3NojwBOXwFor my take on note-taking:The surprising truth about note-taking - youtu.be/cRQqH18wJgwOn reading textbooks (and reading more generally):Three simple tricks to read textbooks more effectively - youtu.be/neQ7D90R0xoLearn more from every paragraph - youtu.be/WRjsOU6mOp4On attitude and framing:The most common obstacle to effective studying - youtu.be/6i7HrP84DMwREFERENCESOn restful periods, see: Wamsley, E. J. (2019). Memory consolidation during waking rest. Trends in cognitive sciences, 23(3), 171-173. ncbi.nlm.nih.gov/pmc/articles/PMC7024394/pdf/nihms-1557224.pdfThe screenshot of the graph showing wakeful resting benefits to wakeful active groups comes from research on auditory learning, here: Gottselig, J. M., Hofer-Tinguely, G., Borbely, A. A., Regel, S. J., Landolt, H. P., Retey, J. V., & Achermann, P. (2004). Sleep and rest facilitate auditory learning. Neuroscience, 127(3), 557-561.On eyestrain, see: Kaur, K., Gurnani, B., Nayak, S., Deori, N., Kaur, S., Jethani, J., ... & Mishra, D. (2022). Digital Eye Strain-A Comprehensive Review. Ophthalmology and Therapy, 11(5), 1655-1680.https://link.springer.com/article/10.1007/s40123-022-00540-9On the benefits of self-testing study strategies, see: Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6On the weak benefits of highlighting and a fantastic review of study strategies more generally, see: Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychological Science in the Public Interest, 14(1), 4–58. doi.org/10.1177/1529100612453266On the value of teaching (or preparing to teach) others, see: Okita, S. Y., & Schwartz, D. L. (2013). Learning by Teaching Human Pupils and Teachable Agents: The Importance of Recursive Feedback. Journal of the Learning Sciences, 22(3), 375–412. doi.org/10.1080/10508406.2013.807263The classic piece on transfer appropriate processing is here: Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16(5), 519–533. doi.org/10.1016/S0022-5371(77)80016-9. For a more modern view in the context of an experiment, see this piece: A. de Winstanley, P., & Ligon Bjork, E. (1997). Processing Instructions and the Generation Effect: A Test of the Multifactor Transfer-appropriate Processing Theory. Memory, 5(3), 401–422. doi.org/10.1080/741941392On deep and shallow processing, see: Craik, F. I. M. (2002). Levels of processing: Past, present... and future? Memory, 10(5–6), 305–318. doi.org/10.1080/09658210244000135On the memory benefit of learning in different environments, one of the classics is: Smith, S. M., Glenberg, A., & Bjork, R. A. (1978). Environmental context and human memory. Memory & Cognition, 6(4), 342-353.On links between exercise, sleep, and learning, see: Roig, M., Cristini, J., Parwanta, Z., Ayotte, B., Rodrigues, L., de Las Heras, B., ... & Wright, D. L. (2022). Exercising the sleepy-ing brain: exercise, sleep, and sleep loss on memory. Exercise and sport sciences reviews, 50(1), 38-48.On links between walking and creative thinking, see: Oppezzo, M., & Schwartz, D. L. (2014). Give your ideas some legs: the positive effect of walking on creative thinking. Journal of experimental psychology: learning, memory, and cognition, 40(4), 1142.ACKNOWLEDGEMENTSSpanish Inquisition clip from: youtube.com/watch?v=yKQ_sQKBASM
How to Read Peer-reviewed Articles on Experiments in Learning Benjamin Keep, PhD, JD 2023-05-03 | Reading peer-reviewed social science research can be quite challenging. But it’s a skill that lets you go beyond media reports of the latest study. Here’s 15 minutes of advice on how to do read experimental studies specifically.0:00 An intro to reading social science research0:39 What do learning researchers study?1:54 The basics of experimental designs5:01 What’s the goal of reading research?5:16 The first big question: why?6:58 The second big question: what happened?8:12 What’s the participant’s experience?9:07 A tip for reading complex graphs.10:23 The third big question: the explanation.11:04 Some complexities when measuring learning13:45 What’s the question again?14:45 Last thoughtsIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my newsletter, Avoiding Folly, here: www.benjaminkeep.comREFERENCESClips come from the following pieces:Example of a longitudinal study: Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363.Example of qualitative data and mixed qualitative and quantitative data come from: Reading Abraham Lincoln: An Expert/Expert Study in the Interpretation of Historical Texts. (2003). 1–28.Example of quantitative data comes from: Bjork, E. L., & Storm, B. C. (2011). Retrieval experience as a modifier of future encoding: Another test effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1113–1124. doi.org/10.1037/a0023549Example of abstract/laboratory experiments comes from: Bjork, R. A. (1975). Retrieval as a memory modifier: An interpretation of negative recency and related phenomena.Example of an applied study (using social studies material in the classroom): Roediger, H. L., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17(4), 382–395. doi.org/10.1037/a0026252Examples of an experimental study, data visualization that describes the nature of the materials and one that describes the course of the experiment from: Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.Research studying the effects of delayed vs immediate feedback are legion. This provides good background reading: Fyfe, E. R., & Rittle-Johnson, B. (2016). Feedback both helps and hinders learning: The causal role of prior knowledge. Journal of Educational Psychology, 108(1), 82–97. doi.org/10.1037/edu0000053The example of worked examples vs open problem solving comes from some of Sweller’s earlier experiments. Here’s a typical example. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and instruction, 2(1), 59-89.The statistics game and passage examples I drew from Arena, D. A., & Schwartz, D. L. (2013). Experience and Explanation: Using Videogames to Prepare Students for Formal Instruction in Statistics. Journal of Science Education and Technology, 23(4), 538–548. doi.org/10.1007/s10956-013-9483-3The screenshot of a statistics passage is from one of the first pages of Regression and Other Stories by Gelman, Hill, and Vehtari, a great textbook on regression.The design-based research example comes from Ketelhut, D. J., Nelson, B. C., Clarke, J., & Dede, C. (2010). A multi‐user virtual environment for building and assessing higher order inquiry skills in science. British Journal of Educational Technology, 41(1), 56-68. They created a virtual environment called “River City” to facilitate science inquiry experiences.The “delayed measures” being different than immediate measures graph is from Taylor, K., & Rohrer, D. (2010). The effects of interleaved practice. Applied Cognitive Psychology, 24(6), 837–848. doi.org/10.1002/acp.1598The “Stats Invaders” clip (the statistics game) comes courtesy of Can Zhang, here: youtube.com/watch?v=J7rx1g8g1Uk
How to Use Free Recall to Learn More Effectively Benjamin Keep, PhD, JD 2023-04-12 | People have some good questions about how to use free recall to study. Here, I try to answer them.0:00 Is one recall mode better than another?1:21 What should the recall interval be?2:20 Do I need to master the material first?2:43 Which is better, encoding or retrieval?4:03 Is free recall just vomiting out words?4:57 Some further thoughtsSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comREFERENCESOn the differences between different modes of recall, see the three references below:Janczyk, M., Aßmann, M., & Grabowski, J. (2018). Oral versus written recall of long-term memory items: Replicating and extending the writing superiority effect across knowledge domains. The American Journal of Psychology, 131(3), 263-272. (Finding a written superiority effect)Bekerian, D. A., & Dennett, J. L. (1990). Spoken and written recall of visual narratives. Applied Cognitive Psychology, 4(3), 175-187. (spoken superiority)Sauerland, M., Krix, A. C., van Kan, N., Glunz, S., & Sak, A. (2014). Speaking is silver, writing is golden? The role of cognitive and social factors in written versus spoken witness accounts. Memory & Cognition, 42, 978-992. (favoring written, but not finding large differences; in the eyewitness area)For a wonderful summary of how tests effect learning, check out: Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6On test expectancy effects (how expecting a test influences encoding), see: McDaniel, M. A., Blischak, D. M., & Challis, B. (1994). The effects of test expectancy on processing and memory of prose. Contemporary educational psychology, 19(2), 230-248.
What is Actually Wrong with Rote Learning? Benjamin Keep, PhD, JD 2023-04-05 | A lot of people don't like rote learning because it's not very fun. But is it effective? I explore this question with a rote learning example in learning to write Chinese characters.0:00 An Intro to Rote Learning1:01 Drawback 1 - Blocked Practice2:57 Drawback 2 - Massed Practice4:05 Drawback 3 - Skill IsolationIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comREFERENCESFor a recent take on spaced (or distributed) practice, including evidence from neurological studies, read:Gerbier, E., & Toppino, T. C. (2015). The effect of distributed practice: Neuroscience, cognition, and education. Trends in Neuroscience and Education, 4(3), 49–59. doi.org/10.1016/j.tine.2015.01.001If you want a more academic take, discussing theories and the available evidence circa 2006, check out:Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi.org/10.1037/0033-2909.132.3.354For a readable piece on the advantages (and potential drawbacks) of interleaved practice, check out:Kang, S. H. (2016). The benefits of interleaved practice for learning. In From the laboratory to the classroom (pp. 79-90). Routledge. taylorfrancis.com/chapters/edit/10.4324/9781315625737-8/benefits-interleaved-practice-learning-sean-kangAnd a great piece on how to leverage interleaved, spaced, and skill-building practice in the classroom is here:Hughes, C. A., & Lee, J. Y. (2019). Effective approaches for scheduling and formatting practice: Distributed, cumulative, and interleaved practice. Teaching Exceptional Children, 51(6), 411-423. researchgate.net/profile/Joo-Young-Lee-8/publication/338701579_Effective_Approaches_for_Scheduling_and_Formatting_Practice_Distributed_Cumulative_and_Interleaved_Practice/links/5db7099aa6fdccac19b5d67f/Effective-Approaches-for-Scheduling-and-Formatting-Practice-Distributed-Cumulative-and-Interleaved-Practice.pdf
This Simple Method Helps You Learn More from Podcasts (and Audiobooks!) Benjamin Keep, PhD, JD 2023-03-29 | There's a lot of great podcasts out there. But what should we do if we want to learn from them? Here's a simple method.00:00 Some drawbacks of using podcasts to learn01:49 Step 1 - Activate prior knowledge02:42 Step 2 - Free recall and fill in the gaps05:49 Step 3 - Consolidate and extendIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comACKNOWLEDGEMENTSScreenshots of Twilight of the Aesir and Hardcore History come from dancarlin.com.Jutland and Saxony screenshots come from Wikipedia.I use Miro for the visualization software clip.REFERENCESMy comments about what free recall self-tests can do for you are based on the large literature related to the testing effect. At a high-level, frequent tests enhance learning in a variety of ways.For a thorough discussion of the possible benefits (including anticipatory/motivational aspects and organizational aspects, like I mention in the video), check out:Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6Here is a recent meta-analysis exploring different potential mechanisms and applications of the testing effect(s):Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin, 147(4), 399–435. doi.org/10.1037/bul0000309
The Knowledge That Underlies Everything | Tacit Knowledge Benjamin Keep, PhD, JD 2023-03-22 | What is tacit (or implicit) knowledge and what does it have to do with learning? To answer this question, we have to explore bike riding, expert blindness, a Chinese room, and how we remember.0:00 An intro to tacit knowledge0:45 Let's try an experiment1:42 "Weak tacit knowledge"3:36 Expert blindness5:31 "Body knowledge"8:03 "Social tacit knowledge" and Searle's Chinese Room11:10 How do you learn tacit knowledge?If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comREFERENCESThe bones of the video come from: Collins, H. (2019). Tacit and explicit knowledge. University of Chicago press. I changed the labels he gave for the different kinds of tact knowledge he identifies because I found them pretty confusing.Michael Polanyi is credited with first identifying and discussing tacit knowledge. His thoughts can be found in: Polanyi, M., & Sen, A. (2009). The tacit dimension. University of Chicago press.The "explicit to implicit" model of skill acquisition is commonly exemplified by the Dreyfus model:Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in directed skill acquisition. California Univ Berkeley Operations Research Center. https://apps.dtic.mil/sti/pdfs/ADA084551.pdfA good introduction to Searle's Chinese Room (with plenty of references), can be found here: https://plato.stanford.edu/entries/chinese-room/ACKNOWLEDGEMENTSStar Trek clip from youtube.com/watch?v=hdjL8WXjlGIThe physics of bike riding visuals come from this gem from 1970: Jones, D. E. (1970). The stability of the bicycle. Physics today, 23(4), 34-40. Available here: https://home.phys.ntnu.no/brukdef/undervisning/tfy4145/arkiv/2010/diverse/UnridableBicycle.pdfThe clip of a robot riding a bicycle came from: youtube.com/watch?v=mT3vfSQePcs“Frampold” entry and other rare word screenshots from the OED found here: oed.com/view/Entry/74166The recipe screenshots are from Serious Eats: seriouseats.com/the-food-lab-how-to-make-creamy-vegetable-soup; seriouseats.com/spicy-carrot-and-ginger-soup-with-harissa.Urban Dictionary add word screen found here: my.urbandictionary.com/add.phpImage of Harry Collins from Alexei Kouprianov on Wikipedia. upload.wikimedia.org/wikipedia/commons/5/5f/Harry_Collins.20101118.jpg
A Simple Way to Learn Complex Skills Benjamin Keep, PhD, JD 2023-03-15 | Sometimes we have to learn very complex skills that can't be broken apart into more basic skills. What do we do then?0:00 An intro to the experiment0:21 Skills with many interdependent parts1:00 The study that explored learning complex skills2:33 The results3:48 Try it outIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com*There was a wrinkle to this study that I didn't mention, which is that groups 2, 3, and 4 all had a different kind of display on the game for the first six sessions, which gave them slightly different information than the standard game. The idea was to support learner's focus on part of the game, but it also meant that there's more than one difference between the control and the other groups. The enhanced display could explain some of the difference between the control group and the other groups, but doesn't explain the differences between groups 2 and 3 and group 4. And the variable-priority approach (what I call rotating focus) that the study helped establish is well-supported by other studies as well - see the meta-analysis in the reference section for more support.ACKNOWLEDGMENTSStarcraft clip from LowkoTV: youtube.com/watch?v=kz9M2uYcFyESpace Fortress mobile game footage from: play.google.com/store/apps/details?id=com.mgmamac.space_fortress&gl=USSpace Fortress DOS game footage from: classicreload.com/space-fortress.htmlYou can play an updated version of the original Space Fortress game at brainturk.com/space-fortressREFERENCESThis video is based on the paper below. I replicated their figure 2 starting at around 2:40. Gopher, D., Weil, M., & Siegel, D. (1989). Practice under changing priorities: An approach to the training of complex skills. Acta Psychologica, 71(1–3), 147–177. doi.org/10.1016/0001-6918(89)90007-3Here is a meta-analysis illustrating the efficacy of what I call "rotating focus" but what is called "variable-priority" training in the literature. Wickens, C. D., Hutchins, S., Carolan, T., & Cumming, J. (2013). Effectiveness of Part-Task Training and Increasing-Difficulty Training Strategies: A Meta-Analysis Approach. Human Factors, 55(2), 461–470. doi.org/10.1177/0018720812451994And if you search more articles from Daniel Gopher, you will see other works on the same theme. Like this one: Brickner, M., & Gopher, D. (1981). Improving Time-Sharing Performance by Enhancing Voluntary Control on Processing Resources. TECHNION-ISRAEL INST OF TECH HAIFA CENTER OF HUMAN ENGINEERING AND INDUSTRIAL SAFETY RESEARCH. https://apps.dtic.mil/sti/pdfs/ADA118558.pdf
Learn More From Every Paragraph | Active Reading Example Benjamin Keep, PhD, JD 2023-03-08 | Reading is about more than just decoding the words and sentences. It's about thinking and reasoning and truly engaging with the material. Here is an example of a more active reading approach.0:00 The paragraph in question1:20 Some ways to approach learning from this paragraph2:08 How I read it5:17 What does reading something actually mean?If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comHere's the video I plug at the end: What Speed readers won’t tell you - youtu.be/D2aZ3THL8BQHere is the book that I took the paragraph from: Emling, S. (2009). The fossil hunter: dinosaurs, evolution, and the woman whose discoveries changed the world. St. Martin's Press. bookshop.org/a/91541/9780230103429 (I am a bookshop.org affiliate, which means I get small commission if you purchase through this link.)I should say that there is more to good reading than what I have outlined in the video, but at least it demonstrates some basics.For a concise summary of what good readers do, check out the first page of the following article: Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading comprehension. Journal of education, 189(1-2), 107-122. asset-pdf.scinapse.io/prod/45283437/45283437.pdfHappy reading!
What Speed Readers Wont Tell You Benjamin Keep, PhD, JD 2023-03-01 | Speed reading seems like a way to learn more efficiently. But is it? I explore what the research says about speed reading (and normal reading).0:00 How to explore the claims of speed readers1:09 Tim Ferris's speed reading techniques3:26 The relationship between eye movements and reading comprehension4:49 Tests of reading comprehension5:52 Maximum words per minute (WPM)6:46 Why do people believe in speed reading?If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comMore learning myths: youtu.be/jcMk3e7u7zMMore on when learning fast backfires: youtu.be/wRb32j6_pD4Thomas Frank’s videos on speed reading go over the research quite well, too. Check them out here: youtube.com/watch?v=jv2BdHXRD3Q (part 1), youtube.com/watch?v=JL4WMHyUhdc (part 2).ACKNOWLEDGEMENTSOpening screenshots of the Spreeder app found here: spreeder.comExcerpts of Tim Ferris's speed reading video comes from: youtube.com/watch?v=ZwEquW_Yij0. The screenshot of comments comes from there, too.Demo of the Spritz speed reading app comes from: readmei.comI used this speed reading tool to display the words "This doesn't work at all" in the video: tools-unite.com/tools/speed-reading-appThe sentence I used to illustrate that people pause at the end of sentences comes from The Man Without Qualities by Robert Musil, translated by Sophie Wilkins and Burton Pike (bookshop.org/p/books/man-without-qualities-robert-musil/18761811): "There is, in short, no great idea that stupidity cannot put to its own uses; it can move in all directions, and put on all the guises of truth." A difficult, but hilarious and profound and tragic book.REFERENCESThe most thorough description of the reading process, and why speed reading doesn’t work, is here: Rayner, K., Schotter, E. R., Masson, M. E., Potter, M. C., & Treiman, R. (2016). So much to read, so little time: How do we read, and can speed reading help?. Psychological Science in the Public Interest, 17(1), 4-34. journals.sagepub.com/doi/pdf/10.1177/1529100615623267The paper that tested serial presentation of words (in other words, no eye movements, just rapid word presentation like popular speed reading apps such as Spritz) is: Acklin, D., & Papesh, M. H. (2017). Modern speed-reading apps do not foster reading comprehension. The American Journal of Psychology, 130(2), 183-199.. researchgate.net/profile/Dina-Acklin/publication/316499956_Modern_Speed-Reading_Apps_Do_Not_Foster_Reading_Comprehension/links/598de0b5a6fdcc1225fca2cc/Modern-Speed-Reading-Apps-Do-Not-Foster-Reading-Comprehension.pdfFor good background reading, especially for seeing how old false claims about speed reading are, check out: Brozo, W. G., & Johns, J. L. (1986). A content and critical analysis of 40 speed reading books. Journal of Reading, 30(3), 242-247.For support of the idea that regressions (which I called backtracking in the video) HELP comprehension, see below. I used their fig 2 (page 4) to show how regressions facilitate reading comprehension. Schotter, E. R., Tran, R., & Rayner, K. (2014). Don’t believe what you read (only once) comprehension is supported by regressions during reading. Psychological science, 25(6), 1218-1226. journals.sagepub.com/doi/pdf/10.1177/0956797614531148For a readable summary of the research on speed reading, check out the introduction to this piece. I used the graph of speed reading vs normal reading in the video as well (fig 3, page 7). Weimer, C. (2021). Effects of Reading Speed and Retrieval Practice on Reading Comprehension. Nysa, the NKU Journal of Student Research; v. 2, Fall 2019. https://dspace.nku.edu/bitstream/handle/11216/3673/OCR%20Weimer_The%20Effects%20of%20Reading%20Speed%20and%20Retrieval%20Practice%20on%20Reading%20Comprehension_Nysa_2_2019.pdf?sequence=1For information on developing reading fluency for non-native speakers, which is a different but related research question, check out this one: Nation, P. (2009). Reading faster. International Journal of English Studies, 9(2). revistas.um.es/ijes/article/download/90791/87591
Active Processing vs Active Learning | Whats the difference? Benjamin Keep, PhD, JD 2023-02-22 | Academic words can often sound the same but have distinct, important meanings. Understanding the difference between active processing and active learning can help you read the studies on learning more effectively0:00 What does active processing mean?0:59 What does active learning mean?2:14 The relationship between active processing and active learningIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my newsletter, Avoiding Folly, here: benjaminkeep.comREFERENCES:On active learning in the classroom, I recommend:Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. doi.org/10.1073/pnas.1319030111Active processing is a big picture idea that crosses many areas of learning, but here is a general paper discussing the role of active processing in remembering:Schwartz, B. L., Son, L. K., Kornell, N., & Finn, B. (n.d.). Four Principles of Memory Improvement: A Guide to Improving Learning Efficiency. https://web.williams.edu/Psychology/Faculty/Kornell/Publications/Schwartz.Son.Kornell.Finn.2011.pdfAnd here is one on the role of active learning in reading: McNamara, D. S. (2010). Strategies to read and learn: Overcoming learning by consumption. Medical Education, 44(4), 340–346. doi.org/10.1111/j.1365-2923.2009.03550.x
What does good learning FEEL like? Benjamin Keep, PhD, JD 2023-02-15 | Learning can be frustrating and exhilarating. Is there an emotion that's associated with optimal learning states?0:00 An introduction0:18 Flow experiences and learning1:37 Happiness and learning2:12 Frustration and learning3:14 Tiredness and learning4:27 A cycle of feelingsIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comHow visualization helps you learn: youtu.be/eeEfyRYlyYQOn effective practice: youtu.be/aIPS4ugcanMREFERENCESFor the classic piece on flow, see: Flow: The Psychology of Optimal Experience by Mihaly Csikszentmihalyi. bookshop.org/p/books/flow-the-psychology-of-optimal-experience-mihaly-csikszentmihalyi/8937185I pulled some quotes on the flow experience from: Jackson, S. A. (1996). Toward a conceptual understanding of the flow experience in elite athletes. Research quarterly for exercise and sport, 67(1), 76-90.I also can’t resist mentioning a method paper. This is the one to read to understand the current state of the research and how researchers have gone about defining flow: Abuhamdeh, S. (2020). Investigating the “flow” experience: Key conceptual and operational issues. Frontiers in psychology, 11, 158. frontiersin.org/articles/10.3389/fpsyg.2020.00158/fullThe link between rest and deliberate practice comes from Ericsson et al.’s first big paper on deliberate practice, cited below at pages 370-71. AFAIK, it hasn’t been an aspect that’s been deeply explored. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological review, 100(3), 363. psycnet.apa.org/doiLanding?doi=10.1037%2F0033-295X.100.3.363The relationship between confusion, frustration, and learning is complex - more complex than I alluded to in the video. Ryan Baker and his colleagues have done a lot of work recently exploring this space - here’s a couple of representative pieces:Richey, J. E., Andres-Bray, J. M. L., Mogessie, M., Scruggs, R., Andres, J. M., Star, J. R., ... & McLaren, B. M. (2019). More confusion and frustration, better learning: The impact of erroneous examples. Computers & Education, 139, 173-190. par.nsf.gov/servlets/purl/10155531 (finding that combined "confusion and frustration" predicted learning)Richey, J. E., McLaren, B. M., Andres-Bray, M., Mogessie, M., Scruggs, R., Baker, R., & Star, J. (2019). Confrustion in Learning from Erroneous Examples: Does Type of Prompted Self-explanation Make a Difference?. In Artificial Intelligence in Education: 20th International Conference, AIED 2019, Chicago, IL, USA, June 25-29, 2019, Proceedings, Part I 20 (pp. 445-457). Springer International Publishing. (finding that combined "frustration and confusion" predicted lower learning).On the role of mistakes in learning, check out: Metcalfe, J. (2017). Learning from errors. Annual review of psychology, 68, 465-489. annualreviews.org/doi/full/10.1146/annurev-psych-010416-044022
I Learned Mental Math In 40 Hours Benjamin Keep, PhD, JD 2023-02-08 | Watch me put research-based learning principles to the test as I learn mental arithmetic. I use testing, lots of relevant, deliberate practice with feedback, spaced practice, interleaving, and review to learn to become as good at mental math as I can in 40 hours.0:00 An introduction0:20 The plan1:08 How I apply the science of learning3:29 My first diagnostic test4:20 Reflections after ten hours5:06 I'm noticing patterns6:23 How to use practice tests to learn8:47 Ten hour mark test9:17 Next stepsIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comHere’s the link to the course I’m using: thegreatcourses.com/courses/secrets-of-mental-mathThe link to the book I mentioned: bookshop.org/a/91541/9780307338402 (I'm a Bookshop.org affiliate, so get a commission if you purchase through this link).REFERENCES:For a readable introduction to interleaving in math from the teacher’s perspective, see: http://pdf.retrievalpractice.org/InterleavingGuide.pdf.For more background on interleaving (like so many of these principles, it get’s complicated), see: Firth, J., Rivers, I., & Boyle, J. (2021). A systematic review of interleaving as a concept learning strategy. Review of Education, 9(2), 642-684. bera-journals.onlinelibrary.wiley.com/doi/pdfdirect/10.1002/rev3.3266For more on spaced practice, see: Latimier, A., Peyre, H., & Ramus, F. (2021). A meta-analytic review of the benefit of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33(3), 959-987. http://www.lscp.net/persons/ramus/docs/EPR20.pdfFor evidence supporting the “keep testing learned items” idea, see this classic: Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf
How Bad Ideas About Learning Spread | 5 Examples Benjamin Keep, PhD, JD 2022-12-14 | A lot of strange ideas circulate among teachers and researchers and parents and students. Why?0:00 The problem1:05 A thought experiment2:43 Ideas without tests and Dale’s Cone of Experience4:09 Learning Styles5:40 Why businesses sell ideas and Baby Einstein7:23 “Science-based” businesses and brain training8:37 Supermemo10:19 What stops good ideas from spreading?11:15 The role of teacher training12:13 Good newsIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comTwo books that I often recommend on learning are: The ABCs of How We Learn by Schwartz, Tsang, and Blair (bookshop.org/a/91541/9780393709261) and Make It Stick by Brown, Roediger, and McDaniel (bookshop.org/a/91541/9780674729018). I'm a Bookshop.org affiliate, so get a small commission if you purchase books through the above links.AcknowledgementsThe green example of Dale’s cone of experience (or learning) comes from commons.wikimedia.org/wiki/File:Edgar_Dale%27s_cone_of_learning.pngThe Baby Einstein clip was from this video: youtube.com/watch?v=tHygdh6fCg4The animated “learning styles” clip comes from: youtube.com/watch?v=_IopcOwfsoUREFERENCESA recent summary of the literature on screen time for young children: Guellai, B., Somogyi, E., Esseily, R., & Chopin, A. (2022). Effects of screen exposure on young children’s cognitive development: A review. Frontiers in Psychology, 4779. frontiersin.org/articles/10.3389/fpsyg.2022.923370/full?On physical activity and development, see: Carson, V., Kuzik, N., Hunter, S., Wiebe, S. A., Spence, J. C., Friedman, A., ... & Hinkley, T. (2015). Systematic review of sedentary behavior and cognitive development in early childhood. Preventive medicine, 78, 115-122. researchgate.net/profile/Alinda-Friedman/publication/280537626_Systematic_Review_of_Sedentary_Behavior_and_Cognitive_Development_in_Early_Childhood/links/5ea46d0292851c1a906f2347/Systematic-Review-of-Sedentary-Behavior-and-Cognitive-Development-in-Early-Childhood.pdfOn brain training, see: Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. (2016). Do “brain-training” programs work?. Psychological Science in the Public Interest, 17(3), 103-186. https://cogsci.northwestern.edu/events/2016-2017-events/simonsEtAl_2016-BrainTraining.pdf.The example I gave of positive (narrow) results was from one of the early papers in the ACTIVE trial: Ball, K., Berch, D. B., Helmers, K. F., Jobe, J. B., Leveck, M. D., Marsiske, M., ... & ACTIVE Study Group. (2002). Effects of cognitive training interventions with older adults: a randomized controlled trial. Jama, 288(18), 2271-2281.Supermemo advocates for an expanding spacing schedule, which isn’t necessarily bad. But the question of whether fixed or expanding intervals are superior is complicated. Perhaps it has something to do with how hard the retrieval is or the number of retrieval attempts. But there’s not a clear winner. See these two pieces for further discussion.Latimier, A., Peyre, H., & Ramus, F. (2021). A meta-analytic review of the benefit of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33(3), 959-987. http://www.lscp.net/persons/ramus/docs/EPR20.pdfCepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological bulletin, 132(3), 354. escholarship.org/content/qt3rr6q10c/qt3rr6q10c.pdf The 2016 teacher training textbook report I mention, is here: Pomerance, L., Greenberg, J., & Walsh, K. (2016). Learning about Learning: What Every New Teacher Needs to Know. National Council on Teacher Quality. files.eric.ed.gov/fulltext/ED570861.pdfThe paper I showed which was a collaboration between researchers and teachers was: Agarwal, P. K., Bain, P. M., & Chamberlain, R. W. (2012). The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist. Educational Psychology Review, 24(3), 437-448.
3 Forgotten Studying Secrets from a 1979 Memory Expert Benjamin Keep, PhD, JD 2022-11-09 | Today, a memory expert reveals his studying secrets. From 40 years ago.0:00 A time machine01:00 Study secret #101:30 Study secret #202:16 Study secret #305:30 What Robert Bjork from 1979 can teach usIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comREFERENCESThis video was based on a Robert Bjork joint:Bjork, R. A. (1979). Information‐processing analysis of college teaching. Educational Psychologist, 14(1), 15-23. researchgate.net/profile/Robert-Bjork-2/publication/261581257_Information-Processing_Analysis_of_College_Teaching/links/5d260916458515c11c23fa01/Information-Processing-Analysis-of-College-Teaching.pdfThe sentence rearrangement study comes from: Bradford, E. J. (1975). Retention of subject-generated verbal material (Doctoral dissertation, ProQuest Information & Learning).For more modern advice from researchers on studying, check out:Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest, 14(1), 4-58.NOTE: I know nothing about immune systems. Just using as a potential example. Please don't take anything I say about them as gospel.ACKNOWLEDGEMENTSImage of Thucydides sculpture from: commons.wikimedia.org/wiki/File:Thucydides_pushkin02.jpgImage of al-Razi sculpture from: commons.wikimedia.org/wiki/File:Persian_Scholar_pavilion_in_Viena_UN_(Rhazes).jpgImage of Louis Pasteur: en.wikipedia.org/wiki/File:Louis_Pasteur,_foto_av_Paul_Nadar,_Crisco_edit.jpgImage of Paul Ehrlich: commons.wikimedia.org/wiki/File:Paul_Ehrlich_1915.jpgThe immune system process image comes from: covalab.com/media/wysiwyg/pictures/pathways/innate_immunity_pathway.jpgThe innate vs adaptive immune system slide comes from: microbenotes.com/differences-between-innate-immunity-and-adaptive-immunity
Learning Beyond Facts | Conceptual Knowledge, Procedural Knowledge, and More Benjamin Keep, PhD, JD 2022-11-03 | After many years in school, it's easy to get the impression that learning is mostly about facts. But there's a whole diverse world of different kinds of knowledge out there. Let's explore it!00:00 Introduction00:17 The fact1:11 The look-up table2:12 Conceptual knowledge - how is a concept different than a fact?3:30 How we learn concepts4:12 The benefits of concepts5:18 Procedural knowledge and skill5:43 The difference between conceptual knowledge and procedural knowledge6:27 Unbalanced knowledge7:30 Bare association8:18 The ability to recognize something8:54 I shouldn't talk about this oneIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comREFERENCESOn the interaction of procedural and conceptual knowledge, see:Rittle-Johnson, B., Schneider, M., & Star, J. R. (2015). Not a one-way street: Bidirectional relations between procedural and conceptual knowledge of mathematics. Educational Psychology Review, 27(4), 587-597. https://www.uni-trier.de/fileadmin/fb1/prof/PSY/PAE/Team/Schneider/Rittle-JohnsonEtAl2015.pdf (Math)Schauble, L. (1996). The development of scientific reasoning in knowledge-rich contexts. Developmental Psychology, 32(1), 102. researchgate.net/profile/Leona-Schauble/publication/232578689_The_development_of_scientific_reasoning_in_knowledge-rich_contexts/links/56eaa8f108ae7858657fde00/The-development-of-scientific-reasoning-in-knowledge-rich-contexts.pdf (Science)For a more complicated, great take on knowledge in problem solving, see:De Jong, T., & Ferguson-Hessler, M. G. (1996). Types and qualities of knowledge. Educational psychologist, 31(2), 105-113. https://ris.utwente.nl/ws/files/6401593/types.pdfFor the difference between varieties of recall and recognition, see:Richardson-Klavehn, A., & Bjork, R. A. (1988). Measures of memory. Annual review of psychology, 39(1), 475-543. researchgate.net/profile/Robert-Bjork-2/publication/234837363_Measures_Of_Memory/links/0deec52efb30acdddd000000/Measures-Of-Memory.pdfACKNOWLEDGEMENTSThe gravity demonstration comes from this video: youtube.com/watch?v=MTY1Kje0yLg.However, there are plenty of good arguments as to why this isn't a terribly helpful analogy: http://newsletter.oapt.ca/files/general-relativiy-analogies.html
The Most Common Obstacle to Effective Studying Benjamin Keep, PhD, JD 2022-10-26 | Effective studying requires effective study strategies and techniques, but it also requires the right conceptualization of what studying is.00:00 Introduction00:54 The big misconception1:54 What makes human memory different from a computer's3:47 Some benefits and drawbacks to the human memory system4:39 The key questionIf you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comHow forgetting works: youtu.be/uv7uhI8nsbsMore effective studying: youtu.be/iW7Fp9Mtq1YBetter note-taking: youtu.be/cRQqH18wJgwFree recall: youtu.be/FatnXnlwAc4Exam prep: youtu.be/GoTYyn3WWxIREFERENCESThe laundry passage comes from this famous 1972 paper, and is often used as a simple example of the power of prior knowledge.Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of verbal learning and verbal behavior, 11(6), 717-726. http://www.ebbinghaus.ru/wp-content/uploads/2010/02/Bransford__Johnson_1972.pdfThe two-different-senses-of-foot demonstration comes from this paper, where the group who were cued to think of two different senses of the to-be-remembered word remembered it better than those who only were cued to think of one sense of that word.Gartman, L. M., & Johnson, N. F. (1972). Massed versus distributed repetition of homographs: A test of the differential-encoding hypothesis. Journal of Verbal Learning and Verbal Behavior, 11(6), 801-808. sciencedirect.com/science/article/abs/pii/S0022537172800161Both of these are also cited in a Robert Bjork paper from 1979. Worth reading. Going to do a full video on this paper soon.Bjork, R. A. (1979). Information‐processing analysis of college teaching. Educational Psychologist, 14(1), 15-23. researchgate.net/profile/Robert-Bjork-2/publication/261581257_Information-Processing_Analysis_of_College_Teaching/links/5d260916458515c11c23fa01/Information-Processing-Analysis-of-College-Teaching.pdf?_sg%5B0%5D=started_experiment_milestone&origin=journalDetailIf you're interested in witness misidentifications, check out www.innocenceproject.org. Eyewitness misidentifications are the leading cause of wrongful convictions, and there are some pretty simple steps to prevent them from happening: innocenceproject.org/eyewitness-identification-reform
What Study Gurus Get Wrong About Learning Benjamin Keep, PhD, JD 2022-10-19 | Study gurus promote active recall and spaced repetition to maximize learning. But is it all its cracked up to be? In the brain, retrieval and encoding processes interact to produce learning. The trick is doing them right.00:00 Introduction01:17 Our brain’s memory systems02:38 Justin’s beef with active recall03:29 Spaced repetition systems, “active recall”, and spaced retrieval practice04:35 Why flashcard systems kind of suck05:12 Justin’s recommendation05:35 What are “desirable difficulties”?07:03 The alternatives to flashcards08:35 A good question to ask yourself when studying10:02 An example study comparing elaborative encoding to retrieval practice11:25 A true statementHere are some of my other videos which are relevant to the discussion:How forgetting works: youtu.be/uv7uhI8nsbsFree recall demo: youtu.be/FatnXnlwAc4Questions about free recall answered: youtu.be/evIuDvQDDSYExample of encoding from reading using organization: youtu.be/WRjsOU6mOp4What researchers knew in 1979: youtu.be/SOgSpCt3H4MAdvice about learning on your own: youtu.be/3zmek5g3XxkOther myths about learning: youtu.be/jcMk3e7u7zMDesirable difficulties: youtu.be/Q7xnl2tyKwY (my first youtube video!)Flashcards: youtu.be/ZIGrHI353noInterleaving: youtu.be/AWTYfzxBwPgSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comTo read more about retrieval practice, check out: benjaminkeep.com/the-wicked-effectiveness-of-retrieval-practiceJustin Sung's video on retrieval: youtube.com/watch?v=--Hu2w0s72YJustin Sung's video on encoding: youtube.com/watch?v=VcT8puLpNKAThe pro baseball footage is from: youtube.com/watch?v=21TmGchVXRAREFERENCESThe article I showed in the video comparing elaborative encoding to retrieval practice is:Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772-775. bit.ly/3S5i9KPBelow, one of the classic pieces on retrieval. It's really an illustration of why you shouldn't just stop trying to remember something just because you have successfully recalled it in the past:Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdfThis is a meta-analysis on the testing effect, which I haven't read in-depth lately, but is worth checking out if you're into this stuff:Rowland, C. A. (2014). The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. Psychological bulletin, 140(6), 1432. bit.ly/3VwGpIIThere's quite a bit of support for the idea that you want to be operating at a "high level" (e.g., synthesizing, applying, critiquing) early in the game. It's not like you need to learn all these rote facts first. A point that Justin makes eloquently in his video on encoding. The piece below is a great discussion of that.Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning?. Journal of Educational Psychology, 111(2), 189. bit.ly/3T5Va3rSome have argued that retrieval is only good for simple materials (like remembering new vocabulary words) and not complex materials (like understanding how Newtonian physics works). Although many of the early studies do focus on "simple" materials, there's plenty of studies that establish the effect of retrieval is just as strong for complex materials.Karpicke, J. D., & Aue, W. R. (2015). The testing effect is alive and well with complex materials. Educational Psychology Review, 27(2), 317-326. link.springer.com/article/10.1007/s10648-015-9309-3
How I got tricked by this one weird trick - eating, the Delbeouf illusion, and meta-analyses Benjamin Keep, PhD, JD 2022-10-12 | Research suggests that using small plates can reduce how much food you eat. This starts with a simple visual illusion, called the Delboeuf illusion. And then it gets more complicated from there.00:00 Introduction00:44 What is the Delboeuf illusion?1:36 How the Delboeuf illusion can make us eat more food.2:48 Was anything I said correct?4:36 Potential explanations for why we eat more when served larger portions.6:09 What this video is actually about.6:49 The meta-analysis strikes back.You can read more about Ben Goldacre here: en.wikipedia.org/wiki/Ben_Goldacre. I recommend his books and articles.Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferences:The original inspiration for this video came from:Van Ittersum, K., & Wansink, B. (2012). Plate size and color suggestibility: The Delboeuf illusion’s bias on serving and eating behavior. Journal of Consumer Research, 39(2), 215-228. academic.oup.com/jcr/article-pdf/39/2/215/5081845/39-2-215.pdfA meta-analysis looking at the small plate ➝ lower intake connection. And finding no consistent relationship.Robinson, E., Nolan, S., Tudur‐Smith, C., Boyland, E. J., Harrold, J. A., Hardman, C. A., & Halford, J. C. (2014). Will smaller plates lead to smaller waists? A systematic review and meta‐analysis of the effect that experimental manipulation of dishware size has on energy consumption. Obesity Reviews, 15(10), 812-821. livrepository.liverpool.ac.uk/3009869/1/Robinson%20et%20al%20Obesity%20Reviews%20AAM.doc And this piece raises questions about the size of the Delboeuf illusion for hungry people.Zitron-Emanuel, N., & Ganel, T. (2018). Food deprivation reduces the susceptibility to size-contrast illusions. Appetite, 128, 138-144. sciencedirect.com/science/article/pii/S0195666318300011How portion size effects caloric intake is still disputed. It seems like there's some anchoring involved (the initial portion suggests what is normal or appropriate) and bite size may play a role as well.Herman, C. P., Polivy, J., Pliner, P., & Vartanian, L. R. (2015). Mechanisms underlying the portion-size effect. Physiology & Behavior, 144, 129-136. http://www2.psy.unsw.edu.au/Users/lvartanian/Publications/Herman,%20Polivy,%20Pliner,%20&%20Vartanian%20(2015).pdfBurger, K. S., Fisher, J. O., & Johnson, S. L. (2011). Mechanisms behind the portion size effect: visibility and bite size. Obesity, 19(3), 546-551. onlinelibrary.wiley.com/doi/pdfdirect/10.1038/oby.2010.233Marchiori, D., Papies, E. K., & Klein, O. (2014). The portion size effect on food intake. An anchoring and adjustment process?. Appetite, 81, 108-115. https://www.academia.edu/download/44357505/The_Portion_Size_Effect_on_Food_Intake_a20160403-22310-r0punj.pdfFinally, this is my favorite meta-analysis, which DOES, in fact, suggest that smaller plates lead to smaller portion sizes and reduced consumption. It's still a "may" though. The effect seems quite vulnerable to user awareness. So if participants are aware that they're in a food study, researchers don't find an effect. Or if they're aware that they just changed their plates to reduce their consumption, you don't see an effect. But there is at least some suggestion that in the long-run, as your awareness of the changed plates fades, you do end up reducing your consumption.Holden, S. S., Zlatevska, N., & Dubelaar, C. (2016). Whether smaller plates reduce consumption depends on who’s serving and who’s looking: a meta-analysis. Journal of the Association for Consumer Research, 1(1), 134-146. https://opus.lib.uts.edu.au/bitstream/10453/41448/1/2015003.email.pdf
7 Tips For Learning Anything On Your Own Benjamin Keep, PhD, JD 2022-10-05 | Learning things on your own is hard. Maybe these tips will help.00:00 Introduction00:42 How to make specific learning goals2:03 What are the key skills?3:21 Gathering learning resources4:02 A note on scheduling4:48 Making the best use of feedback8:12 Watch out for time-sensitive learning goals10:15 When to make a changeSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comREFERENCESOn creating meaningful learning goals, see:cwsei.ubc.ca/resources/instructor/learninggoals, from the Carl Weiman Science InitiativeTo read more about metacognition and illusions of knowing, see:Avhustiuk, M. M., Pasichnyk, I. D., & Kalamazh, R. V. (2018). The illusion of knowing in metacognitive monitoring: Effects of the type of information and of personal, cognitive, metacognitive, and individual psychological characteristics. Europe's Journal of Psychology, 14(2), 317. ncbi.nlm.nih.gov/pmc/articles/PMC6016031On spacing, see my Medium article and the references linked therein: bit.ly/3SBcD3vTo read more about Goodhart's Law and other ways that metrics can backfire, I highly recommend:Muller, J. Z. (2019). The Tyranny of Metrics. Princeton University Press.CREDITSMy blog post example was www.betterexplained.com - a really good website discussing the intuitions behind many math concepts.The Youtube channel example was Mathematical Visual Proofs: @MathVisualProofs The textbook example was "University Physics" - widely regarded as one of the best basic physics books out there. Young, H. D., Freedman, R. A., & Ford, A. L. (2016) Sears and Zemansky's university physics. Pearson education. 14th Edition. There's more recent editions as well.StackExchange is one of the great places to get answers to questions by experts - especially in programming, engineering, and mathematics. The forum post I showed was from CrossValidated - their statistics arm. stackexchange.com/sites#The Magic player was Kenji "NumotTheNummy" Egashira, his channel is here: @NumotTheNummyYT Steph Curry highlights were from: youtube.com/watch?v=7fPcse1phtk.
The true story of the search for planet Vulcan. And what it says about science. Benjamin Keep, PhD, JD 2022-09-29 | The forgotten story of a mystery planet. With some opinions about what the story implies about the practice of science.00:00 Introduction00:46 Problem Number One: Uranus1:50 Problem Number Two: Mercury3:37 Something something provisional nature4:23 Naive falsification (or disconfirmation)6:03 Theory-ladenness of factsWhat a great story, huh? You can find the basic story on Wikipedia.But you might want to check out Thomas Levenson's book "The Hunt for Vulcan: ...And how Albert Einstein Destroyed a Planet, Discovered Relativity, and Deciphered the Universe" available here: bookshop.org/a/91541/9780812988307. I am a Bookshop.org affiliate and get a small commission if you purchase through the link.It's not really the best example of the theory-ladenness of facts, but I feel like it's a good introduction to questions about how science works and how we know what we know.Thanks for watching.Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
The Surprising Truth About Note-taking During Lectures Benjamin Keep, PhD, JD 2022-09-21 | My argument on why the conventional wisdom on note-taking - particularly during lectures - is wrong.00:00 Introduction1:25 Does the writing process help us remember what we heard?1:44 Does reviewing notes later help us remember?2:07 The missing piece.2:55 An alternative method.3:56 Is taking verbatim notes useful?5:02 On paying attention to the right things.5:46 Taking notes during vs after a lecture6:46 My recommended practiceRyhan Hussain's channel: youtube.com/channel/UCg2u5o5dvS5iIu74rRO5x1wSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferences:A lot of the recent research on note-taking has been on whether taking hand-written notes or laptop notes are superior. The evidence for one being better than the other is a bit "meh". As mentioned, the idea that "verbatim notes are bad" seems less true for laptop note-takers.The opening quote comes from:Flanigan, A. E., & Titsworth, S. (2020). The impact of digital distraction on lecture note taking and student learning. Instructional Science, 48(5), 495-524. bit.ly/3RiWQWEThe second quote comes from:Morehead, K., Dunlosky, J., & Rawson, K. A. (2019). How much mightier is the pen than the keyboard for note-taking? A replication and extension of Mueller and Oppenheimer (2014). Educational Psychology Review, 31(3), 753-780. bit.ly/3CYdgzf Experiment 2, here, is also the one that I relate where "no note-taking" performed just as well as any of the note-taking conditions on immediate tests. It's unclear to me why they didn't also use a delayed measure with the no note-taking group. Also, looking at tables 3 and 4, you can see that if students took notes on material germane to the test, then they did well (my second concluding point).Another recent piece is here:Luo, L., Kiewra, K. A., Flanigan, A. E., & Peteranetz, M. S. (2018). Laptop versus longhand note taking: effects on lecture notes and achievement. Instructional Science, 46(6), 947-971. bit.ly/3Qw24xb. The introduction is a fair summary of the mixed evidence on the encoding hypothesis. Unfortunately, they only used immediate tests here, but there's some interesting material on the relative benefits of hand-written vs laptop notes.Below is a more direct test of my suggested approach. The results? Note-taking beats testing on immediate tests, but in two weeks testing wins. Testing even beats note-taking + review, suggesting that the storage function of notes is not particularly strong.Rummer, R., Schweppe, J., Gerst, K., & Wagner, S. (2017). Is testing a more effective learning strategy than note-taking?. Journal of Experimental Psychology: Applied, 23(3), 293. psycnet.apa.org/doiLanding?doi=10.1037%2Fxap0000134And here is another that finds no difference, even though students who took notes during the lecture took more relevant notes than those who took notes after the lecture.Haynes, J. M., McCarley, N. G., & Williams, J. L. (2015). An analysis of notes taken during and after a lecture presentation. North American Journal of Psychology, 17(1). bit.ly/3CUqAELAnd another old piece, same thing, no difference (only the measures weren't great).Eisner, S., & Rohde, K. (1959). Note taking during or after the lecture. Journal of Educational Psychology, 50(6), 301–304. psycnet.apa.org/record/2005-10188-010Certainly students seem to believe that note-taking during lectures improves learning outcomes: Morehead, K., Dunlosky, J., Rawson, K. A., Blasiman, R., & Hollis, R. B. (2019). Note-taking habits of 21st century college students: implications for student learning, memory, and achievement. Memory, 27(6), 807-819. bit.ly/3cKGVBgOn the superiority of free recall / retrieval practice / testing, over re-study, see:Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on psychological science, 1(3), 181-210. https://linguistics.ucla.edu/people/hayes/Teaching/papers/2006_Roediger_Karpicke_Review.pdfKarpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdfKarpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of memory and language, 57(2), 151-162. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdf
Three simple tricks to read textbooks more effectively Benjamin Keep, PhD, JD 2022-09-14 | Textbooks are a slog. But part of the problem is that we read them like we read other things, and we shouldn't. Here's how to up your textbook reading game.00:00 Introduction00:38 Sipping and chugging1:20 A technique for reading more actively2:43 A technique for incorporating free recall practiceSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comThe textbook pages are from "University Physics with Modern Physics" by Young and Freedman. If you're interested in learning physics on your own, check out: susanrigetti.com/physicsReferences:My recommendations here are a variety of what's called "spaced retrieval practice". There is a huge literature that spacing out study sessions (e.g., "sipping") provides long-term learning benefits. Here's a meta-analysis on verbal recall tasks: Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological bulletin, 132(3), 354. escholarship.org/content/qt3rr6q10c/qt3rr6q10c.pdfAnd here's a piece using realistic classroom controls. Seabrook, R., Brown, G. D., & Solity, J. E. (2005). Distributed and massed practice: From laboratory to classroom. Applied cognitive psychology, 19(1), 107-122. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.374.3437&rep=rep1&type=pdfFor my intro to this phenomenon, see: bit.ly/3BKHhl2.My recommendation to do free recall practice prior to coming back to the textbook again comes from evidence on the efficacy of free recall in general (that's the "retrieval"). Here's some good papers on it:Karpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of memory and language, 57(2), 151-162. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdfKarpicke, J. D., & Aue, W. R. (2015). The testing effect is alive and well with complex materials. Educational Psychology Review, 27(2), 317-326. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.6502&rep=rep1&type=pdfFor my intro to retrieval practice, see: bit.ly/3djdOFv.My recommendation on reading the question before seeing the answer (e.g., "active" reading) is based on work in prior knowledge and illusions of knowing.Example problems tell you things, and there is "a time for telling". Seeing the canonical answer before first think about what the problem is short-circuits the learning process. Here's a few papers on this idea:Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and instruction, 16(4), 475-5223. http://aaalab.stanford.edu/papers/time_for_telling.pdfSchwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of educational psychology, 103(4), 759. http://ece.neu.edu/edsnu/mcgruer/USC/PracticingVersusInventing2011-edu-103-4-759.pdfThere's also interesting work exploring the opposite order. See the paper below for a really good discussion:Fyfe, E. R., DeCaro, M. S., & Rittle‐Johnson, B. (2014). An alternative time for telling: When conceptual instruction prior to problem solving improves mathematical knowledge. British journal of educational psychology, 84(3), 502-519. https://www.academia.edu/download/39309388/2014_FyfeDeCaroRittleJohnson_BJEP.pdf
How visualizations help you learn (and how to use them) Benjamin Keep, PhD, JD 2022-09-07 | Do you create visualizations while you learn? If not, you should. Here's why.00:00 Introduction00:31 Three advantages to visualizations2:15 What should you visualize?2:59 The cardinal sinSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comImage acknowledgements:The initial three visualizations come from the Dataisbeautiful subreddit. Here are the credits:Long COVID in the U.S., by MetricT: reddit.com/r/dataisbeautiful/comments/x4hqfs/oc_percent_of_adult_population_currentlyBrazil's GDP, by Latinometrics: reddit.com/r/dataisbeautiful/comments/x441dy/oc_in_2011_brazils_gdp_per_capita_was_27k_aboveMandela Effect, by YouGov_Official: reddit.com/r/dataisbeautiful/comments/x3z9ad/oc_americans_are_more_likely_to_hold_false_ratherJacque Minard's famous illustration of Napoleon's campaign grabbed from http://activehistory.ca/wp-content/uploads/2012/02/Minard.png. Thanks, Active History!Water cycle image: John Evans and Howard Periman, USGS, Public domain, via Wikimedia Commons. upload.wikimedia.org/wikipedia/commons/1/19/Watercyclesummary.jpg.John Snow's map of the broad street water pump cholera outbreak (originally from 1854). en.wikipedia.org/wiki/John_Snow#/media/File:Snow-cholera-map-1.jpg. Causal ecosystem image, from one of Tina Grotzer's papers. Grotzer, T. A., Kamarainen, A. M., Tutwiler, M. S., Metcalf, S., & Dede, C. (2013). Learning to reason about ecosystems dynamics over time: The challenges of an event-based causal focus. BioScience, 63(4), 288-296. academic.oup.com/bioscience/article/63/4/288/253365. She's an academic who does great work on how students think of causal relationships.The "argument" image comes from a piece by Deanna Kuhn, a pioneer in studying argumentation. Kuhn, D., Goh, W., Iordanou, K., & Shaenfield, D. (2008). Arguing on the computer: A microgenetic study of developing argument skills in a computer‐supported environment. Child development, 79(5), 1310-1328. educationforthinking.org/sites/default/files/pdf/09%20Arguing%20on%20the%20computer.pdfReferences:This video was largely inspired by the chapter "V is for Visualization" in the ABCs of How We Learn by Schwartz, Tsang, and Blair. Definitely recommend it.
Why THIS is the best way to learn to ride a bike. | Intro to instructional scaffolding Benjamin Keep, PhD, JD 2022-08-31 | Why are balance bikes effect for teaching kids how to bike ride? It is a perfect example of instructional scaffolding: a modification that helps students meaningfully participate in a target skill.00:00 Introduction00:38 What is instructional scaffolding?01:28 A second advantage of balance bikes.01:52 Figuring out the right scaffolding.This is the video from the Global Cycling Network: youtu.be/UOg1hHmGIQQSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferenceThis is a great, thorough piece on scaffolding, that gives a more precise definition and discusses many (many!) issues that I didn't touch on here, but hope to in future videos:Belland, B. R. (2017). Instructional scaffolding: foundations and evolving definition. In Instructional scaffolding in STEM education (pp. 17-53). Springer, Cham.link.springer.com/chapter/10.1007/978-3-319-02565-0_2
What makes people great chess players? Cognitive scientist answers. Benjamin Keep, PhD, JD 2022-08-11 | What creates a great chess player - their IQ, how they train, their personalities, their motivation? Here's my read of the existing research literature.00:00 Introduction00:41 Will having a high IQ make you good at chess?03:12 Does playing chess improve your IQ?4:15 The influence of training5:41 The influence of personality and motivation6:51 The cultural contextSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferences:The meta-analysis of IQ/cognitive ability with chess skill is here:Burgoyne, A. P., Sala, G., Gobet, F., Macnamara, B. N., Campitelli, G., & Hambrick, D. Z. (2016). The relationship between cognitive ability and chess skill: A comprehensive meta-analysis. Intelligence, 59, 72-83. eprints.lse.ac.uk/102241/1/INTELL_2016_117_The_Relationship_between_Cognitive_Ability_and_Chess_Skill_a_Comprehensive_Meta_Analysis.pdfThe meta-analysis on whether chess playing increases IQ is here:Sala, G., & Gobet, F. (2017). Does far transfer exist? Negative evidence from chess, music, and working memory training. Current directions in psychological science, 26(6), 515-520. journals.sagepub.com/doi/full/10.1177/0963721417712760On the occasional reversal of the relationship between IQ and chess skill, see:Bilalić, M., McLeod, P., & Gobet, F. (2007). Does chess need intelligence?—A study with young chess players. Intelligence, 35(5), 457-470. bura.brunel.ac.uk/bitstream/2438/642/1/Does%20Chess%20Need%20Intelligence-revision-finalINT.pdfThe meta-analysis on training and chess skill is here:Charness, N., Tuffiash, M., Krampe, R., Reingold, E., & Vasyukova, E. (2005). The role of deliberate practice in chess expertise. Applied Cognitive Psychology, 19(2), 151-165. https://clinica.ispa.pt/ficheiros/areas_utilizador/user11/11_-_the_role_of_dp_in_chess_expertise.pdfOn chess and personality, check out: Vollstädt-Klein, S., Grimm, O., Kirsch, P., & Bilalić, M. (2010). Personality of elite male and female chess players and its relation to chess skill. Learning and Individual Differences, 20(5), 517-521. researchgate.net/profile/Merim-Bilalic/publication/220027647_Personality_of_Elite_Male_and_Female_Chess_Players_and_its_Relation_to_Chess_Skill/links/5d49516492851cd046a5a25b/Personality-of-Elite-Male-and-Female-Chess-Players-and-its-Relation-to-Chess-Skill.pdfThe study linking chess skill, deliberate practice, and motivation is here:De Bruin, A. B., Rikers, R. M. J. P., & Schmidt, H. G. (2007). The influence of achievement motivation and chess-specific motivation on deliberate practice. Journal of Sport and Exercise Psychology, 29(5), 561. researchgate.net/profile/Anique-De-Bruin/publication/271075262_The_Influence_of_Achievement_Motivation_and_Chess-Specific_Motivation_on_Deliberate_Practice/links/00b7d539064ca2cf43000000/The-Influence-of-Achievement-Motivation-and-Chess-Specific-Motivation-on-Deliberate-Practice.pdfOn the "domain-specificity" of expertise, the best resource is probably:Ericsson, K. A., Hoffman, R. R., Kozbelt, A., & Williams, A. M. (Eds.). (2018). The Cambridge handbook of expertise and expert performance. Cambridge University Press.But that's the best resource for a lot of the expertise literature! If you can't get your hands on it, here's an interesting article talking about how creativity (something that many people assume is domain-general) is actually quite domain-specific:Baer, J. (2015). The importance of domain-specific expertise in creativity. Roeper Review, 37(3), 165-178. https://www.academia.edu/download/51367773/The_Importance_of_Domain_Specific_Expertise_in_Creativity.pdf
3 Simple Lessons in Learning and Attention | Cognitive Load, External Focus, Distraction, etc. Benjamin Keep, PhD, JD 2022-07-27 | What does attention have to do with learning? A lot. Here's a very short summary of some of the more interesting findings in the research on attention and learning in three simple lessons.00:00 Lesson one - four stories about attention2:21 One way of thinking about attention and learning3:07 Lesson two - what controls attention?4:06 Lesson three - the roles of students and teachersSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comThe artwork I mentioned in the example comes from:Cut with the Kitchen Knife Dada Through the Last Weimar Beer-Belly Cultural Epoch of Germany by Hannah Höch (1919)The Elephant Celebes by Max Ernst (1921)Das Undbild by Kurt Schwitters (1919)The complex scientific figure is kind of a cheat: no one would use that as a powerpoint slide. It's more of an attempt to make a complete map of metabolic pathways in human cells. I found it here: https://faculty.cc.gatech.edu/~turk/bio_sim/articles/metabolic_pathways.pngReferences:The "someone else with a laptop distracts you" study is here:Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24-31.https://www.sciencedirect.com/science/article/pii/S0360131512002254A good review of internal/external focus in motor learning is here:Wulf, G. (2013). Attentional focus and motor learning: a review of 15 years. International Review of sport and Exercise psychology, 6(1), 77-104. http://gwulf.faculty.unlv.edu/wp-content/uploads/2018/11/Wulf_AF_review_2013.pdfSeductive details is a large research topic, but here's a recent meta-analysis that tries to synthesize the existing studies:Sundararajan, N., & Adesope, O. (2020). Keep it coherent: A meta-analysis of the seductive details effect. Educational Psychology Review, 32(3), 707-734. link.springer.com/article/10.1007/s10648-020-09522-4The "thinking 'are these the same or different'" example I drew from the literature on blocked and interleaved practice, which suggests that paying attention to similarities and paying attention to differences offers different affordances in category learning. See pages 2-3 in the following piece.Carvalho, P. F., & Goldstone, R. L. (2014). Putting category learning in order: Category structure and temporal arrangement affect the benefit of interleaved over blocked study. Memory & cognition, 42(3), 481-495. https://pcl.sitehost.iu.edu/papers/blockedinterleaved.pdfGood advice on reducing extraneous cognitive load in undergraduate science courses comes is here (with a couple of other good references in there):cwsei.ubc.ca/sites/default/files/cwsei/resources/literature/Cognitive-Load_CWSEI.pdfSome interesting research on cell phones and attention is here:Thornton, B., Faires, A., Robbins, M., & Rollins, E. (2014). The mere presence of a cell phone may be distracting: Implications for attention and task performance. Social Psychology, 45(6), 479. oarklibrary.com/file/2/0c907a59-76b9-482a-b4a3-1c66294d7048/8d49e16c-14dc-4877-9c64-e75ba9bfdcc2.pdfOn manipulating student attention towards (or away from) deep structure, see (this is also where the clown image comes from):Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of educational psychology, 103(4), 759. http://ece.neu.edu/edsnu/mcgruer/USC/PracticingVersusInventing2011-edu-103-4-759.pdf
How to prepare for a final exam | High school and college Benjamin Keep, PhD, JD 2022-07-13 | Oh crap. You’ve got a test. How are you going to study for it?00:00 Step #11:31 Step #23:00 Step #35:04 Tip #1: How to use visualizations5:40 Tip #2: On study groups6:26 Tip #3: Be reasonable during step 37:03 Open vs. closed book exams7:36 Multiple choice vs essay testsSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comMy advice here is based on a few lines of literature.Steps 1 and 3, which I recommend spending the most time on, comes from the research on free recall and the testing effect. These are just better study than rereading or re-writing your notes. Typical research findings are here:Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772-775. mrbartonmaths.com/resourcesnew/8.%20Research/Memory%20and%20Revision/Retrieval%20practice%20more%20effective%20than%20studying.pdfKarpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968.http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdfKarpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57(2), 151-162.https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdfYang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin. (currently available at gwern.net/docs/spacedrepetition/2021-yang.pdf)An applied research piece exploring the application of tests in the classroom. They tested both short-answer and multiple-choice tests, finding that short answer tests outperformed multiple choice tests (though both outperformed additional reading). McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European journal of cognitive psychology, 19(4-5), 494-513. (currently available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1061.366&rep=rep1&type=pdf)A classic study demonstrating long-term benefits (1 month).Butler, A. C., & Roediger III, H. L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19(4-5), 514-527. (currently available at http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Butler-Roediger-2007_EJCP.pdf)My advice on the "outstanding questions" box is based on the literature on self-questioning and interrogation. It’s also supposed to help you with metacognitive knowledge (which helps you study more effectively because you know what you already understand and what you’re still struggling with). Example research findings are here:Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest, 14(1), 4-58. https://bibliotecadigital.mineduc.cl/bitstream/handle/20.500.12365/17388/dunloskyimprovinglearning.pdf?sequence=1 (The first two sections on elaborative interrogation and self-explanations are relevant here.)Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of educational research, 66(2), 181-221. jgregorymcverry.com/readings/rosenshine1996StudentQuestioning.pdf (In most of these studies people are teaching students how to ask questions, which is an important part of the process.)King, A. (1992). Comparison of self-questioning, summarizing, and notetaking-review as strategies for learning from lectures. American Educational Research Journal, 29(2), 303-323. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.859.5957&rep=rep1&type=pdfMy advice on using visualizations comes from research on learning and visualizations (well summarized in The ABCs of Learning, citation below). Basically, they offload cognitive burden, they can be more specific than words (relating ideas and concepts in more precise ways), and they can help you identify patterns that you might not have recognized before.Schwartz, D. L., Tsang, J. M., & Blair, K. P. (2016). The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them. WW Norton & Company. (I’m referring to “V is for Visualization”. Great introduction to the research on learning more generally.)There are other ways of studying for tests, of course. But I think this is a pretty solid default method.
What is intuition? And where does it come from? Benjamin Keep, PhD, JD 2022-07-06 | We all experience intuitions. And they can seem mysterious and wonderful. The psychological research suggests intuitions are the outcome of a fundamental feature of our minds: two parallel learning systems.00:00 Introduction00:28 Two learning systems01:02 One source of intuition01:42 Another source of intuition02:46 Can you trust your intuition? Should you?Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
The thing that kills creativity | Psychology, insight, and problem solving. Benjamin Keep, PhD, JD 2022-06-29 | What makes us creative problem solvers? The answer takes more than one video. But here are the results of some studies exploring how people solve physics problems: do they see the creative shortcut? Or are they stuck working out the problem in the standard way?00:00 Introduction00:26 What are force diagrams?00:48 The study and its results01:39 Explanation one01:57 Explanation two02:42 Why draw a diagram?Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferences:Kuo, E., Hallinen, N. R., & Conlin, L. D. (2017). When procedures discourage insight: epistemological consequences of prompting novice physics students to construct force diagrams. International Journal of Science Education, 39(7), 814-839. tandfonline.com/doi/abs/10.1080/09500693.2017.1308037Heckler, A. F. (2010). Some consequences of prompting novice physics students to construct force diagrams. International Journal of Science Education, 32(14), 1829-1851.researchgate.net/profile/Andrew-Heckler/publication/233366500_Some_Consequences_of_Prompting_Novice_Physics_Students_to_Construct_Force_Diagrams/links/0deec53220b7f7ae2c000000/Some-Consequences-of-Prompting-Novice-Physics-Students-to-Construct-Force-Diagrams.pdfKuo, E., Hallinen, N. R., & Conlin, L. D. (2015, December). How prompting force diagrams discourages student use of adaptive problem-solving shortcuts. In The Physics Education Research Conference 2015.researchgate.net/profile/Eric-Kuo/publication/299701476_How_prompting_force_diagrams_discourages_student_use_of_adaptive_problem-solving_shortcuts/links/57e3f25d08ae8d5908c162d1/How-prompting-force-diagrams-discourages-student-use-of-adaptive-problem-solving-shortcuts.pdf
Why are chess masters so much better than you? Benjamin Keep, PhD, JD 2022-06-22 | What makes great chess players so great? What's going on in their heads? Do they just have great memories? How far do they look ahead?00:00 Introduction00:24 The early experiments1:46 A brain superpower3:12 Depth of search4:33 Human experts vs AI chess programsSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferences:My summarization of the memory results is really recounting this recent study:Smith, E. T., Bartlett, J. C., Krawczyk, D. C., & Basak, C. (2021). Are the advantages of chess expertise on visuo-spatial working-memory capacity domain specific or domain general?. Memory & Cognition, 49(8), 1600-1616.https://www.researchgate.net/profile/Chandramallika-Basak-2/publication/351245327_Are_the_Advantages_of_Chess_Expertise_on_Visuo-Spatial_Working_Memory_Capacity_Domain_Specific_or_Domain_General/links/609036d0a6fdccaebd073806/Are-the-Advantages-of-Chess-Expertise-on-Visuo-Spatial-Working-Memory-Capacity-Domain-Specific-or-Domain-General.pdfBut there's lots of other research along the same lines.The earliest research was here:de Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton Publishers.Chase, W. G., & Simon, H. A. (1973). The mind's eye in chess. In Visual information processing (pp. 215-281). Academic Press. sciencedirect.com/science/article/pii/B9780121701505500111And here are some other solid articles I looked at:Charness, N. (1992). The impact of chess research on cognitive science. Psychological research, 54(1), 4-9. link.springer.com/article/10.1007/BF01359217Bilalić, M., McLeod, P., & Gobet, F. (2008). Expert and “novice” problem solving strategies in chess: Sixty years of citing de Groot (1946). Thinking & Reasoning, 14(4), 395-408. researchgate.net/profile/Fernand-Gobet/publication/49401082_Expert_and_novice_problem_solving_strategies_in_chess_Sixty_years_of_citing_de_Groot_1946/links/57651cbb08ae1658e2f46b20/Expert-and-novice-problem-solving-strategies-in-chess-Sixty-years-of-citing-de-Groot-1946.pdf?_sg%5B0%5D=started_experiment_milestone&origin=journalDetailConnors, M. H., Burns, B. D., & Campitelli, G. (2011). Expertise in complex decision making: The role of search in chess 70 years after de Groot. Cognitive science, 35(8), 1567-1579. onlinelibrary.wiley.com/doi/full/10.1111/j.1551-6709.2011.01196.x
What Sherlock Holmes Got Wrong | Deduction, Induction, and Abduction Benjamin Keep, PhD, JD 2022-06-15 | Sherlock Holmes is famous for his deductions. But what if they're not deductions at all?00:00 Introduction00:24 Deduction00:57 Do deductions move from the general to the particular?01:39 The key characteristic of deductions02:21 Induction02:45 Do inductions move from the particular to the general?03:21 The key characteristic of inductions04:26 A special kind of induction05:35 Semantics?Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReference:The best discussion of the distinction between the three kinds of reasoning I outlined here comes from "How We Reason" by Philip Johnson-Laird - cognitive scientist extraordinaire. Chapter 13 is on Sherlock and abductions. It's his model that I'm describing.The clips are from the BBC series "Sherlock," starring Benedict Cumberbatch. Used here for educational purposes.
What no one tells you about learning faster. Benjamin Keep, PhD, JD 2022-06-08 | Should you try to 10X your learning? Probably not. There are lots of cases when prioritizing learning speed early on in the learning process backfires, ultimately resulting in poorer learning in the long run. Here are three.00:00 Introduction00:21 Three stories01:32 The catch01:56 The beginnings of an explanationSign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comReferences:I initially came across the "mirror learning" example on page 310 of this book:Magill, R., & Anderson, D. (2010). Motor learning and control. New York: McGraw-Hill Publishing. https://www.academia.edu/download/62953224/Motor_Learning_and_Control__Concepts_and_-_Anderson__David20200414-28877-18hp4j2.pdfThis reference has a fuller description of the idea (though is testing a slightly different hypothesis):Tremblay, L., & Proteau, L. (1998). Specificity of practice: The case of powerlifting. Research quarterly for exercise and sport, 69(3), 284-289. researchgate.net/profile/Luc-Tremblay-2/publication/13507905_Specificity_of_Practice_The_Case_of_Powerlifting/links/5805b11608aef179365e73b4/Specificity-of-Practice-The-Case-of-Powerlifting.pdfNote that learning with a mirror doesn't always impair performance without a mirror (or improve skill acquisition during practice). (See for instance: Lynch, J. A., Chalmers, G. R., Knutzen, K. M., & Martin, L. T. (2009). Effect on performance of learning a Pilates skill with or without a mirror. Journal of Bodywork and Movement Therapies, 13(3), 283-290. sciencedirect.com/science/article/abs/pii/S1360859208001666)The math feedback example comes from this study:Fyfe, E. R., & Rittle-Johnson, B. (2017). Mathematics practice without feedback: A desirable difficulty in a classroom setting. Instructional Science, 45(2), 177-194. link.springer.com/article/10.1007/s11251-016-9401-1The Spanish word study is at:Bahrick, H. P. (1979). Maintenance of knowledge: Questions about memory we forgot to ask. Journal of Experimental Psychology: General, 108(3), 296. psycnet.apa.org/record/1981-00448-001But I'm basing my discussion on the description of the study as made by Robert Bjork, here:Bjork, R. A. (1994). Memory and metamemory considerations in the. Metacognition: Knowing about knowing, 185(7.2). It's really such a great article; one that I've revisited many times. Get in touch if you want a pdf copy.
A Simple Study Tip: Stop Highlighting Benjamin Keep, PhD, JD 2022-05-25 | Highlighting your textbook is how most students know they're being diligent. And there is no shortage of advice on how to highlight. But is it really worth the trouble? Nah. Here's some thoughts on why.00:00 Introduction00:35 Do we know how to highlight?01:39 What does it encourage us to do?02:08 What does it tell us about what we know?Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.comThe onscreen quotes were pulled from a classic review of study techniques:Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public interest, 14(1), 4-58. https://bibliotecadigital.mineduc.cl/bitstream/handle/20.500.12365/17388/dunloskyimprovinglearning.pdf?sequence=1
5 Things Students Should Know About Learning Benjamin Keep, PhD, JD 2022-05-18 | We can spend a lot of our lives as students with misperceptions about learning. Here are some things to keep in mind throughout your career as a student.00:00 Who are you working for?00:50 Effort over the long run.01:12 How should we think about mistakes and confusion?02:13 Spending your time more effectively.03:12 The pretty principle.Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
