Benjamin Keep, PhD, JD
What Gears Teach Us About Thinking
updated
GiveInternet.org Donation Link: giveinternet.org/BenjaminKeep
00:00 Introduction
00:45 Finding the right learning resources
2:04 Getting practice
4:02 Finding a learning community
5:08 Planning your practice
5:54 What happens next?
8:18 A worthy cause
For updates on my course and website, click here: https://forms.gle/9TEUw7bj3YTs7B4H6
To sign up to my (infrequent) newsletter, click here: http://www.benjaminkeep.com.
Edited by Presage: pre-sage.com
Resources featured in the video:
Kalid Azad's Better Explained: betterexplained.com
Paul's Online Math Notes: https://tutorial.math.lamar.edu/
Susan Rigetti's guide to learning physics: susanrigetti.com/physics
Andy Guitar: andyguitar.co.uk
SimplyPiano: hellosimply.com/simply-piano
Tenenbaum, M., & Pollard, H. (1985). Ordinary differential equations: an elementary textbook for students of mathematics, engineering, and the sciences. Courier Corporation. archive.org/details/TenenbaumAndPollardOrdinaryDifferentialEquations
Young, H. D., Freedman, R. A., Ford, A. L., Sears, F. W., & Zemansky, M. W. (2014). Sears and Zemansky's University Physics with Modern Physics.
00:00 Introduction
00:19 The CRT problems
1:20 The structure of CRT problems
3:04 Some hints about what's going wrong
4:53 The structure of insight problems
6:27 Explanation Number One
7:57 Explanation Number Two
8:57 Explanation Number Three
I'm launching a membership site. If you want in, go here: https://forms.gle/JoMA1Xa5Nnnwm8VU9
Recommended reading: benjaminkeep.com/recommended-reading
Sign up to my free (infrequently published) newsletter here: benjaminkeep.com
REFERENCES
This video is mostly based off of Patel, N., Baker, S. G., & Scherer, L. D. (2019). Evaluating the cognitive reflection test as a measure of intuition/reflection, numeracy, and insight problem solving, and the implications for understanding real-world judgments and beliefs. Journal of Experimental Psychology: General, 148(12), 2129–2153. doi.org/10.1037/xge0000592
This is the (classic) paper that introduced the CRT as a measure of reflection. Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives, 19, 25– 42. http://dx.doi.org/10.1257 089533005775196732. It also includes correlations between the math section of the SAT and the CRT (.46) vs the verbal section of the SAT and the CRT (.24).
On the CRT’s correlation to tests measuring numeracy being about .5, see Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring Risk Literacy: The Berlin Numeracy Test. Judgment and Decision Making, 7(1), 25–47. doi.org/10.1017/S1930297500001819
On the associations between reflection/analytical thinking and various beliefs, see Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015). Everyday Consequences of Analytic Thinking. Current Directions in Psychological Science, 24(6), 425-432. doi.org/10.1177/0963721415604610
This mentions some of the earlier results correlating CRT performance to rational thinking, but also extends the CRT to a seven-question test. The four added questions are probably even more reliant on numerical ability than the first three. Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & cognition, 39(7), 1275-1289.
00:00 Introduction
1:08 The problem solving process
1:54 The important sentence
3:02 Skipping the step vs doing the thing
BTW, if you're learning physics on your own, I think Susan Rigetti's recommendations are indispensable. susanrigetti.com/physics
The "2000" problems video I reference is here: youtu.be/_5MQUMtRPmM
References:
The physics book in question is: Young, H. D., Freedman, R. A., & Ford, A. L. (20). Sears and Zemansky's University Physics with Modern Physics. 15th Edition. Pearson education. Although I think any of the earlier editions (14th, 13th, international, etc.) are good. Again, credit to Susan Rigetti for this recommendation.
The papers I referenced on conceptual knowledge and procedural skill:
Kim, E., & Pak, S. J. (2002). Students do not overcome conceptual difficulties after solving 1000 traditional problems. American Journal of Physics, 70(7), 759-765. (available at http://sdsu-physics.org/sdsu_per/articles/ProblemS_ConUnderst.pdf)
Byun, T., & Lee, G. (2014). Why students still can't solve physics problems after solving over 2000 problems. American Journal of Physics, 82(9), 906-913. (available at aapt.scitation.org/doi/abs/10.1119/1.4881606)
00:00 Introduction
0:29 Cognitive load theory
3:17 Retrieval practice
4:17 Skill analysis
6:12 Deliberate practice
8:25 Leveraging prior knowledge
Edited by Presage: https://pre-sage.com.
Sign up to my newsletter here: http://www.benjaminkeep.com.
00:00 Introduction
00:46 What can we encode?
1:46 Two issues in encoding research
4:43 Depth of processing
9:17 The generation effect
13:31 Imagery
17:28 Memory palaces and “time palaces”
20:10 The drawing effect
23:36 Distinctiveness
27:50 Value
31:42 Integration
37:17 Transfer appropriate processing
43:32 The complete list of themes
To learn more about the membership site, click here: https://forms.gle/9TEUw7bj3YTs7B4H6
Edited by Presage: https://pre-sage.com.
References:
Perhaps this goes without saying, but each of the issues that I mention are considerably more nuanced than how I've presented them here. But hopefully the video gives you the basic ideas so that you can dive in further, if you want.
Depth of processing: Craik, F. I., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of experimental Psychology: general, 104(3), 268.
Generation: Bertsch, S., Pesta, B. J., Wiscott, R., & McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & cognition, 35, 201-210.
Imagery: Leopold, C., & Mayer, R. E. (2015). An imagination effect in learning from scientific text. Journal of Educational Psychology, 107(1), 47–63. doi.org/10.1037/a0037142
"Time palaces" and memory palaces Bouffard, N., Stokes, J., Kramer, H. J., & Ekstrom, A. D. (2018). Temporal encoding strategies result in boosts to final free recall performance comparable to spatial ones. Memory & Cognition, 46(1), 17–31. doi.org/10.3758/s13421-017-0742-z
Value and encoding: Hennessee, J. P., Patterson, T. K., Castel, A. D., & Knowlton, B. J. (2019). Forget me not: Encoding processes in value-directed remembering. Journal of Memory and Language, 106, 29–39. doi.org/10.1016/j.jml.2019.02.001
Drawing: Fernandes, M. A., Wammes, J. D., & Meade, M. E. (2018). The Surprisingly Powerful Influence of Drawing on Memory. Current Directions in Psychological Science, 27(5), 302–308. doi.org/10.1177/0963721418755385
Wammes, J. D., Meade, M. E., & Fernandes, M. A. (2016). The drawing effect: Evidence for reliable and robust memory benefits in free recall. Quarterly Journal of Experimental Psychology, 69(9), 1752–1776. doi.org/10.1080/17470218.2015.1094494
Acting things out: Engelkamp, J., & Zimmer, H. D. (1997). Sensory factors in memory for subject-performed tasks. Acta Psychologica, 96(1-2), 43-60.
Production and distinctiveness: Ozubko, J. D., & MacLeod, C. M. (2010). The production effect in memory: evidence that distinctiveness underlies the benefit. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(6), 1543.
Integration and self-explanations: Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182.
Renkl, A. (2002). Worked-out examples: Instructional explanations support learning by self-explanations. Learning and instruction, 12(5), 529-556.
Integration and video games/passages:
Arena, D. A., & Schwartz, D. L. (2014). Experience and explanation: Using videogames to prepare students for formal instruction in statistics. Journal of Science Education and Technology, 23, 538-548.
Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–5223.
Transfer appropriate processing: Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16(5), 519–533. doi.org/10.1016/S0022-5371(77)80016-9
00:00 A quick note
00:47 The problem state space and the towers of Hanoi
4:45 Problems of representation and the Chinese ring puzzle
6:42 Context and variations of the Wason 4-card selection task
9:42 Introduction to insight problems: the candle problem
11:05 Differences between insight and incremental problems
12:15 Barriers to insight: Roman matchstick problems
17:30 Insight problems: too big of a distinction?
19:08 Well-structured and ill-structured problems
21:11 Representation and argument
23:34 Becoming a better problem solver: toothpick problems
26:45 Domain-specific knowledge and strategy change
30:55 What transfers across problem-solving domains?
Sign up to my newsletter here: http://www.benjaminkeep.com.
ACKNOWLEDGEMENTS
The towers of Hanoi footage comes from Math Playground's version of the towers.
The Chinese ring puzzle footage comes from Youtuber @Jojikiba, here youtube.com/watch?v=nydIl2VO12I
The candle problem visuals: youtube.com/watch?v=PEvThYxtnFQ. If you made it or you know who did, please let me know. The link above seems to be the oldest and most complete version.
Paul Zeitz photo: imo-official.org/advisory.aspx?year=2019
Math and STEP problems: imo-official.org/problems/IMO2022SL.pdf and usmle.org/sites/default/files/2021-10/Step_1_Sample_Items.pdf, respectively.
Physics problems: Badeau, R., White, D. R., Ibrahim, B., Ding, L., & Heckler, A. F. (2017). What works with worked examples: Extending self-explanation and analogical comparison to synthesis problems. Physical Review Physics Education Research, 13(2), 020112. doi.org/10.1103/PhysRevPhysEducRes.13.020112
Visit www.eternagame.org for more info on Eterna.
The Chess explanation clip: @BetterChessTraining.
The Starcraft 2 clip: an @ESChamp video, here: youtube.com/watch?v=4okBbHaCQ98
The go examples come from Sensei’s library: senseis.xmp.net/?CornersThenSidesThenCenter
Starcraft 2 map: news.blizzard.com/en-us/starcraft2/23223597/new-ladder-maps-for-2019-season-4
Parity example: artofproblemsolving.com/wiki/index.php?title=Parity
The free body diagram is from a Math and Science video: youtube.com/watch?v=mI11vHsRnC8
The map of Easter Island: en.wikipedia.org/wiki/File:Easter_Island_map-en.svg
The cashflow statement: en.wikipedia.org/wiki/Cash_flow_statement#cite_note-21, but is originally from Epstein, Barry J.; Eva K. Jermakowicz (2007). Interpretation and Application of International Financial Reporting Standards. John Wiley & Sons. pp. 91–97. ISBN 978-0-471-79823-1.
REFERENCES
Simon, H. A., & Newell, A. (1971). Human problem solving: The state of the theory in 1970. American psychologist, 26(2), 145.
The tower of Hanoi state space was adapted from Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive science, 18(1), 87-122.
The Chinese Ring puzzle state space (for 5 rings) is adapted from Kotovsky, K. & Simon, H. A. (1990). What Makes Some Problems Really hard: Explorations in the Problem Space of Difficulty. Cognitive Psychology, 22(2), 143–183.
The “underage drinking” example of the Wason 4-card task comes from this classic:
Griggs, R. A., & Cox, J. R. (1982). The elusive thematic‐materials effect in Wason's selection task. British journal of psychology, 73(3), 407-420.
A short meta-analysis on the Wason 4-card task: Ragni, M., Kola, I., & Johnson-Laird, P. (2017). The Wason Selection task: A Meta-Analysis. In CogSci. researchgate.net/profile/Ilir-Kola/publication/322682384_The_Wason_Selection_Task_A_Meta-Analysis/links/5a688e6f4585156abdffd541/The-Wason-Selection-Task-A-Meta-Analysis.pdf
On the insight experience: Webb, M. E., Little, D. R., & Cropper, S. J. (2016). Insight Is Not in the Problem: Investigating Insight in Problem Solving across Task Types. Frontiers in Psychology, 7. frontiersin.org/articles/10.3389/fpsyg.2016.01424
The Matchstick Roman numeral problems and the discussion of constraints and chunks come from: Öllinger, M., Jones, G., & Knoblich, G. (2008). Investigating the Effect of Mental Set on Insight Problem Solving. Experimental Psychology, 55(4), 269–282. doi.org/10.1027/1618-3169.55.4.269
On strategic change, see: Siegler, R. S. (2002). Microgenetic studies of self-explanation. Microdevelopment: Transition processes in development and learning, 31, 58.
On the importance of representations in ill-structured domains, see: Chandrasekharan, S., & Nersessian, N. J. (2011). Building cognition: the construction of external representations for discovery. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 33, No. 33).
The “Chinese is Hard” podcast: open.spotify.com/show/6DdGLH8neJoAkggOZu4VrN
William Hart’s Youtube channel: youtube.com/@will3267
Danyo Pang’s Youtube channel: youtube.com/@thedanyopang
00:00 That's a lot of words
00:24 Different kinds of vocabulary words
2:06 Word mapping between languages
3:16 Vocabulary goes beyond words
6:25 A story about prior knowledge
9:22 Learning words by example and in context
10:31 Flashcards
12:22 Comprehensible input
13:50 Pushed output
Sign up to my email newsletter, here: http://www.benjaminkeep.com
Acknowledgements:
Definitions of “chew” comes from dictionary.com; definitions for “hard,” “take for granted,” and "repetition" come from Merriam-Webster, slightly adapted.
References:
The observations in this video are a mish-mash of my own personal experience and my reading of the literature on language learning. The best survey of that literature that I know of is How Languages are Learned by Patsy Lightbrown and Nina Spada. The 4th edition is here: bookshop.org/a/91541/9780194541268 (bookshop links are affiliate links, so if you purchase through this link, I do get a small commission; I would try to find this from your local library, though).
It’s written with foreign language teachers in mind (not necessarily foreign language students), but hits on many of the key ideas, mentions a lot of great studies that I won’t enumerate here, and offers a relatively balanced portrait of the research on language learning.
The 10,000 word estimate I mentioned at the beginning of the video is low and I wish I would have been more accurate. A lot depends on how you’re defining a word and how you define what “knowing” means. This is a solid paper on estimating the vocabulary of native American English speakers: Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). How Many Words Do We Know? Practical Estimates of Vocabulary Size Dependent on Word Definition, the Degree of Language Input and the Participant’s Age. Frontiers in Psychology, 7. frontiersin.org/articles/10.3389/fpsyg.2016.01116.
The upshot is that they estimate about ~11,000 word families and 42,000 lemmas on average. A lemma is a root word that you can derive into other things (baked, bake, baking would not be three separate words - just one lemma). Word families are larger groups of related words with similar forms and related meanings.
Comprehensible Input is one of the main ideas in Stephen Krashen’s Monitor Theory. This is the classic piece that lays it out: Krashen, S. (1981). Second language acquisition. Second Language Learning, 3(7), 19-39. https://www.academia.edu/download/35238869/second_languge_acquisition_and_learning.pdf
The idea of pushed output comes from Merrill Swain and her studies of proficiency in immersive classrooms. She's got a lot to say on it over the years, but here's a representative example: Swain, M. (1997). Collaborative dialogue: Its contribution to second language learning. riull.ull.es/xmlui/bitstream/handle/915/30603/RCEI_34_%281997%29_07.pdf?sequence=1
There’s much more to say about comprehensible input and pushed output, and I’ve got plans to release videos on those two ideas at least.
Link to hear news on my membership site: https://forms.gle/2XzhvnGptxeYc28n8
Sign up to my free newsletter, Avoiding Folly, here: http://www.benjaminkeep.com
00:00 Starting with a question
00:53 Lectures and learning
2:10 On the popularity of the lecture format
2:56 Preparing for a lecture
4:38 Making the most of reading assignments
6:33 The goals of preparation: cognitive load
7:37 Create relevant prior knowledge
8:10 Get relevant practical experience
8:29 Begin the organizational process
9:06 Going to the lecture
10:54 Good attention
11:47 Attention drops during lecture
12:29 Dealing with online lectures
13:37 Reviewing after the lecture
14:42 Creative ways of reviewing
16:20 Fundamentals are key
17:26 Frameworks to help you review
19:50 Review vs doing homework
21:38 A little goes a long way
ACKNOWLEDGEMENTS
The scrolling text when I was talking about cognitive load comes from an excerpt from an old Vanity Fair magazine, written by P.G. Wodehouse, available here: vanityfair.com/culture/2014/10/physical-culture-peril-pg-wodehouse-best-of-early-vanity-fair.
The green screen Youtube video (talking about stopping at 10 minutes, 30 minutes) came from here: youtube.com/watch?v=q-7LExVpU4g
I used an example lecture from Scott Page’s Model Thinking on Coursera, found here. The specific excerpt, was taken from the first lecture on aggregation. coursera.org/lecture/model-thinking/aggregation-UWXGF. I definitely recommend Scott Page’s work, especially if you’re interested in social science and social dynamics.
The "taking a test" example came from an MIT course on differential equations: https://ocw.mit.edu/courses/18-03-differential-equations-spring-2010/resources/mit18_03s10_ex1/.
REFERENCES
On attention during lectures. The basic take-away is that passive learning during lectures is similar to a vigilance task (asking people to monitor for low frequency signals from a background of noise - radar monitoring for instance). 10-30 minutes in, steep attention decrements.
Young, M. S., Robinson, S., & Alberts, P. (2009). Students pay attention!: Combating the vigilance decrement to improve learning during lectures. Active Learning in Higher Education, 10(1), 41–55. doi.org/10.1177/1469787408100194
If you are a teacher looking to improve student learning during lectures, this piece from almost 40 years ago still holds up. If you’re a science teacher at the undergraduate level, I recommend visiting the Carl Wieman Science Education Initiative.
Gibbs, G., Habeshaw, S., & Habeshaw, T. (1987). Improving Student Learning During Lectures. Medical Teacher, 9(1), 11–20. doi.org/10.3109/01421598709028976
This is a more modern piece covering some of the same ground:
Cerbin, W. (2018). Improving student learning from lectures. Scholarship of Teaching and Learning in Psychology, 4(3), 151–163. doi.org/10.1037/stl0000113
00:00 Last time...
00:40 Is tracking my hours helpful?
01:08 Is the diagnostic test a good measure?
01:55 Speed vs accuracy
03:22 Interleaved vs deliberate practice
04:54 The importance of seeing progress
05:50 Midway diagnostic results
06:15 Practice and performance gains
07:08 Skill integration
09:33 What the test measures
10:12 Are your mistakes changing?
11:15 What I’m focusing on next
12:21 The major system
14:17 28 hours in
14:39 The development of intuition
16:29 35 hours in
18:38 The return of the major system
19:45 Lessons for learning something on your own
Sign up to my email newsletter on reasoning, social science, and decision-making: http://www.benjaminkeep.com
Interested in courses on learning and studying? Sign up to be notified when my site launches: https://forms.gle/xzi1ovwkzCHKvmP58
The math course I was working from was: thegreatcourses.com/courses/secrets-of-mental-math, but Arthur Benjamin also has a book teaching the same things: bookshop.org/a/91541/9780307338402. The bookshop link is an affiliate link so I get a buck or something if you buy it through this link (thanks, if you do!).
Some thoughts on interleaved practice: youtu.be/AWTYfzxBwPg
Some thoughts on deliberate practice: youtu.be/WbUOY9ioIqw (tackling some myths about deliberate practice); youtu.be/3SZDj5TEhU0 (on criticisms of the deliberate practice model)
A quick intro to the novice to expert transition: youtu.be/qwDzjQsM3yQ
0:00 What I'm trying to do
0:50 What does it mean to learn from a book?
2:27 Writing down themes
4:27 The back-and-forth
6:01 The function of notes
7:01 Taking paths through the material
7:36 Why do my visualizations suck?
8:53 Getting precise
10:20 Why these (bad) visualizations?
11:14 Spacing things out
12:20 What I would do differently next time
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter here: http://www.benjaminkeep.com.
REFERENCES:
The book is China: A History by John Keay.
My practices here come from a variety of sources, but here's some representative work.
This piece is about teaching students higher-level reading strategies to aid deep comprehension (constructing inferences and explanations, self-monitoring, etc.). A useful read that I largely agree with: McNamara, D. S. (2010). Strategies to read and learn: Overcoming learning by consumption. Medical Education, 44(4), 340–346. doi.org/10.1111/j.1365-2923.2009.03550.x
On free recall and retrieval:
Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021). Retrieval Practice Consistently Benefits Student Learning: A Systematic Review of Applied Research in Schools and Classrooms. Educational Psychology Review, 33(4), 1409–1453. doi.org/10.1007/s10648-021-09595-9
Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. doi.org/10.1016/j.tics.2010.09.003
Karpicke, J. D., & Roediger, H. L. (2008). The Critical Importance of Retrieval for Learning. Science, 319(5865), 966–968. doi.org/10.1126/science.1152408
On the value of "spacing things out" AKA long inter-study intervals, see:
Latimier, A., Peyre, H., & Ramus, F. (2021). A Meta-Analytic Review of the Benefit of Spacing out Retrieval Practice Episodes on Retention. Educational Psychology Review, 33(3), 959–987. doi.org/10.1007/s10648-020-09572-8
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi.org/10.1037/0033-2909.132.3.354
Acknowledgements:
Image of Liu Bang (founding emperor of the Han Dynasty) comes from: en.wikipedia.org/wiki/File:Liu-bang.jpg. It’s actually a Ming-era depiction
Image of Xiang Yu (Liu Bang’s rival) comes from: en.wikipedia.org/wiki/File:Xiang_Yu.jpg
Xuanzang image: en.wikipedia.org/wiki/File:Xuanzang_w.jpg
Xuanzang’s route: en.wikipedia.org/wiki/File:Xuanzang_route.jpg
Zhang Qian’s route: en.wikipedia.org/wiki/File:Han_Expansion.png
Zheng He’s ships: en.wikipedia.org/wiki/File:ZhengHeShips.gif
Zheng He’s route: en.wikipedia.org/wiki/File:Voyages_of_Zheng_He.png (author: SY)
Topographic map: en-us.topographic-map.com/map-pxv3q/China
0:00 This video is a follow-up video
0:39 Was there an effect?
3:12 An attempt to visualize the effect
4:36 Sources of variation
5:13 Encoding strategies and non-native speakers
7:30 Why what I just did wasn't really legit
8:14 Variation among experimental designs
10:16 A common source of error
12:12 Oh no, it's an ad...
To be notified when my membership site launches, go here: https://forms.gle/9Q8aDCrSPEkpx8Zn8
Sign up to my email newsletter here: http://www.benjaminkeep.com
REFERENCES
The first one is the meta-analysis that I took screenshots of for the video. The second one is more recent and delves deeper into the theories that explain the effect.
Bertsch, S., Pesta, B. J., Wiscott, R., & McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & Cognition, 35(2), 201–210. doi.org/10.3758/BF03193441
McCurdy, M. P., Viechtbauer, W., Sklenar, A. M., Frankenstein, A. N., & Leshikar, E. D. (2020). Theories of the generation effect and the impact of generation constraint: A meta-analytic review. Psychonomic Bulletin & Review, 27(6), 1139–1165. doi.org/10.3758/s13423-020-01762-3
0:00 An introduction to Bjork's experiments
0:34 Let's try a little experiment.
4:55 Figuring out your score.
5:32 What's the expected result?
5:58 How did Dr. Bjork use this to help people learn?
7:05 The surprising result.
7:44 What's really driving the learning in the second round?
9:49 The larger point.
The follow-up video is here: youtu.be/7N7tIM7sIxU
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter here: http://www.benjaminkeep.com.
REFERENCES
This video and the experiments I describe in it was based on a series of experiments led by Elizabeth Bjork, renowned memory researcher. The first one is the most readable summary of the research; the other three are a bit dense, but give lots of details about how the experiments were run.
Bjork, E. L., de Winstanley, P. A., & Storm, B. C. (2007). Learning how to learn: Can experiencing the outcome of different encoding strategies enhance subsequent encoding? Psychonomic Bulletin & Review, 14(2), 207–211. doi.org/10.3758/BF03194053
A. de Winstanley, P., & Ligon Bjork, E. (1997). Processing Instructions and the Generation Effect: A Test of the Multifactor Transfer-appropriate Processing Theory. Memory, 5(3), 401–422. doi.org/10.1080/741941392
DeWinstanley, P. A., & Bjork, E. L. (2004). Processing strategies and the generation effect: Implications for making a better reader. Memory & Cognition, 32(6), 945–955. doi.org/10.3758/BF03196872
Bjork, E. L., & Storm, B. C. (2011). Retrieval experience as a modifier of future encoding: Another test effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1113–1124. doi.org/10.1037/a0023549
This is the one where they illustrate that students don’t have to experience the benefits of encoding to improve their encoding strategy. Rather, it seems like just knowing the format of the test and having some experience with different encoding strategies is enough (separately and likely additively together) to lead to superior encoding.
Storm, B. C., Hickman, M. L., & Bjork, E. L. (2016). Improving encoding strategies as a function of test knowledge and experience. Memory & Cognition, 44(4), 660–670. doi.org/10.3758/s13421-016-0588-9
The word lists used in the video were generated from Friendly, M. & Dubins, M. (2019), Paivio et al. Word List Generator, Online application, http://euclid.psych.yorku.ca/shiny/Paivio, Accessed: 2023-06-23
0:00 An under-appreciated distinction.
0:34 What is a passive control group?
1:36 What is an active control group?
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter here: http://www.benjaminkeep.com.
REFERENCES
A good explanation of the difference between passive control groups and active control groups begins on page 116, here: Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016). Do “Brain-Training” Programs Work? Psychological Science in the Public Interest, 17(3), 103–186. doi.org/10.1177/1529100616661983
0:00 Memory athletes can do some amazing things
0:34 Techniques memory athletes use
1:59 The argument against using mnemonics - Reason #1
4:13 The argument against using mnemonics - Reason #2
5:50 The argument in favor of using mnemonics
8:23 Incorporating mnemonic techniques into your studying
8:41 The case of single-digit multiplication
9:18 Chinese emperors and Chinese geography
10:40 The periodic table
11:25 Deciding whether to use mnemonic techniques
12:30 Memory emergencies
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
For more on mnemonic techniques, check out the Art of Memory: artofmemory.com
Sign up to my email newsletter, Avoiding Folly, here: http://www.benjaminkeep.com
REFERENCES
The piece below reports something like the consensus among cognitive psychologists: that mnemonics are pretty good for remembering serial items of unrelated information for short periods of time, but have unproven relevance in the classroom.
Putnam, A. L. (2015). Mnemonics in education: Current research and applications. Translational Issues in Psychological Science, 1(2), 130–139. doi.org/10.1037/tps0000023
This is a piece from a psychologist who taught her students the method of loci (AKA memory palace) that showed some modestly positive results, at least on list learning and general attitude toward the method.
McCabe, J. A. (2015). Location, Location, Location! Demonstrating the Mnemonic Benefit of the Method of Loci. Teaching of Psychology, 42(2), 169–173. doi.org/10.1177/0098628315573143
For a novel application of the method of loci, here is a piece exploring how it might be leveraged for emotional regulation.
Werner-Seidler, A., & Dalgleish, T. (2016). The Method of Loci Improves Longer-Term Retention of Self-Affirming Memories and Facilitates Access to Mood-Repairing Memories in Recurrent Depression. Clinical Psychological Science, 4(6), 1065–1072. doi.org/10.1177/2167702615626693
ACKNOWLEDGEMENTS
Footage from a card memorization event is from: youtube.com/watch?v=OApSOU7NIAw
The periodic table is from the NIH website - amazing! pubchem.ncbi.nlm.nih.gov/periodic-table
The map of China rivers comes from Antonioga, here: en.wikipedia.org/wiki/File:China_rivers.svg
Image of the Loess landscape, in China, comes from Till Niermann, here: en.wikipedia.org/wiki/File:Loess_landscape_china.jpg
The image of Gansu province (camel photo) comes from Sigismund von Dobschütz, here: en.wikipedia.org/wiki/File:Mondsichelsee-06.JPG
The map of traditional geographical and cultural regions in China comes from SY, here: en.wikipedia.org/wiki/File:China_Traditional_Divisions.png
Images related to evolutionary processes come from: Elimbis at Wikimedia commons (commons.wikimedia.org/wiki/File:Mutation_and_selection_diagram.svg) Esurnir at the English Wikipedia (Allele frequency, en.wikipedia.org/wiki/File:Allele-frequency.png); and speciation here (en.wikipedia.org/wiki/File:Speciation_modes_edit.svg)
Image of the Tang Dynasty comes from Seasonsinthesun at Wikimedia Commons (commons.wikimedia.org/wiki/File:Tang_Protectorates.png). Image of the Roman Empire comes from Paulusburg at Wikimedia Commons (commons.wikimedia.org/wiki/File:Mapa_Imperio_Occidental_a%C3%B1o_418.svg)
0:00 So... you want to teach your kid math?
0:15 Four categories of learning goals
1:35 Some important conceptual learning goals
3:14 Some important procedural learning goals
4:36 What does your kid associate math with?
5:09 Becoming a problem solver
5:48 Setting up the time and the environment
6:25 How to use examples and abstractions
7:13 Developing more sophisticated strategies
8:28 Letting them solve the problem
9:07 Developing a skeptical and thorough problem solver
9:42 Flip the asking script
10:32 Modeling how to respond to mistakes
11:41 "Secrets"
12:18 Interesting ways of incorporating number lines
12:55 Scaffolding estimation skills
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my newsletter, Avoiding Folly, here: www.benjaminkeep.com
REFERENCES:
For readable summaries of the research in these areas, check out these two excellent posts by Gwen Dewar: parentingscience.com/preschool-board-game-math (preschool math games) and parentingscience.com/preschool-math-lessons (general advice). Parenting Science is just an excellent resource, generally.
On concreteness fading and the power of using both concrete and abstract representations together, see the two references below:
Fyfe, E. R., & Nathan, M. J. (2019). Making “concreteness fading” more concrete as a theory of instruction for promoting transfer. Educational Review, 71(4), 403–422. doi.org/10.1080/00131911.2018.1424116
Moreno, R., Ozogul, G., & Reisslein, M. (2011). Teaching with concrete and abstract visual representations: Effects on students' problem solving, problem representations, and learning perceptions. Journal of educational psychology, 103(1), 32.
On the relationship between conceptual and procedural knowledge and the role that different kinds of examples can play in developing both, see: Rittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101(3), 529. https://dash.harvard.edu/bitstream/handle/1/4889483/CC3b_Final.pdf
On the use of number lines for learning: Lin, C. H. (2022). Developing mental number line games to improve young children’s number knowledge and basic arithmetic skills. Journal of Experimental Child Psychology, 222, 105479. sciencedirect.com/science/article/abs/pii/S0022096522001084
0:00 A brief introduction
0:23 What kinds of study breaks should you take?
1:59 Where should you study?
3:34 What kinds of notes should you take?
4:38 Deep processing, shallow processing, and transfer-appropriate processing
7:11 How should you structure study groups?
10:17 Ways to study that avoid confusion between recognition and recall
11:49 The skill of reading textbooks
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter here: http://www.benjaminkeep.com
RELATED VIDEOS
Marty Lobdell's Study Less, Study Smart is here: youtube.com/watch?v=IlU-zDU6aQ0
For alternatives to highlighting and flashcards, see:
How to do free recall - youtu.be/FatnXnlwAc4
How to Use Free Recall to Learn More Effectively - youtu.be/evIuDvQDDSY
A simple study tip: stop highlighting - youtu.be/_g3NojwBOXw
For my take on note-taking:
The surprising truth about note-taking - youtu.be/cRQqH18wJgw
On reading textbooks (and reading more generally):
Three simple tricks to read textbooks more effectively - youtu.be/neQ7D90R0xo
Learn more from every paragraph - youtu.be/WRjsOU6mOp4
On attitude and framing:
The most common obstacle to effective studying - youtu.be/6i7HrP84DMw
REFERENCES
On restful periods, see: Wamsley, E. J. (2019). Memory consolidation during waking rest. Trends in cognitive sciences, 23(3), 171-173. ncbi.nlm.nih.gov/pmc/articles/PMC7024394/pdf/nihms-1557224.pdf
The screenshot of the graph showing wakeful resting benefits to wakeful active groups comes from research on auditory learning, here: Gottselig, J. M., Hofer-Tinguely, G., Borbely, A. A., Regel, S. J., Landolt, H. P., Retey, J. V., & Achermann, P. (2004). Sleep and rest facilitate auditory learning. Neuroscience, 127(3), 557-561.
On eyestrain, see: Kaur, K., Gurnani, B., Nayak, S., Deori, N., Kaur, S., Jethani, J., ... & Mishra, D. (2022). Digital Eye Strain-A Comprehensive Review. Ophthalmology and Therapy, 11(5), 1655-1680.https://link.springer.com/article/10.1007/s40123-022-00540-9
On the benefits of self-testing study strategies, see: Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6
On the weak benefits of highlighting and a fantastic review of study strategies more generally, see: Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychological Science in the Public Interest, 14(1), 4–58. doi.org/10.1177/1529100612453266
On the value of teaching (or preparing to teach) others, see: Okita, S. Y., & Schwartz, D. L. (2013). Learning by Teaching Human Pupils and Teachable Agents: The Importance of Recursive Feedback. Journal of the Learning Sciences, 22(3), 375–412. doi.org/10.1080/10508406.2013.807263
The classic piece on transfer appropriate processing is here: Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16(5), 519–533. doi.org/10.1016/S0022-5371(77)80016-9. For a more modern view in the context of an experiment, see this piece: A. de Winstanley, P., & Ligon Bjork, E. (1997). Processing Instructions and the Generation Effect: A Test of the Multifactor Transfer-appropriate Processing Theory. Memory, 5(3), 401–422. doi.org/10.1080/741941392
On deep and shallow processing, see: Craik, F. I. M. (2002). Levels of processing: Past, present... and future? Memory, 10(5–6), 305–318. doi.org/10.1080/09658210244000135
On the memory benefit of learning in different environments, one of the classics is: Smith, S. M., Glenberg, A., & Bjork, R. A. (1978). Environmental context and human memory. Memory & Cognition, 6(4), 342-353.
On links between exercise, sleep, and learning, see: Roig, M., Cristini, J., Parwanta, Z., Ayotte, B., Rodrigues, L., de Las Heras, B., ... & Wright, D. L. (2022). Exercising the sleepy-ing brain: exercise, sleep, and sleep loss on memory. Exercise and sport sciences reviews, 50(1), 38-48.
On links between walking and creative thinking, see: Oppezzo, M., & Schwartz, D. L. (2014). Give your ideas some legs: the positive effect of walking on creative thinking. Journal of experimental psychology: learning, memory, and cognition, 40(4), 1142.
ACKNOWLEDGEMENTS
Spanish Inquisition clip from: youtube.com/watch?v=yKQ_sQKBASM
0:00 An intro to reading social science research
0:39 What do learning researchers study?
1:54 The basics of experimental designs
5:01 What’s the goal of reading research?
5:16 The first big question: why?
6:58 The second big question: what happened?
8:12 What’s the participant’s experience?
9:07 A tip for reading complex graphs.
10:23 The third big question: the explanation.
11:04 Some complexities when measuring learning
13:45 What’s the question again?
14:45 Last thoughts
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my newsletter, Avoiding Folly, here: www.benjaminkeep.com
REFERENCES
Clips come from the following pieces:
Example of a longitudinal study: Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363.
Example of qualitative data and mixed qualitative and quantitative data come from: Reading Abraham Lincoln: An Expert/Expert Study in the Interpretation of Historical Texts. (2003). 1–28.
Example of quantitative data comes from: Bjork, E. L., & Storm, B. C. (2011). Retrieval experience as a modifier of future encoding: Another test effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1113–1124. doi.org/10.1037/a0023549
Example of abstract/laboratory experiments comes from: Bjork, R. A. (1975). Retrieval as a memory modifier: An interpretation of negative recency and related phenomena.
Example of an applied study (using social studies material in the classroom): Roediger, H. L., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17(4), 382–395. doi.org/10.1037/a0026252
Examples of an experimental study, data visualization that describes the nature of the materials and one that describes the course of the experiment from: Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.
Research studying the effects of delayed vs immediate feedback are legion. This provides good background reading: Fyfe, E. R., & Rittle-Johnson, B. (2016). Feedback both helps and hinders learning: The causal role of prior knowledge. Journal of Educational Psychology, 108(1), 82–97. doi.org/10.1037/edu0000053
The example of worked examples vs open problem solving comes from some of Sweller’s earlier experiments. Here’s a typical example. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and instruction, 2(1), 59-89.
The statistics game and passage examples I drew from Arena, D. A., & Schwartz, D. L. (2013). Experience and Explanation: Using Videogames to Prepare Students for Formal Instruction in Statistics. Journal of Science Education and Technology, 23(4), 538–548. doi.org/10.1007/s10956-013-9483-3
The screenshot of a statistics passage is from one of the first pages of Regression and Other Stories by Gelman, Hill, and Vehtari, a great textbook on regression.
The design-based research example comes from Ketelhut, D. J., Nelson, B. C., Clarke, J., & Dede, C. (2010). A multi‐user virtual environment for building and assessing higher order inquiry skills in science. British Journal of Educational Technology, 41(1), 56-68. They created a virtual environment called “River City” to facilitate science inquiry experiences.
The “delayed measures” being different than immediate measures graph is from Taylor, K., & Rohrer, D. (2010). The effects of interleaved practice. Applied Cognitive Psychology, 24(6), 837–848. doi.org/10.1002/acp.1598
The “Stats Invaders” clip (the statistics game) comes courtesy of Can Zhang, here: youtube.com/watch?v=J7rx1g8g1Uk
0:00 Is one recall mode better than another?
1:21 What should the recall interval be?
2:20 Do I need to master the material first?
2:43 Which is better, encoding or retrieval?
4:03 Is free recall just vomiting out words?
4:57 Some further thoughts
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
REFERENCES
On the differences between different modes of recall, see the three references below:
Janczyk, M., Aßmann, M., & Grabowski, J. (2018). Oral versus written recall of long-term memory items: Replicating and extending the writing superiority effect across knowledge domains. The American Journal of Psychology, 131(3), 263-272. (Finding a written superiority effect)
Bekerian, D. A., & Dennett, J. L. (1990). Spoken and written recall of visual narratives. Applied Cognitive Psychology, 4(3), 175-187. (spoken superiority)
Sauerland, M., Krix, A. C., van Kan, N., Glunz, S., & Sak, A. (2014). Speaking is silver, writing is golden? The role of cognitive and social factors in written versus spoken witness accounts. Memory & Cognition, 42, 978-992. (favoring written, but not finding large differences; in the eyewitness area)
For a wonderful summary of how tests effect learning, check out: Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6
On test expectancy effects (how expecting a test influences encoding), see: McDaniel, M. A., Blischak, D. M., & Challis, B. (1994). The effects of test expectancy on processing and memory of prose. Contemporary educational psychology, 19(2), 230-248.
0:00 An Intro to Rote Learning
1:01 Drawback 1 - Blocked Practice
2:57 Drawback 2 - Massed Practice
4:05 Drawback 3 - Skill Isolation
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
REFERENCES
For a recent take on spaced (or distributed) practice, including evidence from neurological studies, read:
Gerbier, E., & Toppino, T. C. (2015). The effect of distributed practice: Neuroscience, cognition, and education. Trends in Neuroscience and Education, 4(3), 49–59. doi.org/10.1016/j.tine.2015.01.001
If you want a more academic take, discussing theories and the available evidence circa 2006, check out:
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi.org/10.1037/0033-2909.132.3.354
For a readable piece on the advantages (and potential drawbacks) of interleaved practice, check out:
Kang, S. H. (2016). The benefits of interleaved practice for learning. In From the laboratory to the classroom (pp. 79-90). Routledge. taylorfrancis.com/chapters/edit/10.4324/9781315625737-8/benefits-interleaved-practice-learning-sean-kang
And a great piece on how to leverage interleaved, spaced, and skill-building practice in the classroom is here:
Hughes, C. A., & Lee, J. Y. (2019). Effective approaches for scheduling and formatting practice: Distributed, cumulative, and interleaved practice. Teaching Exceptional Children, 51(6), 411-423. researchgate.net/profile/Joo-Young-Lee-8/publication/338701579_Effective_Approaches_for_Scheduling_and_Formatting_Practice_Distributed_Cumulative_and_Interleaved_Practice/links/5db7099aa6fdccac19b5d67f/Effective-Approaches-for-Scheduling-and-Formatting-Practice-Distributed-Cumulative-and-Interleaved-Practice.pdf
00:00 Some drawbacks of using podcasts to learn
01:49 Step 1 - Activate prior knowledge
02:42 Step 2 - Free recall and fill in the gaps
05:49 Step 3 - Consolidate and extend
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
ACKNOWLEDGEMENTS
Screenshots of Twilight of the Aesir and Hardcore History come from dancarlin.com.
Jutland and Saxony screenshots come from Wikipedia.
I use Miro for the visualization software clip.
REFERENCES
My comments about what free recall self-tests can do for you are based on the large literature related to the testing effect. At a high-level, frequent tests enhance learning in a variety of ways.
For a thorough discussion of the possible benefits (including anticipatory/motivational aspects and organizational aspects, like I mention in the video), check out:
Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6
Here is a recent meta-analysis exploring different potential mechanisms and applications of the testing effect(s):
Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin, 147(4), 399–435. doi.org/10.1037/bul0000309
0:00 An intro to tacit knowledge
0:45 Let's try an experiment
1:42 "Weak tacit knowledge"
3:36 Expert blindness
5:31 "Body knowledge"
8:03 "Social tacit knowledge" and Searle's Chinese Room
11:10 How do you learn tacit knowledge?
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
REFERENCES
The bones of the video come from: Collins, H. (2019). Tacit and explicit knowledge. University of Chicago press. I changed the labels he gave for the different kinds of tact knowledge he identifies because I found them pretty confusing.
Michael Polanyi is credited with first identifying and discussing tacit knowledge. His thoughts can be found in: Polanyi, M., & Sen, A. (2009). The tacit dimension. University of Chicago press.
The "explicit to implicit" model of skill acquisition is commonly exemplified by the Dreyfus model:
Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in directed skill acquisition. California Univ Berkeley Operations Research Center. https://apps.dtic.mil/sti/pdfs/ADA084551.pdf
A good introduction to Searle's Chinese Room (with plenty of references), can be found here: https://plato.stanford.edu/entries/chinese-room/
ACKNOWLEDGEMENTS
Star Trek clip from youtube.com/watch?v=hdjL8WXjlGI
The physics of bike riding visuals come from this gem from 1970: Jones, D. E. (1970). The stability of the bicycle. Physics today, 23(4), 34-40. Available here: https://home.phys.ntnu.no/brukdef/undervisning/tfy4145/arkiv/2010/diverse/UnridableBicycle.pdf
The clip of a robot riding a bicycle came from: youtube.com/watch?v=mT3vfSQePcs
“Frampold” entry and other rare word screenshots from the OED found here: oed.com/view/Entry/74166
The recipe screenshots are from Serious Eats: seriouseats.com/the-food-lab-how-to-make-creamy-vegetable-soup; seriouseats.com/spicy-carrot-and-ginger-soup-with-harissa.
Urban Dictionary add word screen found here: my.urbandictionary.com/add.php
Image of Harry Collins from Alexei Kouprianov on Wikipedia. upload.wikimedia.org/wikipedia/commons/5/5f/Harry_Collins.20101118.jpg
0:00 An intro to the experiment
0:21 Skills with many interdependent parts
1:00 The study that explored learning complex skills
2:33 The results
3:48 Try it out
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
*There was a wrinkle to this study that I didn't mention, which is that groups 2, 3, and 4 all had a different kind of display on the game for the first six sessions, which gave them slightly different information than the standard game. The idea was to support learner's focus on part of the game, but it also meant that there's more than one difference between the control and the other groups. The enhanced display could explain some of the difference between the control group and the other groups, but doesn't explain the differences between groups 2 and 3 and group 4. And the variable-priority approach (what I call rotating focus) that the study helped establish is well-supported by other studies as well - see the meta-analysis in the reference section for more support.
ACKNOWLEDGMENTS
Starcraft clip from LowkoTV: youtube.com/watch?v=kz9M2uYcFyE
Space Fortress mobile game footage from: play.google.com/store/apps/details?id=com.mgmamac.space_fortress&gl=US
Space Fortress DOS game footage from: classicreload.com/space-fortress.html
You can play an updated version of the original Space Fortress game at brainturk.com/space-fortress
REFERENCES
This video is based on the paper below. I replicated their figure 2 starting at around 2:40. Gopher, D., Weil, M., & Siegel, D. (1989). Practice under changing priorities: An approach to the training of complex skills. Acta Psychologica, 71(1–3), 147–177. doi.org/10.1016/0001-6918(89)90007-3
Here is a meta-analysis illustrating the efficacy of what I call "rotating focus" but what is called "variable-priority" training in the literature. Wickens, C. D., Hutchins, S., Carolan, T., & Cumming, J. (2013). Effectiveness of Part-Task Training and Increasing-Difficulty Training Strategies: A Meta-Analysis Approach. Human Factors, 55(2), 461–470. doi.org/10.1177/0018720812451994
And if you search more articles from Daniel Gopher, you will see other works on the same theme. Like this one: Brickner, M., & Gopher, D. (1981). Improving Time-Sharing Performance by Enhancing Voluntary Control on Processing Resources. TECHNION-ISRAEL INST OF TECH HAIFA CENTER OF HUMAN ENGINEERING AND INDUSTRIAL SAFETY RESEARCH. https://apps.dtic.mil/sti/pdfs/ADA118558.pdf
0:00 The paragraph in question
1:20 Some ways to approach learning from this paragraph
2:08 How I read it
5:17 What does reading something actually mean?
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
Here's the video I plug at the end: What Speed readers won’t tell you - youtu.be/D2aZ3THL8BQ
Here is the book that I took the paragraph from: Emling, S. (2009). The fossil hunter: dinosaurs, evolution, and the woman whose discoveries changed the world. St. Martin's Press. bookshop.org/a/91541/9780230103429 (I am a bookshop.org affiliate, which means I get small commission if you purchase through this link.)
I should say that there is more to good reading than what I have outlined in the video, but at least it demonstrates some basics.
For a concise summary of what good readers do, check out the first page of the following article: Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading comprehension. Journal of education, 189(1-2), 107-122. asset-pdf.scinapse.io/prod/45283437/45283437.pdf
Happy reading!
0:00 How to explore the claims of speed readers
1:09 Tim Ferris's speed reading techniques
3:26 The relationship between eye movements and reading comprehension
4:49 Tests of reading comprehension
5:52 Maximum words per minute (WPM)
6:46 Why do people believe in speed reading?
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
More learning myths: youtu.be/jcMk3e7u7zM
More on when learning fast backfires: youtu.be/wRb32j6_pD4
Thomas Frank’s videos on speed reading go over the research quite well, too. Check them out here: youtube.com/watch?v=jv2BdHXRD3Q (part 1), youtube.com/watch?v=JL4WMHyUhdc (part 2).
ACKNOWLEDGEMENTS
Opening screenshots of the Spreeder app found here: spreeder.com
Excerpts of Tim Ferris's speed reading video comes from: youtube.com/watch?v=ZwEquW_Yij0. The screenshot of comments comes from there, too.
Demo of the Spritz speed reading app comes from: readmei.com
I used this speed reading tool to display the words "This doesn't work at all" in the video: tools-unite.com/tools/speed-reading-app
The sentence I used to illustrate that people pause at the end of sentences comes from The Man Without Qualities by Robert Musil, translated by Sophie Wilkins and Burton Pike (bookshop.org/p/books/man-without-qualities-robert-musil/18761811): "There is, in short, no great idea that stupidity cannot put to its own uses; it can move in all directions, and put on all the guises of truth." A difficult, but hilarious and profound and tragic book.
REFERENCES
The most thorough description of the reading process, and why speed reading doesn’t work, is here: Rayner, K., Schotter, E. R., Masson, M. E., Potter, M. C., & Treiman, R. (2016). So much to read, so little time: How do we read, and can speed reading help?. Psychological Science in the Public Interest, 17(1), 4-34. journals.sagepub.com/doi/pdf/10.1177/1529100615623267
The paper that tested serial presentation of words (in other words, no eye movements, just rapid word presentation like popular speed reading apps such as Spritz) is: Acklin, D., & Papesh, M. H. (2017). Modern speed-reading apps do not foster reading comprehension. The American Journal of Psychology, 130(2), 183-199.. researchgate.net/profile/Dina-Acklin/publication/316499956_Modern_Speed-Reading_Apps_Do_Not_Foster_Reading_Comprehension/links/598de0b5a6fdcc1225fca2cc/Modern-Speed-Reading-Apps-Do-Not-Foster-Reading-Comprehension.pdf
For good background reading, especially for seeing how old false claims about speed reading are, check out: Brozo, W. G., & Johns, J. L. (1986). A content and critical analysis of 40 speed reading books. Journal of Reading, 30(3), 242-247.
For support of the idea that regressions (which I called backtracking in the video) HELP comprehension, see below. I used their fig 2 (page 4) to show how regressions facilitate reading comprehension. Schotter, E. R., Tran, R., & Rayner, K. (2014). Don’t believe what you read (only once) comprehension is supported by regressions during reading. Psychological science, 25(6), 1218-1226. journals.sagepub.com/doi/pdf/10.1177/0956797614531148
For a readable summary of the research on speed reading, check out the introduction to this piece. I used the graph of speed reading vs normal reading in the video as well (fig 3, page 7). Weimer, C. (2021). Effects of Reading Speed and Retrieval Practice on Reading Comprehension. Nysa, the NKU Journal of Student Research; v. 2, Fall 2019. https://dspace.nku.edu/bitstream/handle/11216/3673/OCR%20Weimer_The%20Effects%20of%20Reading%20Speed%20and%20Retrieval%20Practice%20on%20Reading%20Comprehension_Nysa_2_2019.pdf?sequence=1
For information on developing reading fluency for non-native speakers, which is a different but related research question, check out this one: Nation, P. (2009). Reading faster. International Journal of English Studies, 9(2). revistas.um.es/ijes/article/download/90791/87591
0:00 What does active processing mean?
0:59 What does active learning mean?
2:14 The relationship between active processing and active learning
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my newsletter, Avoiding Folly, here: benjaminkeep.com
REFERENCES:
On active learning in the classroom, I recommend:
Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. doi.org/10.1073/pnas.1319030111
Active processing is a big picture idea that crosses many areas of learning, but here is a general paper discussing the role of active processing in remembering:
Schwartz, B. L., Son, L. K., Kornell, N., & Finn, B. (n.d.). Four Principles of Memory Improvement: A Guide to Improving Learning Efficiency. https://web.williams.edu/Psychology/Faculty/Kornell/Publications/Schwartz.Son.Kornell.Finn.2011.pdf
And here is one on the role of active learning in reading: McNamara, D. S. (2010). Strategies to read and learn: Overcoming learning by consumption. Medical Education, 44(4), 340–346. doi.org/10.1111/j.1365-2923.2009.03550.x
0:00 An introduction
0:18 Flow experiences and learning
1:37 Happiness and learning
2:12 Frustration and learning
3:14 Tiredness and learning
4:27 A cycle of feelings
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
How visualization helps you learn: youtu.be/eeEfyRYlyYQ
On effective practice: youtu.be/aIPS4ugcanM
REFERENCES
For the classic piece on flow, see: Flow: The Psychology of Optimal Experience by Mihaly Csikszentmihalyi. bookshop.org/p/books/flow-the-psychology-of-optimal-experience-mihaly-csikszentmihalyi/8937185
I pulled some quotes on the flow experience from: Jackson, S. A. (1996). Toward a conceptual understanding of the flow experience in elite athletes. Research quarterly for exercise and sport, 67(1), 76-90.
I also can’t resist mentioning a method paper. This is the one to read to understand the current state of the research and how researchers have gone about defining flow: Abuhamdeh, S. (2020). Investigating the “flow” experience: Key conceptual and operational issues. Frontiers in psychology, 11, 158. frontiersin.org/articles/10.3389/fpsyg.2020.00158/full
The link between rest and deliberate practice comes from Ericsson et al.’s first big paper on deliberate practice, cited below at pages 370-71. AFAIK, it hasn’t been an aspect that’s been deeply explored. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological review, 100(3), 363. psycnet.apa.org/doiLanding?doi=10.1037%2F0033-295X.100.3.363
The relationship between confusion, frustration, and learning is complex - more complex than I alluded to in the video. Ryan Baker and his colleagues have done a lot of work recently exploring this space - here’s a couple of representative pieces:
Richey, J. E., Andres-Bray, J. M. L., Mogessie, M., Scruggs, R., Andres, J. M., Star, J. R., ... & McLaren, B. M. (2019). More confusion and frustration, better learning: The impact of erroneous examples. Computers & Education, 139, 173-190. par.nsf.gov/servlets/purl/10155531 (finding that combined "confusion and frustration" predicted learning)
Richey, J. E., McLaren, B. M., Andres-Bray, M., Mogessie, M., Scruggs, R., Baker, R., & Star, J. (2019). Confrustion in Learning from Erroneous Examples: Does Type of Prompted Self-explanation Make a Difference?. In Artificial Intelligence in Education: 20th International Conference, AIED 2019, Chicago, IL, USA, June 25-29, 2019, Proceedings, Part I 20 (pp. 445-457). Springer International Publishing. (finding that combined "frustration and confusion" predicted lower learning).
On the role of mistakes in learning, check out: Metcalfe, J. (2017). Learning from errors. Annual review of psychology, 68, 465-489. annualreviews.org/doi/full/10.1146/annurev-psych-010416-044022
0:00 An introduction
0:20 The plan
1:08 How I apply the science of learning
3:29 My first diagnostic test
4:20 Reflections after ten hours
5:06 I'm noticing patterns
6:23 How to use practice tests to learn
8:47 Ten hour mark test
9:17 Next steps
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
Here’s the link to the course I’m using: thegreatcourses.com/courses/secrets-of-mental-math
The link to the book I mentioned: bookshop.org/a/91541/9780307338402 (I'm a Bookshop.org affiliate, so get a commission if you purchase through this link).
REFERENCES:
For a readable introduction to interleaving in math from the teacher’s perspective, see: http://pdf.retrievalpractice.org/InterleavingGuide.pdf.
For more background on interleaving (like so many of these principles, it get’s complicated), see: Firth, J., Rivers, I., & Boyle, J. (2021). A systematic review of interleaving as a concept learning strategy. Review of Education, 9(2), 642-684. bera-journals.onlinelibrary.wiley.com/doi/pdfdirect/10.1002/rev3.3266
For more on spaced practice, see: Latimier, A., Peyre, H., & Ramus, F. (2021). A meta-analytic review of the benefit of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33(3), 959-987. http://www.lscp.net/persons/ramus/docs/EPR20.pdf
For evidence supporting the “keep testing learned items” idea, see this classic: Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf
0:00 The problem
1:05 A thought experiment
2:43 Ideas without tests and Dale’s Cone of Experience
4:09 Learning Styles
5:40 Why businesses sell ideas and Baby Einstein
7:23 “Science-based” businesses and brain training
8:37 Supermemo
10:19 What stops good ideas from spreading?
11:15 The role of teacher training
12:13 Good news
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
Two books that I often recommend on learning are: The ABCs of How We Learn by Schwartz, Tsang, and Blair (bookshop.org/a/91541/9780393709261) and Make It Stick by Brown, Roediger, and McDaniel (bookshop.org/a/91541/9780674729018). I'm a Bookshop.org affiliate, so get a small commission if you purchase books through the above links.
Acknowledgements
The green example of Dale’s cone of experience (or learning) comes from commons.wikimedia.org/wiki/File:Edgar_Dale%27s_cone_of_learning.png
The Baby Einstein clip was from this video: youtube.com/watch?v=tHygdh6fCg4
The animated “learning styles” clip comes from: youtube.com/watch?v=_IopcOwfsoU
REFERENCES
A recent summary of the literature on screen time for young children: Guellai, B., Somogyi, E., Esseily, R., & Chopin, A. (2022). Effects of screen exposure on young children’s cognitive development: A review. Frontiers in Psychology, 4779. frontiersin.org/articles/10.3389/fpsyg.2022.923370/full?
On physical activity and development, see: Carson, V., Kuzik, N., Hunter, S., Wiebe, S. A., Spence, J. C., Friedman, A., ... & Hinkley, T. (2015). Systematic review of sedentary behavior and cognitive development in early childhood. Preventive medicine, 78, 115-122. researchgate.net/profile/Alinda-Friedman/publication/280537626_Systematic_Review_of_Sedentary_Behavior_and_Cognitive_Development_in_Early_Childhood/links/5ea46d0292851c1a906f2347/Systematic-Review-of-Sedentary-Behavior-and-Cognitive-Development-in-Early-Childhood.pdf
On brain training, see: Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. (2016). Do “brain-training” programs work?. Psychological Science in the Public Interest, 17(3), 103-186. https://cogsci.northwestern.edu/events/2016-2017-events/simonsEtAl_2016-BrainTraining.pdf.
The example I gave of positive (narrow) results was from one of the early papers in the ACTIVE trial: Ball, K., Berch, D. B., Helmers, K. F., Jobe, J. B., Leveck, M. D., Marsiske, M., ... & ACTIVE Study Group. (2002). Effects of cognitive training interventions with older adults: a randomized controlled trial. Jama, 288(18), 2271-2281.
Supermemo advocates for an expanding spacing schedule, which isn’t necessarily bad. But the question of whether fixed or expanding intervals are superior is complicated. Perhaps it has something to do with how hard the retrieval is or the number of retrieval attempts. But there’s not a clear winner. See these two pieces for further discussion.
Latimier, A., Peyre, H., & Ramus, F. (2021). A meta-analytic review of the benefit of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33(3), 959-987. http://www.lscp.net/persons/ramus/docs/EPR20.pdf
Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological bulletin, 132(3), 354. escholarship.org/content/qt3rr6q10c/qt3rr6q10c.pdf
The 2016 teacher training textbook report I mention, is here: Pomerance, L., Greenberg, J., & Walsh, K. (2016). Learning about Learning: What Every New Teacher Needs to Know. National Council on Teacher Quality. files.eric.ed.gov/fulltext/ED570861.pdf
The paper I showed which was a collaboration between researchers and teachers was: Agarwal, P. K., Bain, P. M., & Chamberlain, R. W. (2012). The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist. Educational Psychology Review, 24(3), 437-448.
0:00 A time machine
01:00 Study secret #1
01:30 Study secret #2
02:16 Study secret #3
05:30 What Robert Bjork from 1979 can teach us
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
REFERENCES
This video was based on a Robert Bjork joint:
Bjork, R. A. (1979). Information‐processing analysis of college teaching. Educational Psychologist, 14(1), 15-23. researchgate.net/profile/Robert-Bjork-2/publication/261581257_Information-Processing_Analysis_of_College_Teaching/links/5d260916458515c11c23fa01/Information-Processing-Analysis-of-College-Teaching.pdf
The sentence rearrangement study comes from: Bradford, E. J. (1975). Retention of subject-generated verbal material (Doctoral dissertation, ProQuest Information & Learning).
For more modern advice from researchers on studying, check out:
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest, 14(1), 4-58.
NOTE: I know nothing about immune systems. Just using as a potential example. Please don't take anything I say about them as gospel.
ACKNOWLEDGEMENTS
Image of Thucydides sculpture from: commons.wikimedia.org/wiki/File:Thucydides_pushkin02.jpg
Image of al-Razi sculpture from: commons.wikimedia.org/wiki/File:Persian_Scholar_pavilion_in_Viena_UN_(Rhazes).jpg
Image of Louis Pasteur: en.wikipedia.org/wiki/File:Louis_Pasteur,_foto_av_Paul_Nadar,_Crisco_edit.jpg
Image of Paul Ehrlich: commons.wikimedia.org/wiki/File:Paul_Ehrlich_1915.jpg
The immune system process image comes from: covalab.com/media/wysiwyg/pictures/pathways/innate_immunity_pathway.jpg
The innate vs adaptive immune system slide comes from: microbenotes.com/differences-between-innate-immunity-and-adaptive-immunity
00:00 Introduction
00:17 The fact
1:11 The look-up table
2:12 Conceptual knowledge - how is a concept different than a fact?
3:30 How we learn concepts
4:12 The benefits of concepts
5:18 Procedural knowledge and skill
5:43 The difference between conceptual knowledge and procedural knowledge
6:27 Unbalanced knowledge
7:30 Bare association
8:18 The ability to recognize something
8:54 I shouldn't talk about this one
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
REFERENCES
On the interaction of procedural and conceptual knowledge, see:
Rittle-Johnson, B., Schneider, M., & Star, J. R. (2015). Not a one-way street: Bidirectional relations between procedural and conceptual knowledge of mathematics. Educational Psychology Review, 27(4), 587-597. https://www.uni-trier.de/fileadmin/fb1/prof/PSY/PAE/Team/Schneider/Rittle-JohnsonEtAl2015.pdf (Math)
Schauble, L. (1996). The development of scientific reasoning in knowledge-rich contexts. Developmental Psychology, 32(1), 102. researchgate.net/profile/Leona-Schauble/publication/232578689_The_development_of_scientific_reasoning_in_knowledge-rich_contexts/links/56eaa8f108ae7858657fde00/The-development-of-scientific-reasoning-in-knowledge-rich-contexts.pdf (Science)
For a more complicated, great take on knowledge in problem solving, see:
De Jong, T., & Ferguson-Hessler, M. G. (1996). Types and qualities of knowledge. Educational psychologist, 31(2), 105-113. https://ris.utwente.nl/ws/files/6401593/types.pdf
For the difference between varieties of recall and recognition, see:
Richardson-Klavehn, A., & Bjork, R. A. (1988). Measures of memory. Annual review of psychology, 39(1), 475-543. researchgate.net/profile/Robert-Bjork-2/publication/234837363_Measures_Of_Memory/links/0deec52efb30acdddd000000/Measures-Of-Memory.pdf
ACKNOWLEDGEMENTS
The gravity demonstration comes from this video: youtube.com/watch?v=MTY1Kje0yLg.
However, there are plenty of good arguments as to why this isn't a terribly helpful analogy: http://newsletter.oapt.ca/files/general-relativiy-analogies.html
00:00 Introduction
00:54 The big misconception
1:54 What makes human memory different from a computer's
3:47 Some benefits and drawbacks to the human memory system
4:39 The key question
If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
How forgetting works: youtu.be/uv7uhI8nsbs
More effective studying: youtu.be/iW7Fp9Mtq1Y
Better note-taking: youtu.be/cRQqH18wJgw
Free recall: youtu.be/FatnXnlwAc4
Exam prep: youtu.be/GoTYyn3WWxI
REFERENCES
The laundry passage comes from this famous 1972 paper, and is often used as a simple example of the power of prior knowledge.
Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of verbal learning and verbal behavior, 11(6), 717-726. http://www.ebbinghaus.ru/wp-content/uploads/2010/02/Bransford__Johnson_1972.pdf
The two-different-senses-of-foot demonstration comes from this paper, where the group who were cued to think of two different senses of the to-be-remembered word remembered it better than those who only were cued to think of one sense of that word.
Gartman, L. M., & Johnson, N. F. (1972). Massed versus distributed repetition of homographs: A test of the differential-encoding hypothesis. Journal of Verbal Learning and Verbal Behavior, 11(6), 801-808. sciencedirect.com/science/article/abs/pii/S0022537172800161
Both of these are also cited in a Robert Bjork paper from 1979. Worth reading. Going to do a full video on this paper soon.
Bjork, R. A. (1979). Information‐processing analysis of college teaching. Educational Psychologist, 14(1), 15-23. researchgate.net/profile/Robert-Bjork-2/publication/261581257_Information-Processing_Analysis_of_College_Teaching/links/5d260916458515c11c23fa01/Information-Processing-Analysis-of-College-Teaching.pdf?_sg%5B0%5D=started_experiment_milestone&origin=journalDetail
If you're interested in witness misidentifications, check out www.innocenceproject.org. Eyewitness misidentifications are the leading cause of wrongful convictions, and there are some pretty simple steps to prevent them from happening: innocenceproject.org/eyewitness-identification-reform
00:00 Introduction
01:17 Our brain’s memory systems
02:38 Justin’s beef with active recall
03:29 Spaced repetition systems, “active recall”, and spaced retrieval practice
04:35 Why flashcard systems kind of suck
05:12 Justin’s recommendation
05:35 What are “desirable difficulties”?
07:03 The alternatives to flashcards
08:35 A good question to ask yourself when studying
10:02 An example study comparing elaborative encoding to retrieval practice
11:25 A true statement
Here are some of my other videos which are relevant to the discussion:
How forgetting works: youtu.be/uv7uhI8nsbs
Free recall demo: youtu.be/FatnXnlwAc4
Questions about free recall answered: youtu.be/evIuDvQDDSY
Example of encoding from reading using organization: youtu.be/WRjsOU6mOp4
What researchers knew in 1979: youtu.be/SOgSpCt3H4M
Advice about learning on your own: youtu.be/3zmek5g3Xxk
Other myths about learning: youtu.be/jcMk3e7u7zM
Desirable difficulties: youtu.be/Q7xnl2tyKwY (my first youtube video!)
Flashcards: youtu.be/ZIGrHI353no
Interleaving: youtu.be/AWTYfzxBwPg
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
To read more about retrieval practice, check out: benjaminkeep.com/the-wicked-effectiveness-of-retrieval-practice
Justin Sung's video on retrieval: youtube.com/watch?v=--Hu2w0s72Y
Justin Sung's video on encoding: youtube.com/watch?v=VcT8puLpNKA
The pro baseball footage is from: youtube.com/watch?v=21TmGchVXRA
REFERENCES
The article I showed in the video comparing elaborative encoding to retrieval practice is:
Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772-775. bit.ly/3S5i9KP
Below, one of the classic pieces on retrieval. It's really an illustration of why you shouldn't just stop trying to remember something just because you have successfully recalled it in the past:
Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf
This is a meta-analysis on the testing effect, which I haven't read in-depth lately, but is worth checking out if you're into this stuff:
Rowland, C. A. (2014). The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. Psychological bulletin, 140(6), 1432. bit.ly/3VwGpII
There's quite a bit of support for the idea that you want to be operating at a "high level" (e.g., synthesizing, applying, critiquing) early in the game. It's not like you need to learn all these rote facts first. A point that Justin makes eloquently in his video on encoding. The piece below is a great discussion of that.
Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning?. Journal of Educational Psychology, 111(2), 189. bit.ly/3T5Va3r
Some have argued that retrieval is only good for simple materials (like remembering new vocabulary words) and not complex materials (like understanding how Newtonian physics works). Although many of the early studies do focus on "simple" materials, there's plenty of studies that establish the effect of retrieval is just as strong for complex materials.
Karpicke, J. D., & Aue, W. R. (2015). The testing effect is alive and well with complex materials. Educational Psychology Review, 27(2), 317-326. link.springer.com/article/10.1007/s10648-015-9309-3
00:00 Introduction
00:44 What is the Delboeuf illusion?
1:36 How the Delboeuf illusion can make us eat more food.
2:48 Was anything I said correct?
4:36 Potential explanations for why we eat more when served larger portions.
6:09 What this video is actually about.
6:49 The meta-analysis strikes back.
You can read more about Ben Goldacre here: en.wikipedia.org/wiki/Ben_Goldacre. I recommend his books and articles.
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:
The original inspiration for this video came from:
Van Ittersum, K., & Wansink, B. (2012). Plate size and color suggestibility: The Delboeuf illusion’s bias on serving and eating behavior. Journal of Consumer Research, 39(2), 215-228. academic.oup.com/jcr/article-pdf/39/2/215/5081845/39-2-215.pdf
A meta-analysis looking at the small plate ➝ lower intake connection. And finding no consistent relationship.
Robinson, E., Nolan, S., Tudur‐Smith, C., Boyland, E. J., Harrold, J. A., Hardman, C. A., & Halford, J. C. (2014). Will smaller plates lead to smaller waists? A systematic review and meta‐analysis of the effect that experimental manipulation of dishware size has on energy consumption. Obesity Reviews, 15(10), 812-821. livrepository.liverpool.ac.uk/3009869/1/Robinson%20et%20al%20Obesity%20Reviews%20AAM.doc
And this piece raises questions about the size of the Delboeuf illusion for hungry people.
Zitron-Emanuel, N., & Ganel, T. (2018). Food deprivation reduces the susceptibility to size-contrast illusions. Appetite, 128, 138-144. sciencedirect.com/science/article/pii/S0195666318300011
How portion size effects caloric intake is still disputed. It seems like there's some anchoring involved (the initial portion suggests what is normal or appropriate) and bite size may play a role as well.
Herman, C. P., Polivy, J., Pliner, P., & Vartanian, L. R. (2015). Mechanisms underlying the portion-size effect. Physiology & Behavior, 144, 129-136. http://www2.psy.unsw.edu.au/Users/lvartanian/Publications/Herman,%20Polivy,%20Pliner,%20&%20Vartanian%20(2015).pdf
Burger, K. S., Fisher, J. O., & Johnson, S. L. (2011). Mechanisms behind the portion size effect: visibility and bite size. Obesity, 19(3), 546-551. onlinelibrary.wiley.com/doi/pdfdirect/10.1038/oby.2010.233
Marchiori, D., Papies, E. K., & Klein, O. (2014). The portion size effect on food intake. An anchoring and adjustment process?. Appetite, 81, 108-115. https://www.academia.edu/download/44357505/The_Portion_Size_Effect_on_Food_Intake_a20160403-22310-r0punj.pdf
Finally, this is my favorite meta-analysis, which DOES, in fact, suggest that smaller plates lead to smaller portion sizes and reduced consumption. It's still a "may" though. The effect seems quite vulnerable to user awareness. So if participants are aware that they're in a food study, researchers don't find an effect. Or if they're aware that they just changed their plates to reduce their consumption, you don't see an effect. But there is at least some suggestion that in the long-run, as your awareness of the changed plates fades, you do end up reducing your consumption.
Holden, S. S., Zlatevska, N., & Dubelaar, C. (2016). Whether smaller plates reduce consumption depends on who’s serving and who’s looking: a meta-analysis. Journal of the Association for Consumer Research, 1(1), 134-146. https://opus.lib.uts.edu.au/bitstream/10453/41448/1/2015003.email.pdf
00:00 Introduction
00:42 How to make specific learning goals
2:03 What are the key skills?
3:21 Gathering learning resources
4:02 A note on scheduling
4:48 Making the best use of feedback
8:12 Watch out for time-sensitive learning goals
10:15 When to make a change
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
REFERENCES
On creating meaningful learning goals, see:
cwsei.ubc.ca/resources/instructor/learninggoals, from the Carl Weiman Science Initiative
To read more about metacognition and illusions of knowing, see:
Avhustiuk, M. M., Pasichnyk, I. D., & Kalamazh, R. V. (2018). The illusion of knowing in metacognitive monitoring: Effects of the type of information and of personal, cognitive, metacognitive, and individual psychological characteristics. Europe's Journal of Psychology, 14(2), 317. ncbi.nlm.nih.gov/pmc/articles/PMC6016031
On spacing, see my Medium article and the references linked therein: bit.ly/3SBcD3v
To read more about Goodhart's Law and other ways that metrics can backfire, I highly recommend:
Muller, J. Z. (2019). The Tyranny of Metrics. Princeton University Press.
CREDITS
My blog post example was www.betterexplained.com - a really good website discussing the intuitions behind many math concepts.
The Youtube channel example was Mathematical Visual Proofs: @MathVisualProofs
The textbook example was "University Physics" - widely regarded as one of the best basic physics books out there. Young, H. D., Freedman, R. A., & Ford, A. L. (2016) Sears and Zemansky's university physics. Pearson education. 14th Edition. There's more recent editions as well.
StackExchange is one of the great places to get answers to questions by experts - especially in programming, engineering, and mathematics. The forum post I showed was from CrossValidated - their statistics arm. stackexchange.com/sites#
The Magic player was Kenji "NumotTheNummy" Egashira, his channel is here: @NumotTheNummyYT
Steph Curry highlights were from: youtube.com/watch?v=7fPcse1phtk.
00:00 Introduction
00:46 Problem Number One: Uranus
1:50 Problem Number Two: Mercury
3:37 Something something provisional nature
4:23 Naive falsification (or disconfirmation)
6:03 Theory-ladenness of facts
What a great story, huh? You can find the basic story on Wikipedia.
But you might want to check out Thomas Levenson's book "The Hunt for Vulcan: ...And how Albert Einstein Destroyed a Planet, Discovered Relativity, and Deciphered the Universe" available here: bookshop.org/a/91541/9780812988307. I am a Bookshop.org affiliate and get a small commission if you purchase through the link.
It's not really the best example of the theory-ladenness of facts, but I feel like it's a good introduction to questions about how science works and how we know what we know.
Thanks for watching.
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
00:00 Introduction
1:25 Does the writing process help us remember what we heard?
1:44 Does reviewing notes later help us remember?
2:07 The missing piece.
2:55 An alternative method.
3:56 Is taking verbatim notes useful?
5:02 On paying attention to the right things.
5:46 Taking notes during vs after a lecture
6:46 My recommended practice
Ryhan Hussain's channel: youtube.com/channel/UCg2u5o5dvS5iIu74rRO5x1w
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:
A lot of the recent research on note-taking has been on whether taking hand-written notes or laptop notes are superior. The evidence for one being better than the other is a bit "meh". As mentioned, the idea that "verbatim notes are bad" seems less true for laptop note-takers.
The opening quote comes from:
Flanigan, A. E., & Titsworth, S. (2020). The impact of digital distraction on lecture note taking and student learning. Instructional Science, 48(5), 495-524. bit.ly/3RiWQWE
The second quote comes from:
Morehead, K., Dunlosky, J., & Rawson, K. A. (2019). How much mightier is the pen than the keyboard for note-taking? A replication and extension of Mueller and Oppenheimer (2014). Educational Psychology Review, 31(3), 753-780. bit.ly/3CYdgzf
Experiment 2, here, is also the one that I relate where "no note-taking" performed just as well as any of the note-taking conditions on immediate tests. It's unclear to me why they didn't also use a delayed measure with the no note-taking group. Also, looking at tables 3 and 4, you can see that if students took notes on material germane to the test, then they did well (my second concluding point).
Another recent piece is here:
Luo, L., Kiewra, K. A., Flanigan, A. E., & Peteranetz, M. S. (2018). Laptop versus longhand note taking: effects on lecture notes and achievement. Instructional Science, 46(6), 947-971. bit.ly/3Qw24xb.
The introduction is a fair summary of the mixed evidence on the encoding hypothesis. Unfortunately, they only used immediate tests here, but there's some interesting material on the relative benefits of hand-written vs laptop notes.
Below is a more direct test of my suggested approach. The results? Note-taking beats testing on immediate tests, but in two weeks testing wins. Testing even beats note-taking + review, suggesting that the storage function of notes is not particularly strong.
Rummer, R., Schweppe, J., Gerst, K., & Wagner, S. (2017). Is testing a more effective learning strategy than note-taking?. Journal of Experimental Psychology: Applied, 23(3), 293. psycnet.apa.org/doiLanding?doi=10.1037%2Fxap0000134
And here is another that finds no difference, even though students who took notes during the lecture took more relevant notes than those who took notes after the lecture.
Haynes, J. M., McCarley, N. G., & Williams, J. L. (2015). An analysis of notes taken during and after a lecture presentation. North American Journal of Psychology, 17(1). bit.ly/3CUqAEL
And another old piece, same thing, no difference (only the measures weren't great).
Eisner, S., & Rohde, K. (1959). Note taking during or after the lecture. Journal of Educational Psychology, 50(6), 301–304. psycnet.apa.org/record/2005-10188-010
Certainly students seem to believe that note-taking during lectures improves learning outcomes:
Morehead, K., Dunlosky, J., Rawson, K. A., Blasiman, R., & Hollis, R. B. (2019). Note-taking habits of 21st century college students: implications for student learning, memory, and achievement. Memory, 27(6), 807-819. bit.ly/3cKGVBg
On the superiority of free recall / retrieval practice / testing, over re-study, see:
Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on psychological science, 1(3), 181-210. https://linguistics.ucla.edu/people/hayes/Teaching/papers/2006_Roediger_Karpicke_Review.pdf
Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf
Karpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of memory and language, 57(2), 151-162. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdf
00:00 Introduction
00:38 Sipping and chugging
1:20 A technique for reading more actively
2:43 A technique for incorporating free recall practice
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
The textbook pages are from "University Physics with Modern Physics" by Young and Freedman. If you're interested in learning physics on your own, check out: susanrigetti.com/physics
References:
My recommendations here are a variety of what's called "spaced retrieval practice". There is a huge literature that spacing out study sessions (e.g., "sipping") provides long-term learning benefits.
Here's a meta-analysis on verbal recall tasks: Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological bulletin, 132(3), 354. escholarship.org/content/qt3rr6q10c/qt3rr6q10c.pdf
And here's a piece using realistic classroom controls. Seabrook, R., Brown, G. D., & Solity, J. E. (2005). Distributed and massed practice: From laboratory to classroom. Applied cognitive psychology, 19(1), 107-122. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.374.3437&rep=rep1&type=pdf
For my intro to this phenomenon, see: bit.ly/3BKHhl2.
My recommendation to do free recall practice prior to coming back to the textbook again comes from evidence on the efficacy of free recall in general (that's the "retrieval"). Here's some good papers on it:
Karpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of memory and language, 57(2), 151-162. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdf
Karpicke, J. D., & Aue, W. R. (2015). The testing effect is alive and well with complex materials. Educational Psychology Review, 27(2), 317-326. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.6502&rep=rep1&type=pdf
For my intro to retrieval practice, see: bit.ly/3djdOFv.
My recommendation on reading the question before seeing the answer (e.g., "active" reading) is based on work in prior knowledge and illusions of knowing.
Example problems tell you things, and there is "a time for telling". Seeing the canonical answer before first think about what the problem is short-circuits the learning process. Here's a few papers on this idea:
Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and instruction, 16(4), 475-5223. http://aaalab.stanford.edu/papers/time_for_telling.pdf
Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of educational psychology, 103(4), 759. http://ece.neu.edu/edsnu/mcgruer/USC/PracticingVersusInventing2011-edu-103-4-759.pdf
There's also interesting work exploring the opposite order. See the paper below for a really good discussion:
Fyfe, E. R., DeCaro, M. S., & Rittle‐Johnson, B. (2014). An alternative time for telling: When conceptual instruction prior to problem solving improves mathematical knowledge. British journal of educational psychology, 84(3), 502-519. https://www.academia.edu/download/39309388/2014_FyfeDeCaroRittleJohnson_BJEP.pdf
00:00 Introduction
00:31 Three advantages to visualizations
2:15 What should you visualize?
2:59 The cardinal sin
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
Image acknowledgements:
The initial three visualizations come from the Dataisbeautiful subreddit. Here are the credits:
Long COVID in the U.S., by MetricT: reddit.com/r/dataisbeautiful/comments/x4hqfs/oc_percent_of_adult_population_currently
Brazil's GDP, by Latinometrics: reddit.com/r/dataisbeautiful/comments/x441dy/oc_in_2011_brazils_gdp_per_capita_was_27k_above
Mandela Effect, by YouGov_Official: reddit.com/r/dataisbeautiful/comments/x3z9ad/oc_americans_are_more_likely_to_hold_false_rather
Jacque Minard's famous illustration of Napoleon's campaign grabbed from http://activehistory.ca/wp-content/uploads/2012/02/Minard.png. Thanks, Active History!
Water cycle image: John Evans and Howard Periman, USGS, Public domain, via Wikimedia Commons. upload.wikimedia.org/wikipedia/commons/1/19/Watercyclesummary.jpg.
John Snow's map of the broad street water pump cholera outbreak (originally from 1854). en.wikipedia.org/wiki/John_Snow#/media/File:Snow-cholera-map-1.jpg.
Causal ecosystem image, from one of Tina Grotzer's papers. Grotzer, T. A., Kamarainen, A. M., Tutwiler, M. S., Metcalf, S., & Dede, C. (2013). Learning to reason about ecosystems dynamics over time: The challenges of an event-based causal focus. BioScience, 63(4), 288-296. academic.oup.com/bioscience/article/63/4/288/253365. She's an academic who does great work on how students think of causal relationships.
The "argument" image comes from a piece by Deanna Kuhn, a pioneer in studying argumentation. Kuhn, D., Goh, W., Iordanou, K., & Shaenfield, D. (2008). Arguing on the computer: A microgenetic study of developing argument skills in a computer‐supported environment. Child development, 79(5), 1310-1328. educationforthinking.org/sites/default/files/pdf/09%20Arguing%20on%20the%20computer.pdf
References:
This video was largely inspired by the chapter "V is for Visualization" in the ABCs of How We Learn by Schwartz, Tsang, and Blair. Definitely recommend it.
00:00 Introduction
00:38 What is instructional scaffolding?
01:28 A second advantage of balance bikes.
01:52 Figuring out the right scaffolding.
This is the video from the Global Cycling Network: youtu.be/UOg1hHmGIQQ
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
Reference
This is a great, thorough piece on scaffolding, that gives a more precise definition and discusses many (many!) issues that I didn't touch on here, but hope to in future videos:
Belland, B. R. (2017). Instructional scaffolding: foundations and evolving definition. In Instructional scaffolding in STEM education (pp. 17-53). Springer, Cham.
link.springer.com/chapter/10.1007/978-3-319-02565-0_2
00:00 Introduction
00:41 Will having a high IQ make you good at chess?
03:12 Does playing chess improve your IQ?
4:15 The influence of training
5:41 The influence of personality and motivation
6:51 The cultural context
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:
The meta-analysis of IQ/cognitive ability with chess skill is here:
Burgoyne, A. P., Sala, G., Gobet, F., Macnamara, B. N., Campitelli, G., & Hambrick, D. Z. (2016). The relationship between cognitive ability and chess skill: A comprehensive meta-analysis. Intelligence, 59, 72-83. eprints.lse.ac.uk/102241/1/INTELL_2016_117_The_Relationship_between_Cognitive_Ability_and_Chess_Skill_a_Comprehensive_Meta_Analysis.pdf
The meta-analysis on whether chess playing increases IQ is here:
Sala, G., & Gobet, F. (2017). Does far transfer exist? Negative evidence from chess, music, and working memory training. Current directions in psychological science, 26(6), 515-520. journals.sagepub.com/doi/full/10.1177/0963721417712760
On the occasional reversal of the relationship between IQ and chess skill, see:
Bilalić, M., McLeod, P., & Gobet, F. (2007). Does chess need intelligence?—A study with young chess players. Intelligence, 35(5), 457-470. bura.brunel.ac.uk/bitstream/2438/642/1/Does%20Chess%20Need%20Intelligence-revision-finalINT.pdf
The meta-analysis on training and chess skill is here:
Charness, N., Tuffiash, M., Krampe, R., Reingold, E., & Vasyukova, E. (2005). The role of deliberate practice in chess expertise. Applied Cognitive Psychology, 19(2), 151-165. https://clinica.ispa.pt/ficheiros/areas_utilizador/user11/11_-_the_role_of_dp_in_chess_expertise.pdf
On chess and personality, check out:
Vollstädt-Klein, S., Grimm, O., Kirsch, P., & Bilalić, M. (2010). Personality of elite male and female chess players and its relation to chess skill. Learning and Individual Differences, 20(5), 517-521. researchgate.net/profile/Merim-Bilalic/publication/220027647_Personality_of_Elite_Male_and_Female_Chess_Players_and_its_Relation_to_Chess_Skill/links/5d49516492851cd046a5a25b/Personality-of-Elite-Male-and-Female-Chess-Players-and-its-Relation-to-Chess-Skill.pdf
The study linking chess skill, deliberate practice, and motivation is here:
De Bruin, A. B., Rikers, R. M. J. P., & Schmidt, H. G. (2007). The influence of achievement motivation and chess-specific motivation on deliberate practice. Journal of Sport and Exercise Psychology, 29(5), 561. researchgate.net/profile/Anique-De-Bruin/publication/271075262_The_Influence_of_Achievement_Motivation_and_Chess-Specific_Motivation_on_Deliberate_Practice/links/00b7d539064ca2cf43000000/The-Influence-of-Achievement-Motivation-and-Chess-Specific-Motivation-on-Deliberate-Practice.pdf
On the "domain-specificity" of expertise, the best resource is probably:
Ericsson, K. A., Hoffman, R. R., Kozbelt, A., & Williams, A. M. (Eds.). (2018). The Cambridge handbook of expertise and expert performance. Cambridge University Press.
But that's the best resource for a lot of the expertise literature! If you can't get your hands on it, here's an interesting article talking about how creativity (something that many people assume is domain-general) is actually quite domain-specific:
Baer, J. (2015). The importance of domain-specific expertise in creativity. Roeper Review, 37(3), 165-178. https://www.academia.edu/download/51367773/The_Importance_of_Domain_Specific_Expertise_in_Creativity.pdf
00:00 Lesson one - four stories about attention
2:21 One way of thinking about attention and learning
3:07 Lesson two - what controls attention?
4:06 Lesson three - the roles of students and teachers
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
The artwork I mentioned in the example comes from:
Cut with the Kitchen Knife Dada Through the Last Weimar Beer-Belly Cultural Epoch of Germany by Hannah Höch (1919)
The Elephant Celebes by Max Ernst (1921)
Das Undbild by Kurt Schwitters (1919)
The complex scientific figure is kind of a cheat: no one would use that as a powerpoint slide. It's more of an attempt to make a complete map of metabolic pathways in human cells. I found it here: https://faculty.cc.gatech.edu/~turk/bio_sim/articles/metabolic_pathways.png
References:
The "someone else with a laptop distracts you" study is here:
Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24-31.https://www.sciencedirect.com/science/article/pii/S0360131512002254
A good review of internal/external focus in motor learning is here:
Wulf, G. (2013). Attentional focus and motor learning: a review of 15 years. International Review of sport and Exercise psychology, 6(1), 77-104. http://gwulf.faculty.unlv.edu/wp-content/uploads/2018/11/Wulf_AF_review_2013.pdf
Seductive details is a large research topic, but here's a recent meta-analysis that tries to synthesize the existing studies:
Sundararajan, N., & Adesope, O. (2020). Keep it coherent: A meta-analysis of the seductive details effect. Educational Psychology Review, 32(3), 707-734. link.springer.com/article/10.1007/s10648-020-09522-4
The "thinking 'are these the same or different'" example I drew from the literature on blocked and interleaved practice, which suggests that paying attention to similarities and paying attention to differences offers different affordances in category learning. See pages 2-3 in the following piece.
Carvalho, P. F., & Goldstone, R. L. (2014). Putting category learning in order: Category structure and temporal arrangement affect the benefit of interleaved over blocked study. Memory & cognition, 42(3), 481-495. https://pcl.sitehost.iu.edu/papers/blockedinterleaved.pdf
Good advice on reducing extraneous cognitive load in undergraduate science courses comes is here (with a couple of other good references in there):
cwsei.ubc.ca/sites/default/files/cwsei/resources/literature/Cognitive-Load_CWSEI.pdf
Some interesting research on cell phones and attention is here:
Thornton, B., Faires, A., Robbins, M., & Rollins, E. (2014). The mere presence of a cell phone may be distracting: Implications for attention and task performance. Social Psychology, 45(6), 479. oarklibrary.com/file/2/0c907a59-76b9-482a-b4a3-1c66294d7048/8d49e16c-14dc-4877-9c64-e75ba9bfdcc2.pdf
On manipulating student attention towards (or away from) deep structure, see (this is also where the clown image comes from):
Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of educational psychology, 103(4), 759. http://ece.neu.edu/edsnu/mcgruer/USC/PracticingVersusInventing2011-edu-103-4-759.pdf
00:00 Step #1
1:31 Step #2
3:00 Step #3
5:04 Tip #1: How to use visualizations
5:40 Tip #2: On study groups
6:26 Tip #3: Be reasonable during step 3
7:03 Open vs. closed book exams
7:36 Multiple choice vs essay tests
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
My advice here is based on a few lines of literature.
Steps 1 and 3, which I recommend spending the most time on, comes from the research on free recall and the testing effect. These are just better study than rereading or re-writing your notes. Typical research findings are here:
Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772-775. mrbartonmaths.com/resourcesnew/8.%20Research/Memory%20and%20Revision/Retrieval%20practice%20more%20effective%20than%20studying.pdf
Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968.
http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf
Karpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57(2), 151-162.
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdf
Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin. (currently available at gwern.net/docs/spacedrepetition/2021-yang.pdf)
An applied research piece exploring the application of tests in the classroom. They tested both short-answer and multiple-choice tests, finding that short answer tests outperformed multiple choice tests (though both outperformed additional reading).
McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European journal of cognitive psychology, 19(4-5), 494-513. (currently available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1061.366&rep=rep1&type=pdf)
A classic study demonstrating long-term benefits (1 month).
Butler, A. C., & Roediger III, H. L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19(4-5), 514-527. (currently available at http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Butler-Roediger-2007_EJCP.pdf)
My advice on the "outstanding questions" box is based on the literature on self-questioning and interrogation. It’s also supposed to help you with metacognitive knowledge (which helps you study more effectively because you know what you already understand and what you’re still struggling with). Example research findings are here:
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest, 14(1), 4-58. https://bibliotecadigital.mineduc.cl/bitstream/handle/20.500.12365/17388/dunloskyimprovinglearning.pdf?sequence=1 (The first two sections on elaborative interrogation and self-explanations are relevant here.)
Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of educational research, 66(2), 181-221. jgregorymcverry.com/readings/rosenshine1996StudentQuestioning.pdf (In most of these studies people are teaching students how to ask questions, which is an important part of the process.)
King, A. (1992). Comparison of self-questioning, summarizing, and notetaking-review as strategies for learning from lectures. American Educational Research Journal, 29(2), 303-323. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.859.5957&rep=rep1&type=pdf
My advice on using visualizations comes from research on learning and visualizations (well summarized in The ABCs of Learning, citation below). Basically, they offload cognitive burden, they can be more specific than words (relating ideas and concepts in more precise ways), and they can help you identify patterns that you might not have recognized before.
Schwartz, D. L., Tsang, J. M., & Blair, K. P. (2016). The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them. WW Norton & Company. (I’m referring to “V is for Visualization”. Great introduction to the research on learning more generally.)
There are other ways of studying for tests, of course. But I think this is a pretty solid default method.
00:00 Introduction
00:28 Two learning systems
01:02 One source of intuition
01:42 Another source of intuition
02:46 Can you trust your intuition? Should you?
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
00:00 Introduction
00:26 What are force diagrams?
00:48 The study and its results
01:39 Explanation one
01:57 Explanation two
02:42 Why draw a diagram?
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:
Kuo, E., Hallinen, N. R., & Conlin, L. D. (2017). When procedures discourage insight: epistemological consequences of prompting novice physics students to construct force diagrams. International Journal of Science Education, 39(7), 814-839. tandfonline.com/doi/abs/10.1080/09500693.2017.1308037
Heckler, A. F. (2010). Some consequences of prompting novice physics students to construct force diagrams. International Journal of Science Education, 32(14), 1829-1851.
researchgate.net/profile/Andrew-Heckler/publication/233366500_Some_Consequences_of_Prompting_Novice_Physics_Students_to_Construct_Force_Diagrams/links/0deec53220b7f7ae2c000000/Some-Consequences-of-Prompting-Novice-Physics-Students-to-Construct-Force-Diagrams.pdf
Kuo, E., Hallinen, N. R., & Conlin, L. D. (2015, December). How prompting force diagrams discourages student use of adaptive problem-solving shortcuts. In The Physics Education Research Conference 2015.
researchgate.net/profile/Eric-Kuo/publication/299701476_How_prompting_force_diagrams_discourages_student_use_of_adaptive_problem-solving_shortcuts/links/57e3f25d08ae8d5908c162d1/How-prompting-force-diagrams-discourages-student-use-of-adaptive-problem-solving-shortcuts.pdf
00:00 Introduction
00:24 The early experiments
1:46 A brain superpower
3:12 Depth of search
4:33 Human experts vs AI chess programs
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:
My summarization of the memory results is really recounting this recent study:
Smith, E. T., Bartlett, J. C., Krawczyk, D. C., & Basak, C. (2021). Are the advantages of chess expertise on visuo-spatial working-memory capacity domain specific or domain general?. Memory & Cognition, 49(8), 1600-1616.https://www.researchgate.net/profile/Chandramallika-Basak-2/publication/351245327_Are_the_Advantages_of_Chess_Expertise_on_Visuo-Spatial_Working_Memory_Capacity_Domain_Specific_or_Domain_General/links/609036d0a6fdccaebd073806/Are-the-Advantages-of-Chess-Expertise-on-Visuo-Spatial-Working-Memory-Capacity-Domain-Specific-or-Domain-General.pdf
But there's lots of other research along the same lines.
The earliest research was here:
de Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton Publishers.
Chase, W. G., & Simon, H. A. (1973). The mind's eye in chess. In Visual information processing (pp. 215-281). Academic Press. sciencedirect.com/science/article/pii/B9780121701505500111
And here are some other solid articles I looked at:
Charness, N. (1992). The impact of chess research on cognitive science. Psychological research, 54(1), 4-9. link.springer.com/article/10.1007/BF01359217
Bilalić, M., McLeod, P., & Gobet, F. (2008). Expert and “novice” problem solving strategies in chess: Sixty years of citing de Groot (1946). Thinking & Reasoning, 14(4), 395-408. researchgate.net/profile/Fernand-Gobet/publication/49401082_Expert_and_novice_problem_solving_strategies_in_chess_Sixty_years_of_citing_de_Groot_1946/links/57651cbb08ae1658e2f46b20/Expert-and-novice-problem-solving-strategies-in-chess-Sixty-years-of-citing-de-Groot-1946.pdf?_sg%5B0%5D=started_experiment_milestone&origin=journalDetail
Connors, M. H., Burns, B. D., & Campitelli, G. (2011). Expertise in complex decision making: The role of search in chess 70 years after de Groot. Cognitive science, 35(8), 1567-1579. onlinelibrary.wiley.com/doi/full/10.1111/j.1551-6709.2011.01196.x
00:00 Introduction
00:24 Deduction
00:57 Do deductions move from the general to the particular?
01:39 The key characteristic of deductions
02:21 Induction
02:45 Do inductions move from the particular to the general?
03:21 The key characteristic of inductions
04:26 A special kind of induction
05:35 Semantics?
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
Reference:
The best discussion of the distinction between the three kinds of reasoning I outlined here comes from "How We Reason" by Philip Johnson-Laird - cognitive scientist extraordinaire. Chapter 13 is on Sherlock and abductions. It's his model that I'm describing.
The clips are from the BBC series "Sherlock," starring Benedict Cumberbatch. Used here for educational purposes.
00:00 Introduction
00:21 Three stories
01:32 The catch
01:56 The beginnings of an explanation
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:
I initially came across the "mirror learning" example on page 310 of this book:
Magill, R., & Anderson, D. (2010). Motor learning and control. New York: McGraw-Hill Publishing. https://www.academia.edu/download/62953224/Motor_Learning_and_Control__Concepts_and_-_Anderson__David20200414-28877-18hp4j2.pdf
This reference has a fuller description of the idea (though is testing a slightly different hypothesis):
Tremblay, L., & Proteau, L. (1998). Specificity of practice: The case of powerlifting. Research quarterly for exercise and sport, 69(3), 284-289. researchgate.net/profile/Luc-Tremblay-2/publication/13507905_Specificity_of_Practice_The_Case_of_Powerlifting/links/5805b11608aef179365e73b4/Specificity-of-Practice-The-Case-of-Powerlifting.pdf
Note that learning with a mirror doesn't always impair performance without a mirror (or improve skill acquisition during practice). (See for instance: Lynch, J. A., Chalmers, G. R., Knutzen, K. M., & Martin, L. T. (2009). Effect on performance of learning a Pilates skill with or without a mirror. Journal of Bodywork and Movement Therapies, 13(3), 283-290. sciencedirect.com/science/article/abs/pii/S1360859208001666)
The math feedback example comes from this study:
Fyfe, E. R., & Rittle-Johnson, B. (2017). Mathematics practice without feedback: A desirable difficulty in a classroom setting. Instructional Science, 45(2), 177-194. link.springer.com/article/10.1007/s11251-016-9401-1
The Spanish word study is at:
Bahrick, H. P. (1979). Maintenance of knowledge: Questions about memory we forgot to ask. Journal of Experimental Psychology: General, 108(3), 296. psycnet.apa.org/record/1981-00448-001
But I'm basing my discussion on the description of the study as made by Robert Bjork, here:
Bjork, R. A. (1994). Memory and metamemory considerations in the. Metacognition: Knowing about knowing, 185(7.2). It's really such a great article; one that I've revisited many times. Get in touch if you want a pdf copy.
00:00 Introduction
00:30 Memories do battle with each other
1:55 A second way that memories interfere with one another
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:
A great discussion of forgetting (especially theories of cue competition) comes from:
Anderson, M. C., & Neely, J. H. (1996). Interference and inhibition in memory retrieval. In Memory (pp. 237-313). Academic Press. http://www.memorycontrol.net/an1996.pdf
The "drinking helps you remember" idea has a relatively long history.
Here's an earlier replication of some of the first work:
Mueller, C. W., Lisman, S. A., & Spear, N. E. (1983). Alcohol enhancement of human memory: Tests of consolidation and interference hypotheses. Psychopharmacology, 80(3), 226-230. https://www.academia.edu/download/46474229/bf0043615820160614-5333-a2occx.pdf
And here's a more recent piece:
Moulton, P. L., Petros, T. V., Apostal, K. J., Park II, R. V., Ronning, E. A., King, B. M., & Penland, J. G. (2005). Alcohol-induced impairment and enhancement of memory: A test of the interference theory. Physiology & behavior, 85(3), 240-245. pubag.nal.usda.gov/download/46671/pdf
The same is true, by the way of other drugs that impair memory formation. They tend to facilitate memory for events and information that happened prior to impairment.
00:00 Introduction
00:35 Do we know how to highlight?
01:39 What does it encourage us to do?
02:08 What does it tell us about what we know?
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
The onscreen quotes were pulled from a classic review of study techniques:
Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public interest, 14(1), 4-58. https://bibliotecadigital.mineduc.cl/bitstream/handle/20.500.12365/17388/dunloskyimprovinglearning.pdf?sequence=1
00:00 Who are you working for?
00:50 Effort over the long run.
01:12 How should we think about mistakes and confusion?
02:13 Spending your time more effectively.
03:12 The pretty principle.
Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com