A Simple Way to Learn Complex Skills @benjaminkeep

Benjamin Keep, PhD, JD Sometimes we have to learn very complex skills that can't be broken apart into more basic skills. What do we do then?

0:00 An intro to the experiment
0:21 Skills with many interdependent parts
1:00 The study that explored learning complex skills
2:33 The results
3:48 Try it out

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

*There was a wrinkle to this study that I didn't mention, which is that groups 2, 3, and 4 all had a different kind of display on the game for the first six sessions, which gave them slightly different information than the standard game. The idea was to support learner's focus on part of the game, but it also meant that there's more than one difference between the control and the other groups. The enhanced display could explain some of the difference between the control group and the other groups, but doesn't explain the differences between groups 2 and 3 and group 4. And the variable-priority approach (what I call rotating focus) that the study helped establish is well-supported by other studies as well - see the meta-analysis in the reference section for more support.

ACKNOWLEDGMENTS

Starcraft clip from LowkoTV: youtube.com/watch?v=kz9M2uYcFyE

Space Fortress mobile game footage from: play.google.com/store/apps/details?id=com.mgmamac.space_fortress&gl=US

Space Fortress DOS game footage from: classicreload.com/space-fortress.html

You can play an updated version of the original Space Fortress game at brainturk.com/space-fortress

REFERENCES

This video is based on the paper below. I replicated their figure 2 starting at around 2:40. Gopher, D., Weil, M., & Siegel, D. (1989). Practice under changing priorities: An approach to the training of complex skills. Acta Psychologica, 71(1–3), 147–177. doi.org/10.1016/0001-6918(89)90007-3

Here is a meta-analysis illustrating the efficacy of what I call "rotating focus" but what is called "variable-priority" training in the literature. Wickens, C. D., Hutchins, S., Carolan, T., & Cumming, J. (2013). Effectiveness of Part-Task Training and Increasing-Difficulty Training Strategies: A Meta-Analysis Approach. Human Factors, 55(2), 461–470. doi.org/10.1177/0018720812451994

And if you search more articles from Daniel Gopher, you will see other works on the same theme. Like this one: Brickner, M., & Gopher, D. (1981). Improving Time-Sharing Performance by Enhancing Voluntary Control on Processing Resources. TECHNION-ISRAEL INST OF TECH HAIFA CENTER OF HUMAN ENGINEERING AND INDUSTRIAL SAFETY RESEARCH. https://apps.dtic.mil/sti/pdfs/ADA118558.pdf

updated 1 year ago

A Simple Way to Learn Complex Skills

Benjamin Keep, PhD, JD 2023-03-15 | Sometimes we have to learn very complex skills that can't be broken apart into more basic skills. What do we do then?

0:00 An intro to the experiment
0:21 Skills with many interdependent parts
1:00 The study that explored learning complex skills
2:33 The results
3:48 Try it out

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

*There was a wrinkle to this study that I didn't mention, which is that groups 2, 3, and 4 all had a different kind of display on the game for the first six sessions, which gave them slightly different information than the standard game. The idea was to support learner's focus on part of the game, but it also meant that there's more than one difference between the control and the other groups. The enhanced display could explain some of the difference between the control group and the other groups, but doesn't explain the differences between groups 2 and 3 and group 4. And the variable-priority approach (what I call rotating focus) that the study helped establish is well-supported by other studies as well - see the meta-analysis in the reference section for more support.

ACKNOWLEDGMENTS

Starcraft clip from LowkoTV: youtube.com/watch?v=kz9M2uYcFyE

Space Fortress mobile game footage from: play.google.com/store/apps/details?id=com.mgmamac.space_fortress&gl=US

Space Fortress DOS game footage from: classicreload.com/space-fortress.html

You can play an updated version of the original Space Fortress game at brainturk.com/space-fortress

REFERENCES

This video is based on the paper below. I replicated their figure 2 starting at around 2:40. Gopher, D., Weil, M., & Siegel, D. (1989). Practice under changing priorities: An approach to the training of complex skills. Acta Psychologica, 71(1–3), 147–177. doi.org/10.1016/0001-6918(89)90007-3

Here is a meta-analysis illustrating the efficacy of what I call "rotating focus" but what is called "variable-priority" training in the literature. Wickens, C. D., Hutchins, S., Carolan, T., & Cumming, J. (2013). Effectiveness of Part-Task Training and Increasing-Difficulty Training Strategies: A Meta-Analysis Approach. Human Factors, 55(2), 461–470. doi.org/10.1177/0018720812451994

And if you search more articles from Daniel Gopher, you will see other works on the same theme. Like this one: Brickner, M., & Gopher, D. (1981). Improving Time-Sharing Performance by Enhancing Voluntary Control on Processing Resources. TECHNION-ISRAEL INST OF TECH HAIFA CENTER OF HUMAN ENGINEERING AND INDUSTRIAL SAFETY RESEARCH. https://apps.dtic.mil/sti/pdfs/ADA118558.pdf

Online Learning Can Be Awesome. Heres How.

Benjamin Keep, PhD, JD 2024-07-31 | There are a lot of wonderful learning resources on the internet. How should we go about using them?

GiveInternet.org Donation Link: giveinternet.org/BenjaminKeep

00:00 Introduction
00:45 Finding the right learning resources
2:04 Getting practice
4:02 Finding a learning community
5:08 Planning your practice
5:54 What happens next?
8:18 A worthy cause

For updates on my course and website, click here: https://forms.gle/9TEUw7bj3YTs7B4H6

To sign up to my (infrequent) newsletter, click here: http://www.benjaminkeep.com.

Edited by Presage: pre-sage.com

Resources featured in the video:

Kalid Azad's Better Explained: betterexplained.com
Paul's Online Math Notes: https://tutorial.math.lamar.edu/
Susan Rigetti's guide to learning physics: susanrigetti.com/physics
Andy Guitar: andyguitar.co.uk
SimplyPiano: hellosimply.com/simply-piano

Tenenbaum, M., & Pollard, H. (1985). Ordinary differential equations: an elementary textbook for students of mathematics, engineering, and the sciences. Courier Corporation. archive.org/details/TenenbaumAndPollardOrdinaryDifferentialEquations

Young, H. D., Freedman, R. A., Ford, A. L., Sears, F. W., & Zemansky, M. W. (2014). Sears and Zemansky's University Physics with Modern Physics.

Why Cant People Solve These Simple Problems?

Benjamin Keep, PhD, JD 2024-07-10 | The cognitive reflection test was meant to measure one thing: the tendency for someone to stop and think before giving a quick answer. The simplicity of the test made it a widely used measure for critical thinking. But some clever experiments illustrate that the CRT is not so simple after all.

00:00 Introduction
00:19 The CRT problems
1:20 The structure of CRT problems
3:04 Some hints about what's going wrong
4:53 The structure of insight problems
6:27 Explanation Number One
7:57 Explanation Number Two
8:57 Explanation Number Three

I'm launching a membership site. If you want in, go here: https://forms.gle/JoMA1Xa5Nnnwm8VU9

Recommended reading: benjaminkeep.com/recommended-reading

Sign up to my free (infrequently published) newsletter here: benjaminkeep.com

REFERENCES

This video is mostly based off of Patel, N., Baker, S. G., & Scherer, L. D. (2019). Evaluating the cognitive reflection test as a measure of intuition/reflection, numeracy, and insight problem solving, and the implications for understanding real-world judgments and beliefs. Journal of Experimental Psychology: General, 148(12), 2129–2153. doi.org/10.1037/xge0000592

This is the (classic) paper that introduced the CRT as a measure of reflection. Frederick, S. (2005). Cognitive reflection and decision making. The Journal of Economic Perspectives, 19, 25– 42. http://dx.doi.org/10.1257 089533005775196732. It also includes correlations between the math section of the SAT and the CRT (.46) vs the verbal section of the SAT and the CRT (.24).

On the CRT’s correlation to tests measuring numeracy being about .5, see Cokely, E. T., Galesic, M., Schulz, E., Ghazal, S., & Garcia-Retamero, R. (2012). Measuring Risk Literacy: The Berlin Numeracy Test. Judgment and Decision Making, 7(1), 25–47. doi.org/10.1017/S1930297500001819

On the associations between reflection/analytical thinking and various beliefs, see Pennycook, G., Fugelsang, J. A., & Koehler, D. J. (2015). Everyday Consequences of Analytic Thinking. Current Directions in Psychological Science, 24(6), 425-432. doi.org/10.1177/0963721415604610

This mentions some of the earlier results correlating CRT performance to rational thinking, but also extends the CRT to a seven-question test. The four added questions are probably even more reliant on numerical ability than the first three. Toplak, M. E., West, R. F., & Stanovich, K. E. (2011). The Cognitive Reflection Test as a predictor of performance on heuristics-and-biases tasks. Memory & cognition, 39(7), 1275-1289.

Everyone misses this problem solving step

Benjamin Keep, PhD, JD 2024-06-26 | There's lots of good advice on problem solving. But the details matter.

00:00 Introduction
1:08 The problem solving process
1:54 The important sentence
3:02 Skipping the step vs doing the thing

BTW, if you're learning physics on your own, I think Susan Rigetti's recommendations are indispensable. susanrigetti.com/physics

The "2000" problems video I reference is here: youtu.be/_5MQUMtRPmM

References:

The physics book in question is: Young, H. D., Freedman, R. A., & Ford, A. L. (20). Sears and Zemansky's University Physics with Modern Physics. 15th Edition. Pearson education. Although I think any of the earlier editions (14th, 13th, international, etc.) are good. Again, credit to Susan Rigetti for this recommendation.

The papers I referenced on conceptual knowledge and procedural skill:

Kim, E., & Pak, S. J. (2002). Students do not overcome conceptual difficulties after solving 1000 traditional problems. American Journal of Physics, 70(7), 759-765. (available at http://sdsu-physics.org/sdsu_per/articles/ProblemS_ConUnderst.pdf)

Byun, T., & Lee, G. (2014). Why students still can't solve physics problems after solving over 2000 problems. American Journal of Physics, 82(9), 906-913. (available at aapt.scitation.org/doi/abs/10.1119/1.4881606)

Learning scientist learns how to cook

Benjamin Keep, PhD, JD 2024-06-12 | A lot of the research on learning focuses on learning in the classroom. But what about all the other stuff we need to learn? Here are some thoughts on learning how to cook.

00:00 Introduction
0:29 Cognitive load theory
3:17 Retrieval practice
4:17 Skill analysis
6:12 Deliberate practice
8:25 Leveraging prior knowledge

Edited by Presage: https://pre-sage.com.

Sign up to my newsletter here: http://www.benjaminkeep.com.

What makes something memorable?

Benjamin Keep, PhD, JD 2024-05-29 | This is my "everything I know about encoding" video. Well, not quite everything. But best I could do in 45 minutes.

00:00 Introduction
00:46 What can we encode?
1:46 Two issues in encoding research
4:43 Depth of processing
9:17 The generation effect
13:31 Imagery
17:28 Memory palaces and “time palaces”
20:10 The drawing effect
23:36 Distinctiveness
27:50 Value
31:42 Integration
37:17 Transfer appropriate processing
43:32 The complete list of themes

To learn more about the membership site, click here: https://forms.gle/9TEUw7bj3YTs7B4H6

Edited by Presage: https://pre-sage.com.

References:

Perhaps this goes without saying, but each of the issues that I mention are considerably more nuanced than how I've presented them here. But hopefully the video gives you the basic ideas so that you can dive in further, if you want.

Depth of processing: Craik, F. I., & Tulving, E. (1975). Depth of processing and the retention of words in episodic memory. Journal of experimental Psychology: general, 104(3), 268.

Generation: Bertsch, S., Pesta, B. J., Wiscott, R., & McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & cognition, 35, 201-210.

Imagery: Leopold, C., & Mayer, R. E. (2015). An imagination effect in learning from scientific text. Journal of Educational Psychology, 107(1), 47–63. doi.org/10.1037/a0037142

"Time palaces" and memory palaces Bouffard, N., Stokes, J., Kramer, H. J., & Ekstrom, A. D. (2018). Temporal encoding strategies result in boosts to final free recall performance comparable to spatial ones. Memory & Cognition, 46(1), 17–31. doi.org/10.3758/s13421-017-0742-z

Value and encoding: Hennessee, J. P., Patterson, T. K., Castel, A. D., & Knowlton, B. J. (2019). Forget me not: Encoding processes in value-directed remembering. Journal of Memory and Language, 106, 29–39. doi.org/10.1016/j.jml.2019.02.001

Drawing: Fernandes, M. A., Wammes, J. D., & Meade, M. E. (2018). The Surprisingly Powerful Influence of Drawing on Memory. Current Directions in Psychological Science, 27(5), 302–308. doi.org/10.1177/0963721418755385

Wammes, J. D., Meade, M. E., & Fernandes, M. A. (2016). The drawing effect: Evidence for reliable and robust memory benefits in free recall. Quarterly Journal of Experimental Psychology, 69(9), 1752–1776. doi.org/10.1080/17470218.2015.1094494

Acting things out: Engelkamp, J., & Zimmer, H. D. (1997). Sensory factors in memory for subject-performed tasks. Acta Psychologica, 96(1-2), 43-60.

Production and distinctiveness: Ozubko, J. D., & MacLeod, C. M. (2010). The production effect in memory: evidence that distinctiveness underlies the benefit. Journal of Experimental Psychology: Learning, Memory, and Cognition, 36(6), 1543.

Integration and self-explanations: Chi, M. T., Bassok, M., Lewis, M. W., Reimann, P., & Glaser, R. (1989). Self-explanations: How students study and use examples in learning to solve problems. Cognitive Science, 13(2), 145–182.

Renkl, A. (2002). Worked-out examples: Instructional explanations support learning by self-explanations. Learning and instruction, 12(5), 529-556.

Integration and video games/passages:

Arena, D. A., & Schwartz, D. L. (2014). Experience and explanation: Using videogames to prepare students for formal instruction in statistics. Journal of Science Education and Technology, 23, 538-548.

Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and Instruction, 16(4), 475–5223.

Transfer appropriate processing: Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16(5), 519–533. doi.org/10.1016/S0022-5371(77)80016-9

The Essentials of Problem Solving

Benjamin Keep, PhD, JD 2024-01-17 | An introduction to the psychology of problem solving. Featured problems: the towers of Hanoi, the Chinese ring puzzle, the Wason 4-card selection task, the candle problem, Roman matchstick problems, and toothpick shape problems.

00:00 A quick note
00:47 The problem state space and the towers of Hanoi
4:45 Problems of representation and the Chinese ring puzzle
6:42 Context and variations of the Wason 4-card selection task
9:42 Introduction to insight problems: the candle problem
11:05 Differences between insight and incremental problems
12:15 Barriers to insight: Roman matchstick problems
17:30 Insight problems: too big of a distinction?
19:08 Well-structured and ill-structured problems
21:11 Representation and argument
23:34 Becoming a better problem solver: toothpick problems
26:45 Domain-specific knowledge and strategy change
30:55 What transfers across problem-solving domains?

Sign up to my newsletter here: http://www.benjaminkeep.com.

ACKNOWLEDGEMENTS

The towers of Hanoi footage comes from Math Playground's version of the towers.

The Chinese ring puzzle footage comes from Youtuber @Jojikiba, here youtube.com/watch?v=nydIl2VO12I

The candle problem visuals: youtube.com/watch?v=PEvThYxtnFQ. If you made it or you know who did, please let me know. The link above seems to be the oldest and most complete version.

Paul Zeitz photo: imo-official.org/advisory.aspx?year=2019

Math and STEP problems: imo-official.org/problems/IMO2022SL.pdf and usmle.org/sites/default/files/2021-10/Step_1_Sample_Items.pdf, respectively.

Physics problems: Badeau, R., White, D. R., Ibrahim, B., Ding, L., & Heckler, A. F. (2017). What works with worked examples: Extending self-explanation and analogical comparison to synthesis problems. Physical Review Physics Education Research, 13(2), 020112. doi.org/10.1103/PhysRevPhysEducRes.13.020112

Visit www.eternagame.org for more info on Eterna.

The Chess explanation clip: @BetterChessTraining.

The Starcraft 2 clip: an @ESChamp video, here: youtube.com/watch?v=4okBbHaCQ98

The go examples come from Sensei’s library: senseis.xmp.net/?CornersThenSidesThenCenter

Starcraft 2 map: news.blizzard.com/en-us/starcraft2/23223597/new-ladder-maps-for-2019-season-4

Parity example: artofproblemsolving.com/wiki/index.php?title=Parity

The free body diagram is from a Math and Science video: youtube.com/watch?v=mI11vHsRnC8

The map of Easter Island: en.wikipedia.org/wiki/File:Easter_Island_map-en.svg

The cashflow statement: en.wikipedia.org/wiki/Cash_flow_statement#cite_note-21, but is originally from Epstein, Barry J.; Eva K. Jermakowicz (2007). Interpretation and Application of International Financial Reporting Standards. John Wiley & Sons. pp. 91–97. ISBN 978-0-471-79823-1.

REFERENCES

Simon, H. A., & Newell, A. (1971). Human problem solving: The state of the theory in 1970. American psychologist, 26(2), 145.

The tower of Hanoi state space was adapted from Zhang, J., & Norman, D. A. (1994). Representations in distributed cognitive tasks. Cognitive science, 18(1), 87-122.

The Chinese Ring puzzle state space (for 5 rings) is adapted from Kotovsky, K. & Simon, H. A. (1990). What Makes Some Problems Really hard: Explorations in the Problem Space of Difficulty. Cognitive Psychology, 22(2), 143–183.

The “underage drinking” example of the Wason 4-card task comes from this classic:
Griggs, R. A., & Cox, J. R. (1982). The elusive thematic‐materials effect in Wason's selection task. British journal of psychology, 73(3), 407-420.

A short meta-analysis on the Wason 4-card task: Ragni, M., Kola, I., & Johnson-Laird, P. (2017). The Wason Selection task: A Meta-Analysis. In CogSci. researchgate.net/profile/Ilir-Kola/publication/322682384_The_Wason_Selection_Task_A_Meta-Analysis/links/5a688e6f4585156abdffd541/The-Wason-Selection-Task-A-Meta-Analysis.pdf

On the insight experience: Webb, M. E., Little, D. R., & Cropper, S. J. (2016). Insight Is Not in the Problem: Investigating Insight in Problem Solving across Task Types. Frontiers in Psychology, 7. frontiersin.org/articles/10.3389/fpsyg.2016.01424

The Matchstick Roman numeral problems and the discussion of constraints and chunks come from: Öllinger, M., Jones, G., & Knoblich, G. (2008). Investigating the Effect of Mental Set on Insight Problem Solving. Experimental Psychology, 55(4), 269–282. doi.org/10.1027/1618-3169.55.4.269

On strategic change, see: Siegler, R. S. (2002). Microgenetic studies of self-explanation. Microdevelopment: Transition processes in development and learning, 31, 58.

On the importance of representations in ill-structured domains, see: Chandrasekharan, S., & Nersessian, N. J. (2011). Building cognition: the construction of external representations for discovery. In Proceedings of the Annual Meeting of the Cognitive Science Society (Vol. 33, No. 33).

Learning Foreign Language Vocabulary - The Fundamentals

Benjamin Keep, PhD, JD 2023-12-07 | Some thoughts on learning foreign language vocabulary.

The “Chinese is Hard” podcast: open.spotify.com/show/6DdGLH8neJoAkggOZu4VrN

William Hart’s Youtube channel: youtube.com/@will3267

Danyo Pang’s Youtube channel: youtube.com/@thedanyopang

00:00 That's a lot of words
00:24 Different kinds of vocabulary words
2:06 Word mapping between languages
3:16 Vocabulary goes beyond words
6:25 A story about prior knowledge
9:22 Learning words by example and in context
10:31 Flashcards
12:22 Comprehensible input
13:50 Pushed output

Sign up to my email newsletter, here: http://www.benjaminkeep.com

Acknowledgements:
Definitions of “chew” comes from dictionary.com; definitions for “hard,” “take for granted,” and "repetition" come from Merriam-Webster, slightly adapted.

References:

The observations in this video are a mish-mash of my own personal experience and my reading of the literature on language learning. The best survey of that literature that I know of is How Languages are Learned by Patsy Lightbrown and Nina Spada. The 4th edition is here: bookshop.org/a/91541/9780194541268 (bookshop links are affiliate links, so if you purchase through this link, I do get a small commission; I would try to find this from your local library, though).

It’s written with foreign language teachers in mind (not necessarily foreign language students), but hits on many of the key ideas, mentions a lot of great studies that I won’t enumerate here, and offers a relatively balanced portrait of the research on language learning.

The 10,000 word estimate I mentioned at the beginning of the video is low and I wish I would have been more accurate. A lot depends on how you’re defining a word and how you define what “knowing” means. This is a solid paper on estimating the vocabulary of native American English speakers: Brysbaert, M., Stevens, M., Mandera, P., & Keuleers, E. (2016). How Many Words Do We Know? Practical Estimates of Vocabulary Size Dependent on Word Definition, the Degree of Language Input and the Participant’s Age. Frontiers in Psychology, 7. frontiersin.org/articles/10.3389/fpsyg.2016.01116.

The upshot is that they estimate about ~11,000 word families and 42,000 lemmas on average. A lemma is a root word that you can derive into other things (baked, bake, baking would not be three separate words - just one lemma). Word families are larger groups of related words with similar forms and related meanings.

Comprehensible Input is one of the main ideas in Stephen Krashen’s Monitor Theory. This is the classic piece that lays it out: Krashen, S. (1981). Second language acquisition. Second Language Learning, 3(7), 19-39. https://www.academia.edu/download/35238869/second_languge_acquisition_and_learning.pdf

The idea of pushed output comes from Merrill Swain and her studies of proficiency in immersive classrooms. She's got a lot to say on it over the years, but here's a representative example: Swain, M. (1997). Collaborative dialogue: Its contribution to second language learning. riull.ull.es/xmlui/bitstream/handle/915/30603/RCEI_34_%281997%29_07.pdf?sequence=1

There’s much more to say about comprehensible input and pushed output, and I’ve got plans to release videos on those two ideas at least.

The Skill of Learning from Lectures

Benjamin Keep, PhD, JD 2023-11-15 | Some viewers asked for a video on learning from lectures. So I finally made one.

Link to hear news on my membership site: https://forms.gle/2XzhvnGptxeYc28n8

Sign up to my free newsletter, Avoiding Folly, here: http://www.benjaminkeep.com

00:00 Starting with a question
00:53 Lectures and learning
2:10 On the popularity of the lecture format
2:56 Preparing for a lecture
4:38 Making the most of reading assignments
6:33 The goals of preparation: cognitive load
7:37 Create relevant prior knowledge
8:10 Get relevant practical experience
8:29 Begin the organizational process
9:06 Going to the lecture
10:54 Good attention
11:47 Attention drops during lecture
12:29 Dealing with online lectures
13:37 Reviewing after the lecture
14:42 Creative ways of reviewing
16:20 Fundamentals are key
17:26 Frameworks to help you review
19:50 Review vs doing homework
21:38 A little goes a long way

ACKNOWLEDGEMENTS

The scrolling text when I was talking about cognitive load comes from an excerpt from an old Vanity Fair magazine, written by P.G. Wodehouse, available here: vanityfair.com/culture/2014/10/physical-culture-peril-pg-wodehouse-best-of-early-vanity-fair.

The green screen Youtube video (talking about stopping at 10 minutes, 30 minutes) came from here: youtube.com/watch?v=q-7LExVpU4g

I used an example lecture from Scott Page’s Model Thinking on Coursera, found here. The specific excerpt, was taken from the first lecture on aggregation. coursera.org/lecture/model-thinking/aggregation-UWXGF. I definitely recommend Scott Page’s work, especially if you’re interested in social science and social dynamics.

The "taking a test" example came from an MIT course on differential equations: https://ocw.mit.edu/courses/18-03-differential-equations-spring-2010/resources/mit18_03s10_ex1/.

REFERENCES

On attention during lectures. The basic take-away is that passive learning during lectures is similar to a vigilance task (asking people to monitor for low frequency signals from a background of noise - radar monitoring for instance). 10-30 minutes in, steep attention decrements.

Young, M. S., Robinson, S., & Alberts, P. (2009). Students pay attention!: Combating the vigilance decrement to improve learning during lectures. Active Learning in Higher Education, 10(1), 41–55. doi.org/10.1177/1469787408100194

If you are a teacher looking to improve student learning during lectures, this piece from almost 40 years ago still holds up. If you’re a science teacher at the undergraduate level, I recommend visiting the Carl Wieman Science Education Initiative.

Gibbs, G., Habeshaw, S., & Habeshaw, T. (1987). Improving Student Learning During Lectures. Medical Teacher, 9(1), 11–20. doi.org/10.3109/01421598709028976

This is a more modern piece covering some of the same ground:

Cerbin, W. (2018). Improving student learning from lectures. Scholarship of Teaching and Learning in Psychology, 4(3), 151–163. doi.org/10.1037/stl0000113

40 Hours to Learn Mental Math | The riveting conclusion?

Benjamin Keep, PhD, JD 2023-10-26 | It can't be true! I finally published the update to my "learning mental math in 40 hours video"! How many eons have passed? First part is here: youtu.be/uCi0_tjOiyQ

00:00 Last time...
00:40 Is tracking my hours helpful?
01:08 Is the diagnostic test a good measure?
01:55 Speed vs accuracy
03:22 Interleaved vs deliberate practice
04:54 The importance of seeing progress
05:50 Midway diagnostic results
06:15 Practice and performance gains
07:08 Skill integration
09:33 What the test measures
10:12 Are your mistakes changing?
11:15 What I’m focusing on next
12:21 The major system
14:17 28 hours in
14:39 The development of intuition
16:29 35 hours in
18:38 The return of the major system
19:45 Lessons for learning something on your own

Sign up to my email newsletter on reasoning, social science, and decision-making: http://www.benjaminkeep.com

Interested in courses on learning and studying? Sign up to be notified when my site launches: https://forms.gle/xzi1ovwkzCHKvmP58

The math course I was working from was: thegreatcourses.com/courses/secrets-of-mental-math, but Arthur Benjamin also has a book teaching the same things: bookshop.org/a/91541/9780307338402. The bookshop link is an affiliate link so I get a buck or something if you buy it through this link (thanks, if you do!).

Some thoughts on interleaved practice: youtu.be/AWTYfzxBwPg
Some thoughts on deliberate practice: youtu.be/WbUOY9ioIqw (tackling some myths about deliberate practice); youtu.be/3SZDj5TEhU0 (on criticisms of the deliberate practice model)
A quick intro to the novice to expert transition: youtu.be/qwDzjQsM3yQ

How to learn from a book (maybe) | note-taking, visualizations, spacing | history example

Benjamin Keep, PhD, JD 2023-10-04 | Does anyone really know how to learn from a book? Here is a record of my first serious attempt to understand and remember what I was reading for the long term, along with various justifications (and rationalizations!) for what I did.

0:00 What I'm trying to do
0:50 What does it mean to learn from a book?
2:27 Writing down themes
4:27 The back-and-forth
6:01 The function of notes
7:01 Taking paths through the material
7:36 Why do my visualizations suck?
8:53 Getting precise
10:20 Why these (bad) visualizations?
11:14 Spacing things out
12:20 What I would do differently next time

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter here: http://www.benjaminkeep.com.

REFERENCES:

The book is China: A History by John Keay.

My practices here come from a variety of sources, but here's some representative work.

This piece is about teaching students higher-level reading strategies to aid deep comprehension (constructing inferences and explanations, self-monitoring, etc.). A useful read that I largely agree with: McNamara, D. S. (2010). Strategies to read and learn: Overcoming learning by consumption. Medical Education, 44(4), 340–346. doi.org/10.1111/j.1365-2923.2009.03550.x

On free recall and retrieval:

Agarwal, P. K., Nunes, L. D., & Blunt, J. R. (2021). Retrieval Practice Consistently Benefits Student Learning: A Systematic Review of Applied Research in Schools and Classrooms. Educational Psychology Review, 33(4), 1409–1453. doi.org/10.1007/s10648-021-09595-9

Roediger, H. L., & Butler, A. C. (2011). The critical role of retrieval practice in long-term retention. Trends in Cognitive Sciences, 15(1), 20–27. doi.org/10.1016/j.tics.2010.09.003

Karpicke, J. D., & Roediger, H. L. (2008). The Critical Importance of Retrieval for Learning. Science, 319(5865), 966–968. doi.org/10.1126/science.1152408

On the value of "spacing things out" AKA long inter-study intervals, see:

Latimier, A., Peyre, H., & Ramus, F. (2021). A Meta-Analytic Review of the Benefit of Spacing out Retrieval Practice Episodes on Retention. Educational Psychology Review, 33(3), 959–987. doi.org/10.1007/s10648-020-09572-8

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi.org/10.1037/0033-2909.132.3.354

Acknowledgements:

Image of Liu Bang (founding emperor of the Han Dynasty) comes from: en.wikipedia.org/wiki/File:Liu-bang.jpg. It’s actually a Ming-era depiction

Image of Xiang Yu (Liu Bang’s rival) comes from: en.wikipedia.org/wiki/File:Xiang_Yu.jpg

Xuanzang image: en.wikipedia.org/wiki/File:Xuanzang_w.jpg

Xuanzang’s route: en.wikipedia.org/wiki/File:Xuanzang_route.jpg

Zhang Qian’s route: en.wikipedia.org/wiki/File:Han_Expansion.png

Zheng He’s ships: en.wikipedia.org/wiki/File:ZhengHeShips.gif

Zheng He’s route: en.wikipedia.org/wiki/File:Voyages_of_Zheng_He.png (author: SY)

Topographic map: en-us.topographic-map.com/map-pxv3q/China

Youtubers tested the generation effect. Did it work?

Benjamin Keep, PhD, JD 2023-09-20 | Did Youtubers replicate the generation effect? At first, it might not seem like it, but let's take a closer look.

0:00 This video is a follow-up video
0:39 Was there an effect?
3:12 An attempt to visualize the effect
4:36 Sources of variation
5:13 Encoding strategies and non-native speakers
7:30 Why what I just did wasn't really legit
8:14 Variation among experimental designs
10:16 A common source of error
12:12 Oh no, it's an ad...

To be notified when my membership site launches, go here: https://forms.gle/9Q8aDCrSPEkpx8Zn8

Sign up to my email newsletter here: http://www.benjaminkeep.com

REFERENCES

The first one is the meta-analysis that I took screenshots of for the video. The second one is more recent and delves deeper into the theories that explain the effect.

Bertsch, S., Pesta, B. J., Wiscott, R., & McDaniel, M. A. (2007). The generation effect: A meta-analytic review. Memory & Cognition, 35(2), 201–210. doi.org/10.3758/BF03193441

McCurdy, M. P., Viechtbauer, W., Sklenar, A. M., Frankenstein, A. N., & Leshikar, E. D. (2020). Theories of the generation effect and the impact of generation constraint: A meta-analytic review. Psychonomic Bulletin & Review, 27(6), 1139–1165. doi.org/10.3758/s13423-020-01762-3

The Experiment That Teaches People How To Learn

Benjamin Keep, PhD, JD 2023-09-06 | Can you teach yourself to learn more effectively? Memory researcher Elizabeth Bjork thinks so. Participate in a short experiment that illustrates an important link between encoding and retrieval.

0:00 An introduction to Bjork's experiments
0:34 Let's try a little experiment.
4:55 Figuring out your score.
5:32 What's the expected result?
5:58 How did Dr. Bjork use this to help people learn?
7:05 The surprising result.
7:44 What's really driving the learning in the second round?
9:49 The larger point.

The follow-up video is here: youtu.be/7N7tIM7sIxU

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter here: http://www.benjaminkeep.com.

REFERENCES

This video and the experiments I describe in it was based on a series of experiments led by Elizabeth Bjork, renowned memory researcher. The first one is the most readable summary of the research; the other three are a bit dense, but give lots of details about how the experiments were run.

Bjork, E. L., de Winstanley, P. A., & Storm, B. C. (2007). Learning how to learn: Can experiencing the outcome of different encoding strategies enhance subsequent encoding? Psychonomic Bulletin & Review, 14(2), 207–211. doi.org/10.3758/BF03194053

A. de Winstanley, P., & Ligon Bjork, E. (1997). Processing Instructions and the Generation Effect: A Test of the Multifactor Transfer-appropriate Processing Theory. Memory, 5(3), 401–422. doi.org/10.1080/741941392

DeWinstanley, P. A., & Bjork, E. L. (2004). Processing strategies and the generation effect: Implications for making a better reader. Memory & Cognition, 32(6), 945–955. doi.org/10.3758/BF03196872

Bjork, E. L., & Storm, B. C. (2011). Retrieval experience as a modifier of future encoding: Another test effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1113–1124. doi.org/10.1037/a0023549

This is the one where they illustrate that students don’t have to experience the benefits of encoding to improve their encoding strategy. Rather, it seems like just knowing the format of the test and having some experience with different encoding strategies is enough (separately and likely additively together) to lead to superior encoding.

Storm, B. C., Hickman, M. L., & Bjork, E. L. (2016). Improving encoding strategies as a function of test knowledge and experience. Memory & Cognition, 44(4), 660–670. doi.org/10.3758/s13421-016-0588-9

The word lists used in the video were generated from Friendly, M. & Dubins, M. (2019), Paivio et al. Word List Generator, Online application, http://euclid.psych.yorku.ca/shiny/Paivio, Accessed: 2023-06-23

Active Control vs Passive Control in Randomized Controlled Trials | Whats the Difference?

Benjamin Keep, PhD, JD 2023-06-21 | One of the critical decisions that researchers make when designing a controlled trial is the choice of the control: what should be the control group? The answer depends on what we're interested in testing. The distinction I make here is an important one to keep in mind.

0:00 An under-appreciated distinction.
0:34 What is a passive control group?
1:36 What is an active control group?

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter here: http://www.benjaminkeep.com.

REFERENCES

A good explanation of the difference between passive control groups and active control groups begins on page 116, here: Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. L. (2016). Do “Brain-Training” Programs Work? Psychological Science in the Public Interest, 17(3), 103–186. doi.org/10.1177/1529100616661983

Two Reasons NOT to Copy Memory Athletes

Benjamin Keep, PhD, JD 2023-06-14 | Memory athletes use special techniques to perform amazing feats of memory. Are these techniques appropriate for classroom materials?

0:00 Memory athletes can do some amazing things
0:34 Techniques memory athletes use
1:59 The argument against using mnemonics - Reason #1
4:13 The argument against using mnemonics - Reason #2
5:50 The argument in favor of using mnemonics
8:23 Incorporating mnemonic techniques into your studying
8:41 The case of single-digit multiplication
9:18 Chinese emperors and Chinese geography
10:40 The periodic table
11:25 Deciding whether to use mnemonic techniques
12:30 Memory emergencies

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

For more on mnemonic techniques, check out the Art of Memory: artofmemory.com

Sign up to my email newsletter, Avoiding Folly, here: http://www.benjaminkeep.com

REFERENCES

The piece below reports something like the consensus among cognitive psychologists: that mnemonics are pretty good for remembering serial items of unrelated information for short periods of time, but have unproven relevance in the classroom.

Putnam, A. L. (2015). Mnemonics in education: Current research and applications. Translational Issues in Psychological Science, 1(2), 130–139. doi.org/10.1037/tps0000023

This is a piece from a psychologist who taught her students the method of loci (AKA memory palace) that showed some modestly positive results, at least on list learning and general attitude toward the method.

McCabe, J. A. (2015). Location, Location, Location! Demonstrating the Mnemonic Benefit of the Method of Loci. Teaching of Psychology, 42(2), 169–173. doi.org/10.1177/0098628315573143

For a novel application of the method of loci, here is a piece exploring how it might be leveraged for emotional regulation.
Werner-Seidler, A., & Dalgleish, T. (2016). The Method of Loci Improves Longer-Term Retention of Self-Affirming Memories and Facilitates Access to Mood-Repairing Memories in Recurrent Depression. Clinical Psychological Science, 4(6), 1065–1072. doi.org/10.1177/2167702615626693

ACKNOWLEDGEMENTS

Footage from a card memorization event is from: youtube.com/watch?v=OApSOU7NIAw

The periodic table is from the NIH website - amazing! pubchem.ncbi.nlm.nih.gov/periodic-table

The map of China rivers comes from Antonioga, here: en.wikipedia.org/wiki/File:China_rivers.svg

Image of the Loess landscape, in China, comes from Till Niermann, here: en.wikipedia.org/wiki/File:Loess_landscape_china.jpg

The image of Gansu province (camel photo) comes from Sigismund von Dobschütz, here: en.wikipedia.org/wiki/File:Mondsichelsee-06.JPG

The map of traditional geographical and cultural regions in China comes from SY, here: en.wikipedia.org/wiki/File:China_Traditional_Divisions.png

Images related to evolutionary processes come from: Elimbis at Wikimedia commons (commons.wikimedia.org/wiki/File:Mutation_and_selection_diagram.svg) Esurnir at the English Wikipedia (Allele frequency, en.wikipedia.org/wiki/File:Allele-frequency.png); and speciation here (en.wikipedia.org/wiki/File:Speciation_modes_edit.svg)

Image of the Tang Dynasty comes from Seasonsinthesun at Wikimedia Commons (commons.wikimedia.org/wiki/File:Tang_Protectorates.png). Image of the Roman Empire comes from Paulusburg at Wikimedia Commons (commons.wikimedia.org/wiki/File:Mapa_Imperio_Occidental_a%C3%B1o_418.svg)

Teach Your Kid Basic Math

Benjamin Keep, PhD, JD 2023-05-17 | Teaching your kid basic math is a little more complicated than it seems. Here I go over some important learning goals and what I've tried with my own son to try to achieve them.

0:00 So... you want to teach your kid math?
0:15 Four categories of learning goals
1:35 Some important conceptual learning goals
3:14 Some important procedural learning goals
4:36 What does your kid associate math with?
5:09 Becoming a problem solver
5:48 Setting up the time and the environment
6:25 How to use examples and abstractions
7:13 Developing more sophisticated strategies
8:28 Letting them solve the problem
9:07 Developing a skeptical and thorough problem solver
9:42 Flip the asking script
10:32 Modeling how to respond to mistakes
11:41 "Secrets"
12:18 Interesting ways of incorporating number lines
12:55 Scaffolding estimation skills

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my newsletter, Avoiding Folly, here: www.benjaminkeep.com

REFERENCES:

For readable summaries of the research in these areas, check out these two excellent posts by Gwen Dewar: parentingscience.com/preschool-board-game-math (preschool math games) and parentingscience.com/preschool-math-lessons (general advice). Parenting Science is just an excellent resource, generally.

On concreteness fading and the power of using both concrete and abstract representations together, see the two references below:

Fyfe, E. R., & Nathan, M. J. (2019). Making “concreteness fading” more concrete as a theory of instruction for promoting transfer. Educational Review, 71(4), 403–422. doi.org/10.1080/00131911.2018.1424116

Moreno, R., Ozogul, G., & Reisslein, M. (2011). Teaching with concrete and abstract visual representations: Effects on students' problem solving, problem representations, and learning perceptions. Journal of educational psychology, 103(1), 32.

On the relationship between conceptual and procedural knowledge and the role that different kinds of examples can play in developing both, see: Rittle-Johnson, B., & Star, J. R. (2009). Compared with what? The effects of different comparisons on conceptual knowledge and procedural flexibility for equation solving. Journal of Educational Psychology, 101(3), 529. https://dash.harvard.edu/bitstream/handle/1/4889483/CC3b_Final.pdf

On the use of number lines for learning: Lin, C. H. (2022). Developing mental number line games to improve young children’s number knowledge and basic arithmetic skills. Journal of Experimental Child Psychology, 222, 105479. sciencedirect.com/science/article/abs/pii/S0022096522001084

Study Less, Study Smart(er) - Extending Marty Lobdells Study Advice

Benjamin Keep, PhD, JD 2023-05-10 | Marty Lobdell's video, "Study Less, Study Smart" remains excellent advice for college students on how to study. Here are answers to some of the most common questions that come up when you follow his advice.

0:00 A brief introduction
0:23 What kinds of study breaks should you take?
1:59 Where should you study?
3:34 What kinds of notes should you take?
4:38 Deep processing, shallow processing, and transfer-appropriate processing
7:11 How should you structure study groups?
10:17 Ways to study that avoid confusion between recognition and recall
11:49 The skill of reading textbooks

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter here: http://www.benjaminkeep.com

RELATED VIDEOS

Marty Lobdell's Study Less, Study Smart is here: youtube.com/watch?v=IlU-zDU6aQ0

For alternatives to highlighting and flashcards, see:
How to do free recall - youtu.be/FatnXnlwAc4
How to Use Free Recall to Learn More Effectively - youtu.be/evIuDvQDDSY
A simple study tip: stop highlighting - youtu.be/_g3NojwBOXw

For my take on note-taking:
The surprising truth about note-taking - youtu.be/cRQqH18wJgw

On reading textbooks (and reading more generally):
Three simple tricks to read textbooks more effectively - youtu.be/neQ7D90R0xo
Learn more from every paragraph - youtu.be/WRjsOU6mOp4

On attitude and framing:
The most common obstacle to effective studying - youtu.be/6i7HrP84DMw

REFERENCES

On restful periods, see: Wamsley, E. J. (2019). Memory consolidation during waking rest. Trends in cognitive sciences, 23(3), 171-173. ncbi.nlm.nih.gov/pmc/articles/PMC7024394/pdf/nihms-1557224.pdf

The screenshot of the graph showing wakeful resting benefits to wakeful active groups comes from research on auditory learning, here: Gottselig, J. M., Hofer-Tinguely, G., Borbely, A. A., Regel, S. J., Landolt, H. P., Retey, J. V., & Achermann, P. (2004). Sleep and rest facilitate auditory learning. Neuroscience, 127(3), 557-561.

On eyestrain, see: Kaur, K., Gurnani, B., Nayak, S., Deori, N., Kaur, S., Jethani, J., ... & Mishra, D. (2022). Digital Eye Strain-A Comprehensive Review. Ophthalmology and Therapy, 11(5), 1655-1680.https://link.springer.com/article/10.1007/s40123-022-00540-9

On the benefits of self-testing study strategies, see: Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6

On the weak benefits of highlighting and a fantastic review of study strategies more generally, see: Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving Students’ Learning With Effective Learning Techniques: Promising Directions From Cognitive and Educational Psychology. Psychological Science in the Public Interest, 14(1), 4–58. doi.org/10.1177/1529100612453266

On the value of teaching (or preparing to teach) others, see: Okita, S. Y., & Schwartz, D. L. (2013). Learning by Teaching Human Pupils and Teachable Agents: The Importance of Recursive Feedback. Journal of the Learning Sciences, 22(3), 375–412. doi.org/10.1080/10508406.2013.807263

The classic piece on transfer appropriate processing is here: Morris, C. D., Bransford, J. D., & Franks, J. J. (1977). Levels of processing versus transfer appropriate processing. Journal of Verbal Learning and Verbal Behavior, 16(5), 519–533. doi.org/10.1016/S0022-5371(77)80016-9. For a more modern view in the context of an experiment, see this piece: A. de Winstanley, P., & Ligon Bjork, E. (1997). Processing Instructions and the Generation Effect: A Test of the Multifactor Transfer-appropriate Processing Theory. Memory, 5(3), 401–422. doi.org/10.1080/741941392

On deep and shallow processing, see: Craik, F. I. M. (2002). Levels of processing: Past, present... and future? Memory, 10(5–6), 305–318. doi.org/10.1080/09658210244000135

On the memory benefit of learning in different environments, one of the classics is: Smith, S. M., Glenberg, A., & Bjork, R. A. (1978). Environmental context and human memory. Memory & Cognition, 6(4), 342-353.

On links between exercise, sleep, and learning, see: Roig, M., Cristini, J., Parwanta, Z., Ayotte, B., Rodrigues, L., de Las Heras, B., ... & Wright, D. L. (2022). Exercising the sleepy-ing brain: exercise, sleep, and sleep loss on memory. Exercise and sport sciences reviews, 50(1), 38-48.

On links between walking and creative thinking, see: Oppezzo, M., & Schwartz, D. L. (2014). Give your ideas some legs: the positive effect of walking on creative thinking. Journal of experimental psychology: learning, memory, and cognition, 40(4), 1142.

ACKNOWLEDGEMENTS

Spanish Inquisition clip from: youtube.com/watch?v=yKQ_sQKBASM

How to Read Peer-reviewed Articles on Experiments in Learning

Benjamin Keep, PhD, JD 2023-05-03 | Reading peer-reviewed social science research can be quite challenging. But it’s a skill that lets you go beyond media reports of the latest study. Here’s 15 minutes of advice on how to do read experimental studies specifically.

0:00 An intro to reading social science research
0:39 What do learning researchers study?
1:54 The basics of experimental designs
5:01 What’s the goal of reading research?
5:16 The first big question: why?
6:58 The second big question: what happened?
8:12 What’s the participant’s experience?
9:07 A tip for reading complex graphs.
10:23 The third big question: the explanation.
11:04 Some complexities when measuring learning
13:45 What’s the question again?
14:45 Last thoughts

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my newsletter, Avoiding Folly, here: www.benjaminkeep.com

REFERENCES

Clips come from the following pieces:

Example of a longitudinal study: Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological Review, 100(3), 363.

Example of qualitative data and mixed qualitative and quantitative data come from: Reading Abraham Lincoln: An Expert/Expert Study in the Interpretation of Historical Texts. (2003). 1–28.

Example of quantitative data comes from: Bjork, E. L., & Storm, B. C. (2011). Retrieval experience as a modifier of future encoding: Another test effect. Journal of Experimental Psychology: Learning, Memory, and Cognition, 37(5), 1113–1124. doi.org/10.1037/a0023549

Example of abstract/laboratory experiments comes from: Bjork, R. A. (1975). Retrieval as a memory modifier: An interpretation of negative recency and related phenomena.

Example of an applied study (using social studies material in the classroom): Roediger, H. L., Agarwal, P. K., McDaniel, M. A., & McDermott, K. B. (2011). Test-enhanced learning in the classroom: Long-term improvements from quizzing. Journal of Experimental Psychology: Applied, 17(4), 382–395. doi.org/10.1037/a0026252

Examples of an experimental study, data visualization that describes the nature of the materials and one that describes the course of the experiment from: Schwartz, D. L., & Martin, T. (2004). Inventing to prepare for future learning: The hidden efficiency of encouraging original student production in statistics instruction. Cognition and Instruction, 22(2), 129–184.

Research studying the effects of delayed vs immediate feedback are legion. This provides good background reading: Fyfe, E. R., & Rittle-Johnson, B. (2016). Feedback both helps and hinders learning: The causal role of prior knowledge. Journal of Educational Psychology, 108(1), 82–97. doi.org/10.1037/edu0000053

The example of worked examples vs open problem solving comes from some of Sweller’s earlier experiments. Here’s a typical example. Sweller, J., & Cooper, G. A. (1985). The use of worked examples as a substitute for problem solving in learning algebra. Cognition and instruction, 2(1), 59-89.

The statistics game and passage examples I drew from Arena, D. A., & Schwartz, D. L. (2013). Experience and Explanation: Using Videogames to Prepare Students for Formal Instruction in Statistics. Journal of Science Education and Technology, 23(4), 538–548. doi.org/10.1007/s10956-013-9483-3

The screenshot of a statistics passage is from one of the first pages of Regression and Other Stories by Gelman, Hill, and Vehtari, a great textbook on regression.

The design-based research example comes from Ketelhut, D. J., Nelson, B. C., Clarke, J., & Dede, C. (2010). A multi‐user virtual environment for building and assessing higher order inquiry skills in science. British Journal of Educational Technology, 41(1), 56-68. They created a virtual environment called “River City” to facilitate science inquiry experiences.

The “delayed measures” being different than immediate measures graph is from Taylor, K., & Rohrer, D. (2010). The effects of interleaved practice. Applied Cognitive Psychology, 24(6), 837–848. doi.org/10.1002/acp.1598

The “Stats Invaders” clip (the statistics game) comes courtesy of Can Zhang, here: youtube.com/watch?v=J7rx1g8g1Uk

How to Use Free Recall to Learn More Effectively

Benjamin Keep, PhD, JD 2023-04-12 | People have some good questions about how to use free recall to study. Here, I try to answer them.

0:00 Is one recall mode better than another?
1:21 What should the recall interval be?
2:20 Do I need to master the material first?
2:43 Which is better, encoding or retrieval?
4:03 Is free recall just vomiting out words?
4:57 Some further thoughts

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

REFERENCES

On the differences between different modes of recall, see the three references below:

Janczyk, M., Aßmann, M., & Grabowski, J. (2018). Oral versus written recall of long-term memory items: Replicating and extending the writing superiority effect across knowledge domains. The American Journal of Psychology, 131(3), 263-272. (Finding a written superiority effect)

Bekerian, D. A., & Dennett, J. L. (1990). Spoken and written recall of visual narratives. Applied Cognitive Psychology, 4(3), 175-187. (spoken superiority)

Sauerland, M., Krix, A. C., van Kan, N., Glunz, S., & Sak, A. (2014). Speaking is silver, writing is golden? The role of cognitive and social factors in written versus spoken witness accounts. Memory & Cognition, 42, 978-992. (favoring written, but not finding large differences; in the eyewitness area)

For a wonderful summary of how tests effect learning, check out: Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6

On test expectancy effects (how expecting a test influences encoding), see: McDaniel, M. A., Blischak, D. M., & Challis, B. (1994). The effects of test expectancy on processing and memory of prose. Contemporary educational psychology, 19(2), 230-248.

What is Actually Wrong with Rote Learning?

Benjamin Keep, PhD, JD 2023-04-05 | A lot of people don't like rote learning because it's not very fun. But is it effective? I explore this question with a rote learning example in learning to write Chinese characters.

0:00 An Intro to Rote Learning
1:01 Drawback 1 - Blocked Practice
2:57 Drawback 2 - Massed Practice
4:05 Drawback 3 - Skill Isolation

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

REFERENCES

For a recent take on spaced (or distributed) practice, including evidence from neurological studies, read:

Gerbier, E., & Toppino, T. C. (2015). The effect of distributed practice: Neuroscience, cognition, and education. Trends in Neuroscience and Education, 4(3), 49–59. doi.org/10.1016/j.tine.2015.01.001

If you want a more academic take, discussing theories and the available evidence circa 2006, check out:

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological Bulletin, 132(3), 354–380. doi.org/10.1037/0033-2909.132.3.354

For a readable piece on the advantages (and potential drawbacks) of interleaved practice, check out:

Kang, S. H. (2016). The benefits of interleaved practice for learning. In From the laboratory to the classroom (pp. 79-90). Routledge. taylorfrancis.com/chapters/edit/10.4324/9781315625737-8/benefits-interleaved-practice-learning-sean-kang

And a great piece on how to leverage interleaved, spaced, and skill-building practice in the classroom is here:

Hughes, C. A., & Lee, J. Y. (2019). Effective approaches for scheduling and formatting practice: Distributed, cumulative, and interleaved practice. Teaching Exceptional Children, 51(6), 411-423. researchgate.net/profile/Joo-Young-Lee-8/publication/338701579_Effective_Approaches_for_Scheduling_and_Formatting_Practice_Distributed_Cumulative_and_Interleaved_Practice/links/5db7099aa6fdccac19b5d67f/Effective-Approaches-for-Scheduling-and-Formatting-Practice-Distributed-Cumulative-and-Interleaved-Practice.pdf

This Simple Method Helps You Learn More from Podcasts (and Audiobooks!)

Benjamin Keep, PhD, JD 2023-03-29 | There's a lot of great podcasts out there. But what should we do if we want to learn from them? Here's a simple method.

00:00 Some drawbacks of using podcasts to learn
01:49 Step 1 - Activate prior knowledge
02:42 Step 2 - Free recall and fill in the gaps
05:49 Step 3 - Consolidate and extend

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

ACKNOWLEDGEMENTS

Screenshots of Twilight of the Aesir and Hardcore History come from dancarlin.com.

Jutland and Saxony screenshots come from Wikipedia.

I use Miro for the visualization software clip.

REFERENCES

My comments about what free recall self-tests can do for you are based on the large literature related to the testing effect. At a high-level, frequent tests enhance learning in a variety of ways.

For a thorough discussion of the possible benefits (including anticipatory/motivational aspects and organizational aspects, like I mention in the video), check out:

Roediger III, H. L., Putnam, A. L., & Smith, M. A. (2011). Ten Benefits of Testing and Their Applications to Educational Practice. In Psychology of Learning and Motivation (Vol. 55, pp. 1–36). Elsevier. doi.org/10.1016/B978-0-12-387691-1.00001-6

Here is a recent meta-analysis exploring different potential mechanisms and applications of the testing effect(s):

Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin, 147(4), 399–435. doi.org/10.1037/bul0000309

The Knowledge That Underlies Everything | Tacit Knowledge

Benjamin Keep, PhD, JD 2023-03-22 | What is tacit (or implicit) knowledge and what does it have to do with learning? To answer this question, we have to explore bike riding, expert blindness, a Chinese room, and how we remember.

0:00 An intro to tacit knowledge
0:45 Let's try an experiment
1:42 "Weak tacit knowledge"
3:36 Expert blindness
5:31 "Body knowledge"
8:03 "Social tacit knowledge" and Searle's Chinese Room
11:10 How do you learn tacit knowledge?

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

REFERENCES

The bones of the video come from: Collins, H. (2019). Tacit and explicit knowledge. University of Chicago press. I changed the labels he gave for the different kinds of tact knowledge he identifies because I found them pretty confusing.

Michael Polanyi is credited with first identifying and discussing tacit knowledge. His thoughts can be found in: Polanyi, M., & Sen, A. (2009). The tacit dimension. University of Chicago press.

The "explicit to implicit" model of skill acquisition is commonly exemplified by the Dreyfus model:
Dreyfus, S. E., & Dreyfus, H. L. (1980). A five-stage model of the mental activities involved in directed skill acquisition. California Univ Berkeley Operations Research Center. https://apps.dtic.mil/sti/pdfs/ADA084551.pdf

A good introduction to Searle's Chinese Room (with plenty of references), can be found here: https://plato.stanford.edu/entries/chinese-room/

ACKNOWLEDGEMENTS

Star Trek clip from youtube.com/watch?v=hdjL8WXjlGI

The physics of bike riding visuals come from this gem from 1970: Jones, D. E. (1970). The stability of the bicycle. Physics today, 23(4), 34-40. Available here: https://home.phys.ntnu.no/brukdef/undervisning/tfy4145/arkiv/2010/diverse/UnridableBicycle.pdf

The clip of a robot riding a bicycle came from: youtube.com/watch?v=mT3vfSQePcs

“Frampold” entry and other rare word screenshots from the OED found here: oed.com/view/Entry/74166

The recipe screenshots are from Serious Eats: seriouseats.com/the-food-lab-how-to-make-creamy-vegetable-soup; seriouseats.com/spicy-carrot-and-ginger-soup-with-harissa.

Urban Dictionary add word screen found here: my.urbandictionary.com/add.php

Image of Harry Collins from Alexei Kouprianov on Wikipedia. upload.wikimedia.org/wikipedia/commons/5/5f/Harry_Collins.20101118.jpg

Learn More From Every Paragraph | Active Reading Example

Benjamin Keep, PhD, JD 2023-03-08 | Reading is about more than just decoding the words and sentences. It's about thinking and reasoning and truly engaging with the material. Here is an example of a more active reading approach.

0:00 The paragraph in question
1:20 Some ways to approach learning from this paragraph
2:08 How I read it
5:17 What does reading something actually mean?

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

Here's the video I plug at the end: What Speed readers won’t tell you - youtu.be/D2aZ3THL8BQ

Here is the book that I took the paragraph from: Emling, S. (2009). The fossil hunter: dinosaurs, evolution, and the woman whose discoveries changed the world. St. Martin's Press. bookshop.org/a/91541/9780230103429 (I am a bookshop.org affiliate, which means I get small commission if you purchase through this link.)

I should say that there is more to good reading than what I have outlined in the video, but at least it demonstrates some basics.

For a concise summary of what good readers do, check out the first page of the following article: Duke, N. K., & Pearson, P. D. (2009). Effective practices for developing reading comprehension. Journal of education, 189(1-2), 107-122. asset-pdf.scinapse.io/prod/45283437/45283437.pdf

Happy reading!

What Speed Readers Wont Tell You

Benjamin Keep, PhD, JD 2023-03-01 | Speed reading seems like a way to learn more efficiently. But is it? I explore what the research says about speed reading (and normal reading).

0:00 How to explore the claims of speed readers
1:09 Tim Ferris's speed reading techniques
3:26 The relationship between eye movements and reading comprehension
4:49 Tests of reading comprehension
5:52 Maximum words per minute (WPM)
6:46 Why do people believe in speed reading?

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

More learning myths: youtu.be/jcMk3e7u7zM
More on when learning fast backfires: youtu.be/wRb32j6_pD4

Thomas Frank’s videos on speed reading go over the research quite well, too. Check them out here: youtube.com/watch?v=jv2BdHXRD3Q (part 1), youtube.com/watch?v=JL4WMHyUhdc (part 2).

ACKNOWLEDGEMENTS

Opening screenshots of the Spreeder app found here: spreeder.com

Excerpts of Tim Ferris's speed reading video comes from: youtube.com/watch?v=ZwEquW_Yij0. The screenshot of comments comes from there, too.

Demo of the Spritz speed reading app comes from: readmei.com

I used this speed reading tool to display the words "This doesn't work at all" in the video: tools-unite.com/tools/speed-reading-app

The sentence I used to illustrate that people pause at the end of sentences comes from The Man Without Qualities by Robert Musil, translated by Sophie Wilkins and Burton Pike (bookshop.org/p/books/man-without-qualities-robert-musil/18761811): "There is, in short, no great idea that stupidity cannot put to its own uses; it can move in all directions, and put on all the guises of truth." A difficult, but hilarious and profound and tragic book.

REFERENCES

The most thorough description of the reading process, and why speed reading doesn’t work, is here: Rayner, K., Schotter, E. R., Masson, M. E., Potter, M. C., & Treiman, R. (2016). So much to read, so little time: How do we read, and can speed reading help?. Psychological Science in the Public Interest, 17(1), 4-34. journals.sagepub.com/doi/pdf/10.1177/1529100615623267

The paper that tested serial presentation of words (in other words, no eye movements, just rapid word presentation like popular speed reading apps such as Spritz) is: Acklin, D., & Papesh, M. H. (2017). Modern speed-reading apps do not foster reading comprehension. The American Journal of Psychology, 130(2), 183-199.. researchgate.net/profile/Dina-Acklin/publication/316499956_Modern_Speed-Reading_Apps_Do_Not_Foster_Reading_Comprehension/links/598de0b5a6fdcc1225fca2cc/Modern-Speed-Reading-Apps-Do-Not-Foster-Reading-Comprehension.pdf

For good background reading, especially for seeing how old false claims about speed reading are, check out: Brozo, W. G., & Johns, J. L. (1986). A content and critical analysis of 40 speed reading books. Journal of Reading, 30(3), 242-247.

For support of the idea that regressions (which I called backtracking in the video) HELP comprehension, see below. I used their fig 2 (page 4) to show how regressions facilitate reading comprehension. Schotter, E. R., Tran, R., & Rayner, K. (2014). Don’t believe what you read (only once) comprehension is supported by regressions during reading. Psychological science, 25(6), 1218-1226. journals.sagepub.com/doi/pdf/10.1177/0956797614531148

For a readable summary of the research on speed reading, check out the introduction to this piece. I used the graph of speed reading vs normal reading in the video as well (fig 3, page 7). Weimer, C. (2021). Effects of Reading Speed and Retrieval Practice on Reading Comprehension. Nysa, the NKU Journal of Student Research; v. 2, Fall 2019. https://dspace.nku.edu/bitstream/handle/11216/3673/OCR%20Weimer_The%20Effects%20of%20Reading%20Speed%20and%20Retrieval%20Practice%20on%20Reading%20Comprehension_Nysa_2_2019.pdf?sequence=1

For information on developing reading fluency for non-native speakers, which is a different but related research question, check out this one: Nation, P. (2009). Reading faster. International Journal of English Studies, 9(2). revistas.um.es/ijes/article/download/90791/87591

Active Processing vs Active Learning | Whats the difference?

Benjamin Keep, PhD, JD 2023-02-22 | Academic words can often sound the same but have distinct, important meanings. Understanding the difference between active processing and active learning can help you read the studies on learning more effectively

0:00 What does active processing mean?
0:59 What does active learning mean?
2:14 The relationship between active processing and active learning

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my newsletter, Avoiding Folly, here: benjaminkeep.com

REFERENCES:

On active learning in the classroom, I recommend:

Freeman, S., Eddy, S. L., McDonough, M., Smith, M. K., Okoroafor, N., Jordt, H., & Wenderoth, M. P. (2014). Active learning increases student performance in science, engineering, and mathematics. Proceedings of the National Academy of Sciences, 111(23), 8410–8415. doi.org/10.1073/pnas.1319030111

Active processing is a big picture idea that crosses many areas of learning, but here is a general paper discussing the role of active processing in remembering:

Schwartz, B. L., Son, L. K., Kornell, N., & Finn, B. (n.d.). Four Principles of Memory Improvement: A Guide to Improving Learning Efficiency. https://web.williams.edu/Psychology/Faculty/Kornell/Publications/Schwartz.Son.Kornell.Finn.2011.pdf

And here is one on the role of active learning in reading: McNamara, D. S. (2010). Strategies to read and learn: Overcoming learning by consumption. Medical Education, 44(4), 340–346. doi.org/10.1111/j.1365-2923.2009.03550.x

What does good learning FEEL like?

Benjamin Keep, PhD, JD 2023-02-15 | Learning can be frustrating and exhilarating. Is there an emotion that's associated with optimal learning states?

0:00 An introduction
0:18 Flow experiences and learning
1:37 Happiness and learning
2:12 Frustration and learning
3:14 Tiredness and learning
4:27 A cycle of feelings

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

How visualization helps you learn: youtu.be/eeEfyRYlyYQ
On effective practice: youtu.be/aIPS4ugcanM

REFERENCES

For the classic piece on flow, see: Flow: The Psychology of Optimal Experience by Mihaly Csikszentmihalyi. bookshop.org/p/books/flow-the-psychology-of-optimal-experience-mihaly-csikszentmihalyi/8937185

I pulled some quotes on the flow experience from: Jackson, S. A. (1996). Toward a conceptual understanding of the flow experience in elite athletes. Research quarterly for exercise and sport, 67(1), 76-90.

I also can’t resist mentioning a method paper. This is the one to read to understand the current state of the research and how researchers have gone about defining flow: Abuhamdeh, S. (2020). Investigating the “flow” experience: Key conceptual and operational issues. Frontiers in psychology, 11, 158. frontiersin.org/articles/10.3389/fpsyg.2020.00158/full

The link between rest and deliberate practice comes from Ericsson et al.’s first big paper on deliberate practice, cited below at pages 370-71. AFAIK, it hasn’t been an aspect that’s been deeply explored. Ericsson, K. A., Krampe, R. T., & Tesch-Römer, C. (1993). The role of deliberate practice in the acquisition of expert performance. Psychological review, 100(3), 363. psycnet.apa.org/doiLanding?doi=10.1037%2F0033-295X.100.3.363

The relationship between confusion, frustration, and learning is complex - more complex than I alluded to in the video. Ryan Baker and his colleagues have done a lot of work recently exploring this space - here’s a couple of representative pieces:

Richey, J. E., Andres-Bray, J. M. L., Mogessie, M., Scruggs, R., Andres, J. M., Star, J. R., ... & McLaren, B. M. (2019). More confusion and frustration, better learning: The impact of erroneous examples. Computers & Education, 139, 173-190. par.nsf.gov/servlets/purl/10155531 (finding that combined "confusion and frustration" predicted learning)

Richey, J. E., McLaren, B. M., Andres-Bray, M., Mogessie, M., Scruggs, R., Baker, R., & Star, J. (2019). Confrustion in Learning from Erroneous Examples: Does Type of Prompted Self-explanation Make a Difference?. In Artificial Intelligence in Education: 20th International Conference, AIED 2019, Chicago, IL, USA, June 25-29, 2019, Proceedings, Part I 20 (pp. 445-457). Springer International Publishing. (finding that combined "frustration and confusion" predicted lower learning).

On the role of mistakes in learning, check out: Metcalfe, J. (2017). Learning from errors. Annual review of psychology, 68, 465-489. annualreviews.org/doi/full/10.1146/annurev-psych-010416-044022

I Learned Mental Math In 40 Hours

Benjamin Keep, PhD, JD 2023-02-08 | Watch me put research-based learning principles to the test as I learn mental arithmetic. I use testing, lots of relevant, deliberate practice with feedback, spaced practice, interleaving, and review to learn to become as good at mental math as I can in 40 hours.

0:00 An introduction
0:20 The plan
1:08 How I apply the science of learning
3:29 My first diagnostic test
4:20 Reflections after ten hours
5:06 I'm noticing patterns
6:23 How to use practice tests to learn
8:47 Ten hour mark test
9:17 Next steps

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

Here’s the link to the course I’m using: thegreatcourses.com/courses/secrets-of-mental-math

The link to the book I mentioned: bookshop.org/a/91541/9780307338402 (I'm a Bookshop.org affiliate, so get a commission if you purchase through this link).

REFERENCES:

For a readable introduction to interleaving in math from the teacher’s perspective, see: http://pdf.retrievalpractice.org/InterleavingGuide.pdf.

For more background on interleaving (like so many of these principles, it get’s complicated), see: Firth, J., Rivers, I., & Boyle, J. (2021). A systematic review of interleaving as a concept learning strategy. Review of Education, 9(2), 642-684. bera-journals.onlinelibrary.wiley.com/doi/pdfdirect/10.1002/rev3.3266

For more on spaced practice, see: Latimier, A., Peyre, H., & Ramus, F. (2021). A meta-analytic review of the benefit of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33(3), 959-987. http://www.lscp.net/persons/ramus/docs/EPR20.pdf

For evidence supporting the “keep testing learned items” idea, see this classic: Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf

How Bad Ideas About Learning Spread | 5 Examples

Benjamin Keep, PhD, JD 2022-12-14 | A lot of strange ideas circulate among teachers and researchers and parents and students. Why?

0:00 The problem
1:05 A thought experiment
2:43 Ideas without tests and Dale’s Cone of Experience
4:09 Learning Styles
5:40 Why businesses sell ideas and Baby Einstein
7:23 “Science-based” businesses and brain training
8:37 Supermemo
10:19 What stops good ideas from spreading?
11:15 The role of teacher training
12:13 Good news

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

Two books that I often recommend on learning are: The ABCs of How We Learn by Schwartz, Tsang, and Blair (bookshop.org/a/91541/9780393709261) and Make It Stick by Brown, Roediger, and McDaniel (bookshop.org/a/91541/9780674729018). I'm a Bookshop.org affiliate, so get a small commission if you purchase books through the above links.

Acknowledgements

The green example of Dale’s cone of experience (or learning) comes from commons.wikimedia.org/wiki/File:Edgar_Dale%27s_cone_of_learning.png

The Baby Einstein clip was from this video: youtube.com/watch?v=tHygdh6fCg4

The animated “learning styles” clip comes from: youtube.com/watch?v=_IopcOwfsoU

REFERENCES

A recent summary of the literature on screen time for young children: Guellai, B., Somogyi, E., Esseily, R., & Chopin, A. (2022). Effects of screen exposure on young children’s cognitive development: A review. Frontiers in Psychology, 4779. frontiersin.org/articles/10.3389/fpsyg.2022.923370/full?

On physical activity and development, see: Carson, V., Kuzik, N., Hunter, S., Wiebe, S. A., Spence, J. C., Friedman, A., ... & Hinkley, T. (2015). Systematic review of sedentary behavior and cognitive development in early childhood. Preventive medicine, 78, 115-122. researchgate.net/profile/Alinda-Friedman/publication/280537626_Systematic_Review_of_Sedentary_Behavior_and_Cognitive_Development_in_Early_Childhood/links/5ea46d0292851c1a906f2347/Systematic-Review-of-Sedentary-Behavior-and-Cognitive-Development-in-Early-Childhood.pdf

On brain training, see: Simons, D. J., Boot, W. R., Charness, N., Gathercole, S. E., Chabris, C. F., Hambrick, D. Z., & Stine-Morrow, E. A. (2016). Do “brain-training” programs work?. Psychological Science in the Public Interest, 17(3), 103-186. https://cogsci.northwestern.edu/events/2016-2017-events/simonsEtAl_2016-BrainTraining.pdf.

The example I gave of positive (narrow) results was from one of the early papers in the ACTIVE trial: Ball, K., Berch, D. B., Helmers, K. F., Jobe, J. B., Leveck, M. D., Marsiske, M., ... & ACTIVE Study Group. (2002). Effects of cognitive training interventions with older adults: a randomized controlled trial. Jama, 288(18), 2271-2281.

Supermemo advocates for an expanding spacing schedule, which isn’t necessarily bad. But the question of whether fixed or expanding intervals are superior is complicated. Perhaps it has something to do with how hard the retrieval is or the number of retrieval attempts. But there’s not a clear winner. See these two pieces for further discussion.

Latimier, A., Peyre, H., & Ramus, F. (2021). A meta-analytic review of the benefit of spacing out retrieval practice episodes on retention. Educational Psychology Review, 33(3), 959-987. http://www.lscp.net/persons/ramus/docs/EPR20.pdf

Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological bulletin, 132(3), 354. escholarship.org/content/qt3rr6q10c/qt3rr6q10c.pdf

The 2016 teacher training textbook report I mention, is here: Pomerance, L., Greenberg, J., & Walsh, K. (2016). Learning about Learning: What Every New Teacher Needs to Know. National Council on Teacher Quality. files.eric.ed.gov/fulltext/ED570861.pdf

The paper I showed which was a collaboration between researchers and teachers was: Agarwal, P. K., Bain, P. M., & Chamberlain, R. W. (2012). The value of applied research: Retrieval practice improves classroom learning and recommendations from a teacher, a principal, and a scientist. Educational Psychology Review, 24(3), 437-448.

3 Forgotten Studying Secrets from a 1979 Memory Expert

Benjamin Keep, PhD, JD 2022-11-09 | Today, a memory expert reveals his studying secrets. From 40 years ago.

0:00 A time machine
01:00 Study secret #1
01:30 Study secret #2
02:16 Study secret #3
05:30 What Robert Bjork from 1979 can teach us

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

REFERENCES

This video was based on a Robert Bjork joint:

Bjork, R. A. (1979). Information‐processing analysis of college teaching. Educational Psychologist, 14(1), 15-23. researchgate.net/profile/Robert-Bjork-2/publication/261581257_Information-Processing_Analysis_of_College_Teaching/links/5d260916458515c11c23fa01/Information-Processing-Analysis-of-College-Teaching.pdf

The sentence rearrangement study comes from: Bradford, E. J. (1975). Retention of subject-generated verbal material (Doctoral dissertation, ProQuest Information & Learning).

For more modern advice from researchers on studying, check out:

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest, 14(1), 4-58.

NOTE: I know nothing about immune systems. Just using as a potential example. Please don't take anything I say about them as gospel.

ACKNOWLEDGEMENTS

Image of Thucydides sculpture from: commons.wikimedia.org/wiki/File:Thucydides_pushkin02.jpg

Image of al-Razi sculpture from: commons.wikimedia.org/wiki/File:Persian_Scholar_pavilion_in_Viena_UN_(Rhazes).jpg

Image of Louis Pasteur: en.wikipedia.org/wiki/File:Louis_Pasteur,_foto_av_Paul_Nadar,_Crisco_edit.jpg

Image of Paul Ehrlich: commons.wikimedia.org/wiki/File:Paul_Ehrlich_1915.jpg

The immune system process image comes from: covalab.com/media/wysiwyg/pictures/pathways/innate_immunity_pathway.jpg

The innate vs adaptive immune system slide comes from: microbenotes.com/differences-between-innate-immunity-and-adaptive-immunity

Learning Beyond Facts | Conceptual Knowledge, Procedural Knowledge, and More

Benjamin Keep, PhD, JD 2022-11-03 | After many years in school, it's easy to get the impression that learning is mostly about facts. But there's a whole diverse world of different kinds of knowledge out there. Let's explore it!

00:00 Introduction
00:17 The fact
1:11 The look-up table
2:12 Conceptual knowledge - how is a concept different than a fact?
3:30 How we learn concepts
4:12 The benefits of concepts
5:18 Procedural knowledge and skill
5:43 The difference between conceptual knowledge and procedural knowledge
6:27 Unbalanced knowledge
7:30 Bare association
8:18 The ability to recognize something
8:54 I shouldn't talk about this one

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

REFERENCES

On the interaction of procedural and conceptual knowledge, see:

Rittle-Johnson, B., Schneider, M., & Star, J. R. (2015). Not a one-way street: Bidirectional relations between procedural and conceptual knowledge of mathematics. Educational Psychology Review, 27(4), 587-597. https://www.uni-trier.de/fileadmin/fb1/prof/PSY/PAE/Team/Schneider/Rittle-JohnsonEtAl2015.pdf (Math)

Schauble, L. (1996). The development of scientific reasoning in knowledge-rich contexts. Developmental Psychology, 32(1), 102. researchgate.net/profile/Leona-Schauble/publication/232578689_The_development_of_scientific_reasoning_in_knowledge-rich_contexts/links/56eaa8f108ae7858657fde00/The-development-of-scientific-reasoning-in-knowledge-rich-contexts.pdf (Science)

For a more complicated, great take on knowledge in problem solving, see:

De Jong, T., & Ferguson-Hessler, M. G. (1996). Types and qualities of knowledge. Educational psychologist, 31(2), 105-113. https://ris.utwente.nl/ws/files/6401593/types.pdf

For the difference between varieties of recall and recognition, see:

Richardson-Klavehn, A., & Bjork, R. A. (1988). Measures of memory. Annual review of psychology, 39(1), 475-543. researchgate.net/profile/Robert-Bjork-2/publication/234837363_Measures_Of_Memory/links/0deec52efb30acdddd000000/Measures-Of-Memory.pdf

ACKNOWLEDGEMENTS

The gravity demonstration comes from this video: youtube.com/watch?v=MTY1Kje0yLg.

However, there are plenty of good arguments as to why this isn't a terribly helpful analogy: http://newsletter.oapt.ca/files/general-relativiy-analogies.html

The Most Common Obstacle to Effective Studying

Benjamin Keep, PhD, JD 2022-10-26 | Effective studying requires effective study strategies and techniques, but it also requires the right conceptualization of what studying is.

00:00 Introduction
00:54 The big misconception
1:54 What makes human memory different from a computer's
3:47 Some benefits and drawbacks to the human memory system
4:39 The key question

If you want to be the first to know about the courses I'm releasing, sign up here: https://forms.gle/px7ZmXkvJW26uFWp8

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

How forgetting works: youtu.be/uv7uhI8nsbs
More effective studying: youtu.be/iW7Fp9Mtq1Y
Better note-taking: youtu.be/cRQqH18wJgw
Free recall: youtu.be/FatnXnlwAc4
Exam prep: youtu.be/GoTYyn3WWxI

REFERENCES

The laundry passage comes from this famous 1972 paper, and is often used as a simple example of the power of prior knowledge.

Bransford, J. D., & Johnson, M. K. (1972). Contextual prerequisites for understanding: Some investigations of comprehension and recall. Journal of verbal learning and verbal behavior, 11(6), 717-726. http://www.ebbinghaus.ru/wp-content/uploads/2010/02/Bransford__Johnson_1972.pdf

The two-different-senses-of-foot demonstration comes from this paper, where the group who were cued to think of two different senses of the to-be-remembered word remembered it better than those who only were cued to think of one sense of that word.

Gartman, L. M., & Johnson, N. F. (1972). Massed versus distributed repetition of homographs: A test of the differential-encoding hypothesis. Journal of Verbal Learning and Verbal Behavior, 11(6), 801-808. sciencedirect.com/science/article/abs/pii/S0022537172800161

Both of these are also cited in a Robert Bjork paper from 1979. Worth reading. Going to do a full video on this paper soon.

Bjork, R. A. (1979). Information‐processing analysis of college teaching. Educational Psychologist, 14(1), 15-23. researchgate.net/profile/Robert-Bjork-2/publication/261581257_Information-Processing_Analysis_of_College_Teaching/links/5d260916458515c11c23fa01/Information-Processing-Analysis-of-College-Teaching.pdf?_sg%5B0%5D=started_experiment_milestone&origin=journalDetail

If you're interested in witness misidentifications, check out www.innocenceproject.org. Eyewitness misidentifications are the leading cause of wrongful convictions, and there are some pretty simple steps to prevent them from happening: innocenceproject.org/eyewitness-identification-reform

What Study Gurus Get Wrong About Learning

Benjamin Keep, PhD, JD 2022-10-19 | Study gurus promote active recall and spaced repetition to maximize learning. But is it all its cracked up to be? In the brain, retrieval and encoding processes interact to produce learning. The trick is doing them right.

00:00 Introduction
01:17 Our brain’s memory systems
02:38 Justin’s beef with active recall
03:29 Spaced repetition systems, “active recall”, and spaced retrieval practice
04:35 Why flashcard systems kind of suck
05:12 Justin’s recommendation
05:35 What are “desirable difficulties”?
07:03 The alternatives to flashcards
08:35 A good question to ask yourself when studying
10:02 An example study comparing elaborative encoding to retrieval practice
11:25 A true statement

Here are some of my other videos which are relevant to the discussion:

How forgetting works: youtu.be/uv7uhI8nsbs
Free recall demo: youtu.be/FatnXnlwAc4
Questions about free recall answered: youtu.be/evIuDvQDDSY
Example of encoding from reading using organization: youtu.be/WRjsOU6mOp4
What researchers knew in 1979: youtu.be/SOgSpCt3H4M
Advice about learning on your own: youtu.be/3zmek5g3Xxk
Other myths about learning: youtu.be/jcMk3e7u7zM
Desirable difficulties: youtu.be/Q7xnl2tyKwY (my first youtube video!)
Flashcards: youtu.be/ZIGrHI353no
Interleaving: youtu.be/AWTYfzxBwPg

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

To read more about retrieval practice, check out: benjaminkeep.com/the-wicked-effectiveness-of-retrieval-practice

Justin Sung's video on retrieval: youtube.com/watch?v=--Hu2w0s72Y
Justin Sung's video on encoding: youtube.com/watch?v=VcT8puLpNKA

The pro baseball footage is from: youtube.com/watch?v=21TmGchVXRA

REFERENCES

The article I showed in the video comparing elaborative encoding to retrieval practice is:

Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772-775. bit.ly/3S5i9KP

Below, one of the classic pieces on retrieval. It's really an illustration of why you shouldn't just stop trying to remember something just because you have successfully recalled it in the past:

Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf

This is a meta-analysis on the testing effect, which I haven't read in-depth lately, but is worth checking out if you're into this stuff:

Rowland, C. A. (2014). The effect of testing versus restudy on retention: a meta-analytic review of the testing effect. Psychological bulletin, 140(6), 1432. bit.ly/3VwGpII

There's quite a bit of support for the idea that you want to be operating at a "high level" (e.g., synthesizing, applying, critiquing) early in the game. It's not like you need to learn all these rote facts first. A point that Justin makes eloquently in his video on encoding. The piece below is a great discussion of that.

Agarwal, P. K. (2019). Retrieval practice & Bloom’s taxonomy: Do students need fact knowledge before higher order learning?. Journal of Educational Psychology, 111(2), 189. bit.ly/3T5Va3r

Some have argued that retrieval is only good for simple materials (like remembering new vocabulary words) and not complex materials (like understanding how Newtonian physics works). Although many of the early studies do focus on "simple" materials, there's plenty of studies that establish the effect of retrieval is just as strong for complex materials.

Karpicke, J. D., & Aue, W. R. (2015). The testing effect is alive and well with complex materials. Educational Psychology Review, 27(2), 317-326. link.springer.com/article/10.1007/s10648-015-9309-3

How I got tricked by this one weird trick - eating, the Delbeouf illusion, and meta-analyses

Benjamin Keep, PhD, JD 2022-10-12 | Research suggests that using small plates can reduce how much food you eat. This starts with a simple visual illusion, called the Delboeuf illusion. And then it gets more complicated from there.

00:00 Introduction
00:44 What is the Delboeuf illusion?
1:36 How the Delboeuf illusion can make us eat more food.
2:48 Was anything I said correct?
4:36 Potential explanations for why we eat more when served larger portions.
6:09 What this video is actually about.
6:49 The meta-analysis strikes back.

You can read more about Ben Goldacre here: en.wikipedia.org/wiki/Ben_Goldacre. I recommend his books and articles.

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

References:

The original inspiration for this video came from:

Van Ittersum, K., & Wansink, B. (2012). Plate size and color suggestibility: The Delboeuf illusion’s bias on serving and eating behavior. Journal of Consumer Research, 39(2), 215-228. academic.oup.com/jcr/article-pdf/39/2/215/5081845/39-2-215.pdf

A meta-analysis looking at the small plate ➝ lower intake connection. And finding no consistent relationship.

Robinson, E., Nolan, S., Tudur‐Smith, C., Boyland, E. J., Harrold, J. A., Hardman, C. A., & Halford, J. C. (2014). Will smaller plates lead to smaller waists? A systematic review and meta‐analysis of the effect that experimental manipulation of dishware size has on energy consumption. Obesity Reviews, 15(10), 812-821. livrepository.liverpool.ac.uk/3009869/1/Robinson%20et%20al%20Obesity%20Reviews%20AAM.doc

And this piece raises questions about the size of the Delboeuf illusion for hungry people.

Zitron-Emanuel, N., & Ganel, T. (2018). Food deprivation reduces the susceptibility to size-contrast illusions. Appetite, 128, 138-144. sciencedirect.com/science/article/pii/S0195666318300011

How portion size effects caloric intake is still disputed. It seems like there's some anchoring involved (the initial portion suggests what is normal or appropriate) and bite size may play a role as well.

Herman, C. P., Polivy, J., Pliner, P., & Vartanian, L. R. (2015). Mechanisms underlying the portion-size effect. Physiology & Behavior, 144, 129-136. http://www2.psy.unsw.edu.au/Users/lvartanian/Publications/Herman,%20Polivy,%20Pliner,%20&%20Vartanian%20(2015).pdf

Burger, K. S., Fisher, J. O., & Johnson, S. L. (2011). Mechanisms behind the portion size effect: visibility and bite size. Obesity, 19(3), 546-551. onlinelibrary.wiley.com/doi/pdfdirect/10.1038/oby.2010.233

Marchiori, D., Papies, E. K., & Klein, O. (2014). The portion size effect on food intake. An anchoring and adjustment process?. Appetite, 81, 108-115. https://www.academia.edu/download/44357505/The_Portion_Size_Effect_on_Food_Intake_a20160403-22310-r0punj.pdf

Finally, this is my favorite meta-analysis, which DOES, in fact, suggest that smaller plates lead to smaller portion sizes and reduced consumption. It's still a "may" though. The effect seems quite vulnerable to user awareness. So if participants are aware that they're in a food study, researchers don't find an effect. Or if they're aware that they just changed their plates to reduce their consumption, you don't see an effect. But there is at least some suggestion that in the long-run, as your awareness of the changed plates fades, you do end up reducing your consumption.

Holden, S. S., Zlatevska, N., & Dubelaar, C. (2016). Whether smaller plates reduce consumption depends on who’s serving and who’s looking: a meta-analysis. Journal of the Association for Consumer Research, 1(1), 134-146. https://opus.lib.uts.edu.au/bitstream/10453/41448/1/2015003.email.pdf

7 Tips For Learning Anything On Your Own

Benjamin Keep, PhD, JD 2022-10-05 | Learning things on your own is hard. Maybe these tips will help.

00:00 Introduction
00:42 How to make specific learning goals
2:03 What are the key skills?
3:21 Gathering learning resources
4:02 A note on scheduling
4:48 Making the best use of feedback
8:12 Watch out for time-sensitive learning goals
10:15 When to make a change

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

REFERENCES

On creating meaningful learning goals, see:

cwsei.ubc.ca/resources/instructor/learninggoals, from the Carl Weiman Science Initiative

To read more about metacognition and illusions of knowing, see:

Avhustiuk, M. M., Pasichnyk, I. D., & Kalamazh, R. V. (2018). The illusion of knowing in metacognitive monitoring: Effects of the type of information and of personal, cognitive, metacognitive, and individual psychological characteristics. Europe's Journal of Psychology, 14(2), 317. ncbi.nlm.nih.gov/pmc/articles/PMC6016031

On spacing, see my Medium article and the references linked therein: bit.ly/3SBcD3v

To read more about Goodhart's Law and other ways that metrics can backfire, I highly recommend:

Muller, J. Z. (2019). The Tyranny of Metrics. Princeton University Press.

CREDITS

My blog post example was www.betterexplained.com - a really good website discussing the intuitions behind many math concepts.

The Youtube channel example was Mathematical Visual Proofs: @MathVisualProofs

The textbook example was "University Physics" - widely regarded as one of the best basic physics books out there. Young, H. D., Freedman, R. A., & Ford, A. L. (2016) Sears and Zemansky's university physics. Pearson education. 14th Edition. There's more recent editions as well.

StackExchange is one of the great places to get answers to questions by experts - especially in programming, engineering, and mathematics. The forum post I showed was from CrossValidated - their statistics arm. stackexchange.com/sites#

The Magic player was Kenji "NumotTheNummy" Egashira, his channel is here: @NumotTheNummyYT

Steph Curry highlights were from: youtube.com/watch?v=7fPcse1phtk.

The true story of the search for planet Vulcan. And what it says about science.

Benjamin Keep, PhD, JD 2022-09-29 | The forgotten story of a mystery planet. With some opinions about what the story implies about the practice of science.

00:00 Introduction
00:46 Problem Number One: Uranus
1:50 Problem Number Two: Mercury
3:37 Something something provisional nature
4:23 Naive falsification (or disconfirmation)
6:03 Theory-ladenness of facts

What a great story, huh? You can find the basic story on Wikipedia.

But you might want to check out Thomas Levenson's book "The Hunt for Vulcan: ...And how Albert Einstein Destroyed a Planet, Discovered Relativity, and Deciphered the Universe" available here: bookshop.org/a/91541/9780812988307. I am a Bookshop.org affiliate and get a small commission if you purchase through the link.

It's not really the best example of the theory-ladenness of facts, but I feel like it's a good introduction to questions about how science works and how we know what we know.

Thanks for watching.

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

The Surprising Truth About Note-taking During Lectures

Benjamin Keep, PhD, JD 2022-09-21 | My argument on why the conventional wisdom on note-taking - particularly during lectures - is wrong.

00:00 Introduction
1:25 Does the writing process help us remember what we heard?
1:44 Does reviewing notes later help us remember?
2:07 The missing piece.
2:55 An alternative method.
3:56 Is taking verbatim notes useful?
5:02 On paying attention to the right things.
5:46 Taking notes during vs after a lecture
6:46 My recommended practice

Ryhan Hussain's channel: youtube.com/channel/UCg2u5o5dvS5iIu74rRO5x1w

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

References:

A lot of the recent research on note-taking has been on whether taking hand-written notes or laptop notes are superior. The evidence for one being better than the other is a bit "meh". As mentioned, the idea that "verbatim notes are bad" seems less true for laptop note-takers.

The opening quote comes from:

Flanigan, A. E., & Titsworth, S. (2020). The impact of digital distraction on lecture note taking and student learning. Instructional Science, 48(5), 495-524. bit.ly/3RiWQWE

The second quote comes from:

Morehead, K., Dunlosky, J., & Rawson, K. A. (2019). How much mightier is the pen than the keyboard for note-taking? A replication and extension of Mueller and Oppenheimer (2014). Educational Psychology Review, 31(3), 753-780. bit.ly/3CYdgzf

Experiment 2, here, is also the one that I relate where "no note-taking" performed just as well as any of the note-taking conditions on immediate tests. It's unclear to me why they didn't also use a delayed measure with the no note-taking group. Also, looking at tables 3 and 4, you can see that if students took notes on material germane to the test, then they did well (my second concluding point).

Another recent piece is here:

Luo, L., Kiewra, K. A., Flanigan, A. E., & Peteranetz, M. S. (2018). Laptop versus longhand note taking: effects on lecture notes and achievement. Instructional Science, 46(6), 947-971. bit.ly/3Qw24xb.

The introduction is a fair summary of the mixed evidence on the encoding hypothesis. Unfortunately, they only used immediate tests here, but there's some interesting material on the relative benefits of hand-written vs laptop notes.

Below is a more direct test of my suggested approach. The results? Note-taking beats testing on immediate tests, but in two weeks testing wins. Testing even beats note-taking + review, suggesting that the storage function of notes is not particularly strong.

Rummer, R., Schweppe, J., Gerst, K., & Wagner, S. (2017). Is testing a more effective learning strategy than note-taking?. Journal of Experimental Psychology: Applied, 23(3), 293. psycnet.apa.org/doiLanding?doi=10.1037%2Fxap0000134

And here is another that finds no difference, even though students who took notes during the lecture took more relevant notes than those who took notes after the lecture.

Haynes, J. M., McCarley, N. G., & Williams, J. L. (2015). An analysis of notes taken during and after a lecture presentation. North American Journal of Psychology, 17(1). bit.ly/3CUqAEL

And another old piece, same thing, no difference (only the measures weren't great).

Eisner, S., & Rohde, K. (1959). Note taking during or after the lecture. Journal of Educational Psychology, 50(6), 301–304. psycnet.apa.org/record/2005-10188-010

Certainly students seem to believe that note-taking during lectures improves learning outcomes:

Morehead, K., Dunlosky, J., Rawson, K. A., Blasiman, R., & Hollis, R. B. (2019). Note-taking habits of 21st century college students: implications for student learning, memory, and achievement. Memory, 27(6), 807-819. bit.ly/3cKGVBg

On the superiority of free recall / retrieval practice / testing, over re-study, see:

Roediger III, H. L., & Karpicke, J. D. (2006). The power of testing memory: Basic research and implications for educational practice. Perspectives on psychological science, 1(3), 181-210. https://linguistics.ucla.edu/people/hayes/Teaching/papers/2006_Roediger_Karpicke_Review.pdf

Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968. http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf

Karpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of memory and language, 57(2), 151-162. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdf

Three simple tricks to read textbooks more effectively

Benjamin Keep, PhD, JD 2022-09-14 | Textbooks are a slog. But part of the problem is that we read them like we read other things, and we shouldn't. Here's how to up your textbook reading game.

00:00 Introduction
00:38 Sipping and chugging
1:20 A technique for reading more actively
2:43 A technique for incorporating free recall practice

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

The textbook pages are from "University Physics with Modern Physics" by Young and Freedman. If you're interested in learning physics on your own, check out: susanrigetti.com/physics

References:

My recommendations here are a variety of what's called "spaced retrieval practice". There is a huge literature that spacing out study sessions (e.g., "sipping") provides long-term learning benefits.

Here's a meta-analysis on verbal recall tasks: Cepeda, N. J., Pashler, H., Vul, E., Wixted, J. T., & Rohrer, D. (2006). Distributed practice in verbal recall tasks: A review and quantitative synthesis. Psychological bulletin, 132(3), 354. escholarship.org/content/qt3rr6q10c/qt3rr6q10c.pdf

And here's a piece using realistic classroom controls. Seabrook, R., Brown, G. D., & Solity, J. E. (2005). Distributed and massed practice: From laboratory to classroom. Applied cognitive psychology, 19(1), 107-122. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.374.3437&rep=rep1&type=pdf

For my intro to this phenomenon, see: bit.ly/3BKHhl2.

My recommendation to do free recall practice prior to coming back to the textbook again comes from evidence on the efficacy of free recall in general (that's the "retrieval"). Here's some good papers on it:

Karpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of memory and language, 57(2), 151-162. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdf

Karpicke, J. D., & Aue, W. R. (2015). The testing effect is alive and well with complex materials. Educational Psychology Review, 27(2), 317-326. http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.721.6502&rep=rep1&type=pdf

For my intro to retrieval practice, see: bit.ly/3djdOFv.

My recommendation on reading the question before seeing the answer (e.g., "active" reading) is based on work in prior knowledge and illusions of knowing.

Example problems tell you things, and there is "a time for telling". Seeing the canonical answer before first think about what the problem is short-circuits the learning process. Here's a few papers on this idea:

Schwartz, D. L., & Bransford, J. D. (1998). A time for telling. Cognition and instruction, 16(4), 475-5223. http://aaalab.stanford.edu/papers/time_for_telling.pdf

Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of educational psychology, 103(4), 759. http://ece.neu.edu/edsnu/mcgruer/USC/PracticingVersusInventing2011-edu-103-4-759.pdf

There's also interesting work exploring the opposite order. See the paper below for a really good discussion:

Fyfe, E. R., DeCaro, M. S., & Rittle‐Johnson, B. (2014). An alternative time for telling: When conceptual instruction prior to problem solving improves mathematical knowledge. British journal of educational psychology, 84(3), 502-519. https://www.academia.edu/download/39309388/2014_FyfeDeCaroRittleJohnson_BJEP.pdf

How visualizations help you learn (and how to use them)

Benjamin Keep, PhD, JD 2022-09-07 | Do you create visualizations while you learn? If not, you should. Here's why.

00:00 Introduction
00:31 Three advantages to visualizations
2:15 What should you visualize?
2:59 The cardinal sin

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

Image acknowledgements:

The initial three visualizations come from the Dataisbeautiful subreddit. Here are the credits:

Long COVID in the U.S., by MetricT: reddit.com/r/dataisbeautiful/comments/x4hqfs/oc_percent_of_adult_population_currently

Brazil's GDP, by Latinometrics: reddit.com/r/dataisbeautiful/comments/x441dy/oc_in_2011_brazils_gdp_per_capita_was_27k_above

Mandela Effect, by YouGov_Official: reddit.com/r/dataisbeautiful/comments/x3z9ad/oc_americans_are_more_likely_to_hold_false_rather

Jacque Minard's famous illustration of Napoleon's campaign grabbed from http://activehistory.ca/wp-content/uploads/2012/02/Minard.png. Thanks, Active History!

Water cycle image: John Evans and Howard Periman, USGS, Public domain, via Wikimedia Commons. upload.wikimedia.org/wikipedia/commons/1/19/Watercyclesummary.jpg.

John Snow's map of the broad street water pump cholera outbreak (originally from 1854). en.wikipedia.org/wiki/John_Snow#/media/File:Snow-cholera-map-1.jpg.

Causal ecosystem image, from one of Tina Grotzer's papers. Grotzer, T. A., Kamarainen, A. M., Tutwiler, M. S., Metcalf, S., & Dede, C. (2013). Learning to reason about ecosystems dynamics over time: The challenges of an event-based causal focus. BioScience, 63(4), 288-296. academic.oup.com/bioscience/article/63/4/288/253365. She's an academic who does great work on how students think of causal relationships.

The "argument" image comes from a piece by Deanna Kuhn, a pioneer in studying argumentation. Kuhn, D., Goh, W., Iordanou, K., & Shaenfield, D. (2008). Arguing on the computer: A microgenetic study of developing argument skills in a computer‐supported environment. Child development, 79(5), 1310-1328. educationforthinking.org/sites/default/files/pdf/09%20Arguing%20on%20the%20computer.pdf

References:

This video was largely inspired by the chapter "V is for Visualization" in the ABCs of How We Learn by Schwartz, Tsang, and Blair. Definitely recommend it.

Why THIS is the best way to learn to ride a bike. | Intro to instructional scaffolding

Benjamin Keep, PhD, JD 2022-08-31 | Why are balance bikes effect for teaching kids how to bike ride? It is a perfect example of instructional scaffolding: a modification that helps students meaningfully participate in a target skill.

00:00 Introduction
00:38 What is instructional scaffolding?
01:28 A second advantage of balance bikes.
01:52 Figuring out the right scaffolding.

This is the video from the Global Cycling Network: youtu.be/UOg1hHmGIQQ

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

Reference

This is a great, thorough piece on scaffolding, that gives a more precise definition and discusses many (many!) issues that I didn't touch on here, but hope to in future videos:

Belland, B. R. (2017). Instructional scaffolding: foundations and evolving definition. In Instructional scaffolding in STEM education (pp. 17-53). Springer, Cham.
link.springer.com/chapter/10.1007/978-3-319-02565-0_2

What makes people great chess players? Cognitive scientist answers.

Benjamin Keep, PhD, JD 2022-08-11 | What creates a great chess player - their IQ, how they train, their personalities, their motivation? Here's my read of the existing research literature.

00:00 Introduction
00:41 Will having a high IQ make you good at chess?
03:12 Does playing chess improve your IQ?
4:15 The influence of training
5:41 The influence of personality and motivation
6:51 The cultural context

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

References:

The meta-analysis of IQ/cognitive ability with chess skill is here:

Burgoyne, A. P., Sala, G., Gobet, F., Macnamara, B. N., Campitelli, G., & Hambrick, D. Z. (2016). The relationship between cognitive ability and chess skill: A comprehensive meta-analysis. Intelligence, 59, 72-83. eprints.lse.ac.uk/102241/1/INTELL_2016_117_The_Relationship_between_Cognitive_Ability_and_Chess_Skill_a_Comprehensive_Meta_Analysis.pdf

The meta-analysis on whether chess playing increases IQ is here:

Sala, G., & Gobet, F. (2017). Does far transfer exist? Negative evidence from chess, music, and working memory training. Current directions in psychological science, 26(6), 515-520. journals.sagepub.com/doi/full/10.1177/0963721417712760

On the occasional reversal of the relationship between IQ and chess skill, see:

Bilalić, M., McLeod, P., & Gobet, F. (2007). Does chess need intelligence?—A study with young chess players. Intelligence, 35(5), 457-470. bura.brunel.ac.uk/bitstream/2438/642/1/Does%20Chess%20Need%20Intelligence-revision-finalINT.pdf

The meta-analysis on training and chess skill is here:

Charness, N., Tuffiash, M., Krampe, R., Reingold, E., & Vasyukova, E. (2005). The role of deliberate practice in chess expertise. Applied Cognitive Psychology, 19(2), 151-165. https://clinica.ispa.pt/ficheiros/areas_utilizador/user11/11_-_the_role_of_dp_in_chess_expertise.pdf

On chess and personality, check out:

Vollstädt-Klein, S., Grimm, O., Kirsch, P., & Bilalić, M. (2010). Personality of elite male and female chess players and its relation to chess skill. Learning and Individual Differences, 20(5), 517-521. researchgate.net/profile/Merim-Bilalic/publication/220027647_Personality_of_Elite_Male_and_Female_Chess_Players_and_its_Relation_to_Chess_Skill/links/5d49516492851cd046a5a25b/Personality-of-Elite-Male-and-Female-Chess-Players-and-its-Relation-to-Chess-Skill.pdf

The study linking chess skill, deliberate practice, and motivation is here:

De Bruin, A. B., Rikers, R. M. J. P., & Schmidt, H. G. (2007). The influence of achievement motivation and chess-specific motivation on deliberate practice. Journal of Sport and Exercise Psychology, 29(5), 561. researchgate.net/profile/Anique-De-Bruin/publication/271075262_The_Influence_of_Achievement_Motivation_and_Chess-Specific_Motivation_on_Deliberate_Practice/links/00b7d539064ca2cf43000000/The-Influence-of-Achievement-Motivation-and-Chess-Specific-Motivation-on-Deliberate-Practice.pdf

On the "domain-specificity" of expertise, the best resource is probably:

Ericsson, K. A., Hoffman, R. R., Kozbelt, A., & Williams, A. M. (Eds.). (2018). The Cambridge handbook of expertise and expert performance. Cambridge University Press.

But that's the best resource for a lot of the expertise literature! If you can't get your hands on it, here's an interesting article talking about how creativity (something that many people assume is domain-general) is actually quite domain-specific:

Baer, J. (2015). The importance of domain-specific expertise in creativity. Roeper Review, 37(3), 165-178. https://www.academia.edu/download/51367773/The_Importance_of_Domain_Specific_Expertise_in_Creativity.pdf

3 Simple Lessons in Learning and Attention | Cognitive Load, External Focus, Distraction, etc.

Benjamin Keep, PhD, JD 2022-07-27 | What does attention have to do with learning? A lot. Here's a very short summary of some of the more interesting findings in the research on attention and learning in three simple lessons.

00:00 Lesson one - four stories about attention
2:21 One way of thinking about attention and learning
3:07 Lesson two - what controls attention?
4:06 Lesson three - the roles of students and teachers

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

The artwork I mentioned in the example comes from:

Cut with the Kitchen Knife Dada Through the Last Weimar Beer-Belly Cultural Epoch of Germany by Hannah Höch (1919)

The Elephant Celebes by Max Ernst (1921)

Das Undbild by Kurt Schwitters (1919)

The complex scientific figure is kind of a cheat: no one would use that as a powerpoint slide. It's more of an attempt to make a complete map of metabolic pathways in human cells. I found it here: https://faculty.cc.gatech.edu/~turk/bio_sim/articles/metabolic_pathways.png

References:

The "someone else with a laptop distracts you" study is here:

Sana, F., Weston, T., & Cepeda, N. J. (2013). Laptop multitasking hinders classroom learning for both users and nearby peers. Computers & Education, 62, 24-31.https://www.sciencedirect.com/science/article/pii/S0360131512002254

A good review of internal/external focus in motor learning is here:

Wulf, G. (2013). Attentional focus and motor learning: a review of 15 years. International Review of sport and Exercise psychology, 6(1), 77-104. http://gwulf.faculty.unlv.edu/wp-content/uploads/2018/11/Wulf_AF_review_2013.pdf

Seductive details is a large research topic, but here's a recent meta-analysis that tries to synthesize the existing studies:

Sundararajan, N., & Adesope, O. (2020). Keep it coherent: A meta-analysis of the seductive details effect. Educational Psychology Review, 32(3), 707-734. link.springer.com/article/10.1007/s10648-020-09522-4

The "thinking 'are these the same or different'" example I drew from the literature on blocked and interleaved practice, which suggests that paying attention to similarities and paying attention to differences offers different affordances in category learning. See pages 2-3 in the following piece.

Carvalho, P. F., & Goldstone, R. L. (2014). Putting category learning in order: Category structure and temporal arrangement affect the benefit of interleaved over blocked study. Memory & cognition, 42(3), 481-495. https://pcl.sitehost.iu.edu/papers/blockedinterleaved.pdf

Good advice on reducing extraneous cognitive load in undergraduate science courses comes is here (with a couple of other good references in there):

cwsei.ubc.ca/sites/default/files/cwsei/resources/literature/Cognitive-Load_CWSEI.pdf

Some interesting research on cell phones and attention is here:

Thornton, B., Faires, A., Robbins, M., & Rollins, E. (2014). The mere presence of a cell phone may be distracting: Implications for attention and task performance. Social Psychology, 45(6), 479. oarklibrary.com/file/2/0c907a59-76b9-482a-b4a3-1c66294d7048/8d49e16c-14dc-4877-9c64-e75ba9bfdcc2.pdf

On manipulating student attention towards (or away from) deep structure, see (this is also where the clown image comes from):

Schwartz, D. L., Chase, C. C., Oppezzo, M. A., & Chin, D. B. (2011). Practicing versus inventing with contrasting cases: The effects of telling first on learning and transfer. Journal of educational psychology, 103(4), 759. http://ece.neu.edu/edsnu/mcgruer/USC/PracticingVersusInventing2011-edu-103-4-759.pdf

How to prepare for a final exam | High school and college

Benjamin Keep, PhD, JD 2022-07-13 | Oh crap. You’ve got a test. How are you going to study for it?

00:00 Step #1
1:31 Step #2
3:00 Step #3
5:04 Tip #1: How to use visualizations
5:40 Tip #2: On study groups
6:26 Tip #3: Be reasonable during step 3
7:03 Open vs. closed book exams
7:36 Multiple choice vs essay tests

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

My advice here is based on a few lines of literature.

Steps 1 and 3, which I recommend spending the most time on, comes from the research on free recall and the testing effect. These are just better study than rereading or re-writing your notes. Typical research findings are here:

Karpicke, J. D., & Blunt, J. R. (2011). Retrieval practice produces more learning than elaborative studying with concept mapping. Science, 331(6018), 772-775. mrbartonmaths.com/resourcesnew/8.%20Research/Memory%20and%20Revision/Retrieval%20practice%20more%20effective%20than%20studying.pdf

Karpicke, J. D., & Roediger III, H. L. (2008). The critical importance of retrieval for learning. science, 319(5865), 966-968.
http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Karpicke-Roediger-2008_Sci.pdf

Karpicke, J. D., & Roediger III, H. L. (2007). Repeated retrieval during learning is the key to long-term retention. Journal of Memory and Language, 57(2), 151-162.
https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.558.9401&rep=rep1&type=pdf

Yang, C., Luo, L., Vadillo, M. A., Yu, R., & Shanks, D. R. (2021). Testing (quizzing) boosts classroom learning: A systematic and meta-analytic review. Psychological Bulletin. (currently available at gwern.net/docs/spacedrepetition/2021-yang.pdf)

An applied research piece exploring the application of tests in the classroom. They tested both short-answer and multiple-choice tests, finding that short answer tests outperformed multiple choice tests (though both outperformed additional reading).

McDaniel, M. A., Anderson, J. L., Derbish, M. H., & Morrisette, N. (2007). Testing the testing effect in the classroom. European journal of cognitive psychology, 19(4-5), 494-513. (currently available at http://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.1061.366&rep=rep1&type=pdf)

A classic study demonstrating long-term benefits (1 month).

Butler, A. C., & Roediger III, H. L. (2007). Testing improves long-term retention in a simulated classroom setting. European Journal of Cognitive Psychology, 19(4-5), 514-527. (currently available at http://psychnet.wustl.edu/memory/wp-content/uploads/2018/04/Butler-Roediger-2007_EJCP.pdf)

My advice on the "outstanding questions" box is based on the literature on self-questioning and interrogation. It’s also supposed to help you with metacognitive knowledge (which helps you study more effectively because you know what you already understand and what you’re still struggling with). Example research findings are here:

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the public interest, 14(1), 4-58. https://bibliotecadigital.mineduc.cl/bitstream/handle/20.500.12365/17388/dunloskyimprovinglearning.pdf?sequence=1 (The first two sections on elaborative interrogation and self-explanations are relevant here.)

Rosenshine, B., Meister, C., & Chapman, S. (1996). Teaching students to generate questions: A review of the intervention studies. Review of educational research, 66(2), 181-221. jgregorymcverry.com/readings/rosenshine1996StudentQuestioning.pdf (In most of these studies people are teaching students how to ask questions, which is an important part of the process.)

King, A. (1992). Comparison of self-questioning, summarizing, and notetaking-review as strategies for learning from lectures. American Educational Research Journal, 29(2), 303-323. https://citeseerx.ist.psu.edu/viewdoc/download?doi=10.1.1.859.5957&rep=rep1&type=pdf

My advice on using visualizations comes from research on learning and visualizations (well summarized in The ABCs of Learning, citation below). Basically, they offload cognitive burden, they can be more specific than words (relating ideas and concepts in more precise ways), and they can help you identify patterns that you might not have recognized before.

Schwartz, D. L., Tsang, J. M., & Blair, K. P. (2016). The ABCs of how we learn: 26 scientifically proven approaches, how they work, and when to use them. WW Norton & Company. (I’m referring to “V is for Visualization”. Great introduction to the research on learning more generally.)

There are other ways of studying for tests, of course. But I think this is a pretty solid default method.

What is intuition? And where does it come from?

Benjamin Keep, PhD, JD 2022-07-06 | We all experience intuitions. And they can seem mysterious and wonderful. The psychological research suggests intuitions are the outcome of a fundamental feature of our minds: two parallel learning systems.

00:00 Introduction
00:28 Two learning systems
01:02 One source of intuition
01:42 Another source of intuition
02:46 Can you trust your intuition? Should you?

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

The thing that kills creativity | Psychology, insight, and problem solving.

Benjamin Keep, PhD, JD 2022-06-29 | What makes us creative problem solvers? The answer takes more than one video. But here are the results of some studies exploring how people solve physics problems: do they see the creative shortcut? Or are they stuck working out the problem in the standard way?

00:00 Introduction
00:26 What are force diagrams?
00:48 The study and its results
01:39 Explanation one
01:57 Explanation two
02:42 Why draw a diagram?

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

References:

Kuo, E., Hallinen, N. R., & Conlin, L. D. (2017). When procedures discourage insight: epistemological consequences of prompting novice physics students to construct force diagrams. International Journal of Science Education, 39(7), 814-839. tandfonline.com/doi/abs/10.1080/09500693.2017.1308037

Heckler, A. F. (2010). Some consequences of prompting novice physics students to construct force diagrams. International Journal of Science Education, 32(14), 1829-1851.
researchgate.net/profile/Andrew-Heckler/publication/233366500_Some_Consequences_of_Prompting_Novice_Physics_Students_to_Construct_Force_Diagrams/links/0deec53220b7f7ae2c000000/Some-Consequences-of-Prompting-Novice-Physics-Students-to-Construct-Force-Diagrams.pdf

Kuo, E., Hallinen, N. R., & Conlin, L. D. (2015, December). How prompting force diagrams discourages student use of adaptive problem-solving shortcuts. In The Physics Education Research Conference 2015.
researchgate.net/profile/Eric-Kuo/publication/299701476_How_prompting_force_diagrams_discourages_student_use_of_adaptive_problem-solving_shortcuts/links/57e3f25d08ae8d5908c162d1/How-prompting-force-diagrams-discourages-student-use-of-adaptive-problem-solving-shortcuts.pdf

Why are chess masters so much better than you?

Benjamin Keep, PhD, JD 2022-06-22 | What makes great chess players so great? What's going on in their heads? Do they just have great memories? How far do they look ahead?

00:00 Introduction
00:24 The early experiments
1:46 A brain superpower
3:12 Depth of search
4:33 Human experts vs AI chess programs

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

References:

My summarization of the memory results is really recounting this recent study:

Smith, E. T., Bartlett, J. C., Krawczyk, D. C., & Basak, C. (2021). Are the advantages of chess expertise on visuo-spatial working-memory capacity domain specific or domain general?. Memory & Cognition, 49(8), 1600-1616.https://www.researchgate.net/profile/Chandramallika-Basak-2/publication/351245327_Are_the_Advantages_of_Chess_Expertise_on_Visuo-Spatial_Working_Memory_Capacity_Domain_Specific_or_Domain_General/links/609036d0a6fdccaebd073806/Are-the-Advantages-of-Chess-Expertise-on-Visuo-Spatial-Working-Memory-Capacity-Domain-Specific-or-Domain-General.pdf

But there's lots of other research along the same lines.

The earliest research was here:

de Groot, A. D. (1965). Thought and choice in chess. The Hague: Mouton Publishers.

Chase, W. G., & Simon, H. A. (1973). The mind's eye in chess. In Visual information processing (pp. 215-281). Academic Press. sciencedirect.com/science/article/pii/B9780121701505500111

And here are some other solid articles I looked at:

Charness, N. (1992). The impact of chess research on cognitive science. Psychological research, 54(1), 4-9. link.springer.com/article/10.1007/BF01359217

Bilalić, M., McLeod, P., & Gobet, F. (2008). Expert and “novice” problem solving strategies in chess: Sixty years of citing de Groot (1946). Thinking & Reasoning, 14(4), 395-408. researchgate.net/profile/Fernand-Gobet/publication/49401082_Expert_and_novice_problem_solving_strategies_in_chess_Sixty_years_of_citing_de_Groot_1946/links/57651cbb08ae1658e2f46b20/Expert-and-novice-problem-solving-strategies-in-chess-Sixty-years-of-citing-de-Groot-1946.pdf?_sg%5B0%5D=started_experiment_milestone&origin=journalDetail

Connors, M. H., Burns, B. D., & Campitelli, G. (2011). Expertise in complex decision making: The role of search in chess 70 years after de Groot. Cognitive science, 35(8), 1567-1579. onlinelibrary.wiley.com/doi/full/10.1111/j.1551-6709.2011.01196.x

What Sherlock Holmes Got Wrong | Deduction, Induction, and Abduction

Benjamin Keep, PhD, JD 2022-06-15 | Sherlock Holmes is famous for his deductions. But what if they're not deductions at all?

00:00 Introduction
00:24 Deduction
00:57 Do deductions move from the general to the particular?
01:39 The key characteristic of deductions
02:21 Induction
02:45 Do inductions move from the particular to the general?
03:21 The key characteristic of inductions
04:26 A special kind of induction
05:35 Semantics?

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

Reference:

The best discussion of the distinction between the three kinds of reasoning I outlined here comes from "How We Reason" by Philip Johnson-Laird - cognitive scientist extraordinaire. Chapter 13 is on Sherlock and abductions. It's his model that I'm describing.

The clips are from the BBC series "Sherlock," starring Benedict Cumberbatch. Used here for educational purposes.

What no one tells you about learning faster.

Benjamin Keep, PhD, JD 2022-06-08 | Should you try to 10X your learning? Probably not. There are lots of cases when prioritizing learning speed early on in the learning process backfires, ultimately resulting in poorer learning in the long run. Here are three.

00:00 Introduction
00:21 Three stories
01:32 The catch
01:56 The beginnings of an explanation

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com
References:

I initially came across the "mirror learning" example on page 310 of this book:

Magill, R., & Anderson, D. (2010). Motor learning and control. New York: McGraw-Hill Publishing. https://www.academia.edu/download/62953224/Motor_Learning_and_Control__Concepts_and_-_Anderson__David20200414-28877-18hp4j2.pdf

This reference has a fuller description of the idea (though is testing a slightly different hypothesis):

Tremblay, L., & Proteau, L. (1998). Specificity of practice: The case of powerlifting. Research quarterly for exercise and sport, 69(3), 284-289. researchgate.net/profile/Luc-Tremblay-2/publication/13507905_Specificity_of_Practice_The_Case_of_Powerlifting/links/5805b11608aef179365e73b4/Specificity-of-Practice-The-Case-of-Powerlifting.pdf

Note that learning with a mirror doesn't always impair performance without a mirror (or improve skill acquisition during practice). (See for instance: Lynch, J. A., Chalmers, G. R., Knutzen, K. M., & Martin, L. T. (2009). Effect on performance of learning a Pilates skill with or without a mirror. Journal of Bodywork and Movement Therapies, 13(3), 283-290. sciencedirect.com/science/article/abs/pii/S1360859208001666)

The math feedback example comes from this study:
Fyfe, E. R., & Rittle-Johnson, B. (2017). Mathematics practice without feedback: A desirable difficulty in a classroom setting. Instructional Science, 45(2), 177-194. link.springer.com/article/10.1007/s11251-016-9401-1

The Spanish word study is at:

Bahrick, H. P. (1979). Maintenance of knowledge: Questions about memory we forgot to ask. Journal of Experimental Psychology: General, 108(3), 296. psycnet.apa.org/record/1981-00448-001

But I'm basing my discussion on the description of the study as made by Robert Bjork, here:

Bjork, R. A. (1994). Memory and metamemory considerations in the. Metacognition: Knowing about knowing, 185(7.2). It's really such a great article; one that I've revisited many times. Get in touch if you want a pdf copy.

Forgetting doesnt work like you think.

Benjamin Keep, PhD, JD 2022-06-01 | Everyone forgets things. We forget what we read, what we study, what we learn. But where do our memories go, after we've forgotten them? And why did we forget in the first place?

00:00 Introduction
00:30 Memories do battle with each other
1:55 A second way that memories interfere with one another

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

References:

A great discussion of forgetting (especially theories of cue competition) comes from:

Anderson, M. C., & Neely, J. H. (1996). Interference and inhibition in memory retrieval. In Memory (pp. 237-313). Academic Press. http://www.memorycontrol.net/an1996.pdf

The "drinking helps you remember" idea has a relatively long history.

Here's an earlier replication of some of the first work:

Mueller, C. W., Lisman, S. A., & Spear, N. E. (1983). Alcohol enhancement of human memory: Tests of consolidation and interference hypotheses. Psychopharmacology, 80(3), 226-230. https://www.academia.edu/download/46474229/bf0043615820160614-5333-a2occx.pdf

And here's a more recent piece:

Moulton, P. L., Petros, T. V., Apostal, K. J., Park II, R. V., Ronning, E. A., King, B. M., & Penland, J. G. (2005). Alcohol-induced impairment and enhancement of memory: A test of the interference theory. Physiology & behavior, 85(3), 240-245. pubag.nal.usda.gov/download/46671/pdf

The same is true, by the way of other drugs that impair memory formation. They tend to facilitate memory for events and information that happened prior to impairment.

A Simple Study Tip: Stop Highlighting

Benjamin Keep, PhD, JD 2022-05-25 | Highlighting your textbook is how most students know they're being diligent. And there is no shortage of advice on how to highlight. But is it really worth the trouble? Nah. Here's some thoughts on why.

00:00 Introduction
00:35 Do we know how to highlight?
01:39 What does it encourage us to do?
02:08 What does it tell us about what we know?

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com

The onscreen quotes were pulled from a classic review of study techniques:

Dunlosky, J., Rawson, K. A., Marsh, E. J., Nathan, M. J., & Willingham, D. T. (2013). Improving students’ learning with effective learning techniques: Promising directions from cognitive and educational psychology. Psychological Science in the Public interest, 14(1), 4-58. https://bibliotecadigital.mineduc.cl/bitstream/handle/20.500.12365/17388/dunloskyimprovinglearning.pdf?sequence=1

5 Things Students Should Know About Learning

Benjamin Keep, PhD, JD 2022-05-18 | We can spend a lot of our lives as students with misperceptions about learning. Here are some things to keep in mind throughout your career as a student.

00:00 Who are you working for?
00:50 Effort over the long run.
01:12 How should we think about mistakes and confusion?
02:13 Spending your time more effectively.
03:12 The pretty principle.

Sign up to my email newsletter, Avoiding Folly, here: benjaminkeep.com