Uploaded May 2023 | Updated September 2026, 1 week ago
About the speaker
Prof Russell "Russ" Poldrack is a Professor of psychology at Stanford University. See more information https://poldracklab.stanford.edu/
About the talk
Artificial intelligence (AI) tools based on large language models have achieved human-level performance on some computer programming tasks. We report several experiments using GPT-4 to generate computer code. These experiments demonstrate that AI code generation using the current generation of tools, while powerful, requires substantial human validation to ensure accurate performance. We also demonstrate that GPT-4 refactoring of existing code can significantly improve that code along several established metrics for code quality, and we show that GPT-4 can generate tests with substantial coverage, but that many of the tests fail when applied to the associated code. These findings suggest that while AI coding tools are very powerful, they still require humans in the loop to ensure validity and accuracy of the results.
About the RIOTS Club
Visit riotscience.co.uk or Twitter @riotsciencenclub. To subscribe to our mailing list, please email riotscienceclub@kcl.ac.uk.
About the speaker
Prof Russell "Russ" Poldrack is a Professor of psychology at Stanford University. See more information https://poldracklab.stanford.edu/
About the talk
Artificial intelligence (AI) tools based on large language models have achieved human-level performance on some computer programming tasks. We report several experiments using GPT-4 to generate computer code. These experiments demonstrate that AI code generation using the current generation of tools, while powerful, requires substantial human validation to ensure accurate performance. We also demonstrate that GPT-4 refactoring of existing code can significantly improve that code along several established metrics for code quality, and we show that GPT-4 can generate tests with substantial coverage, but that many of the tests fail when applied to the associated code. These findings suggest that while AI coding tools are very powerful, they still require humans in the loop to ensure validity and accuracy of the results.
About the RIOTS Club
Visit riotscience.co.uk or Twitter @riotsciencenclub. To subscribe to our mailing list, please email riotscienceclub@kcl.ac.uk.
