Uploaded May 2023 | Updated September 2026, 3 weeks ago
It's a bit high-school-play level right now, but I think it has potential with (meaningful) effort!
Children-guided story intro written by GPT-4:
1. GPT-4 described the background audio and sound effects as a prompt to give to musiclm
2. Generated the music via musiclm
3. Generated the voices via TortoiseTTS
4. GPT-4 prompts to generate Midjourney images for various timestamps (results chosen by me)
5. Placed together in an unreal engine 5 scene with spatial audio (very amateur, this could be done significantly better - my first time doing it!)
6. Exported as a short story intro
All content was generated via various ML models playing with one another, but stitched together via a human. I expect each step could be improved and automated *significantly* further, and potentially approach real-enough-time to tell a story or a 3d audio podcast with sound effects and scene reconstructions that was compelling.
It's a bit high-school-play level right now, but I think it has potential with (meaningful) effort!
Children-guided story intro written by GPT-4:
1. GPT-4 described the background audio and sound effects as a prompt to give to musiclm
2. Generated the music via musiclm
3. Generated the voices via TortoiseTTS
4. GPT-4 prompts to generate Midjourney images for various timestamps (results chosen by me)
5. Placed together in an unreal engine 5 scene with spatial audio (very amateur, this could be done significantly better - my first time doing it!)
6. Exported as a short story intro
All content was generated via various ML models playing with one another, but stitched together via a human. I expect each step could be improved and automated *significantly* further, and potentially approach real-enough-time to tell a story or a 3d audio podcast with sound effects and scene reconstructions that was compelling.
