Uploaded December 2023 | Updated September 2026, 2 weeks ago

---

In this vlog, I, Alex Raccuglia, discuss a significant issue related to Whisper, the open-source model released by OpenAI, which is widely used for speech-to-text transcription, including in the app I produce called Transcriber. I express my admiration for Whisper but highlight a critical problem in its architecture.

I delve into how Whisper transcribes audio in batches of 30 seconds, which leads to delays in generating transcriptions and occasionally results in the loss of punctuation. I explain that the default prompt length for Whisper is 224 tokens, roughly equivalent to 150 words. After the initial prompt, Whisper may lose the context, especially for longer audio files, resulting in the omission of punctuation marks.

I share my attempt to address this issue by dividing longer audio files into smaller blocks for individual analysis, but this approach has its challenges, including significant computation time. Additionally, I detail a user's experience with a 42-minute audio file divided into 172 blocks, leading to a prolonged transcription process.

I express frustration about the lack of a solution to retain the initial prompt after the first 150 words and emphasize that this limitation affects the quality of Transcriber, my app. I mention the possibility of using OpenAI GPT 3.5, 4, or 4.5 to manually add punctuation marks to the transcriptions but acknowledge the potential cost to users.

I conclude the vlog by sharing my feelings of sadness and the realization that finding a solution to this problem is challenging. I encourage viewers, especially engineers, to comment and engage in a discussion about potential solutions or alternative perspectives. Despite the difficulties presented by this issue, I remain open and transparent about my exploration and search for a resolution.