How Whisper Turns Speech Into Text
Whisper is the model that powers the transcription here. It runs entirely in your browser, but the ideas behind it are the same whether it runs locally or in the cloud. Understanding them at a high level explains why some meetings transcribe cleanly and others need editing.
Audio becomes a picture of sound
The first step is not language at all. The recording is chopped into short slices and turned into a spectrogram โ a visual representation of sound that shows which frequencies are present at each moment. Loud and quiet, high and low, voice and hiss all become patterns the model can read. The model never "hears" your voice the way a person does; it reads this picture of the sound.
The decoder turns patterns into words
A transformer decoder reads the spectrogram and predicts what was said, one piece at a time. It does not look up words in a dictionary โ it has learned, from training data, which sound patterns tend to follow which other sound patterns in speech. Output comes out as text, with punctuation, because the model learned that pauses and intonation usually line up with sentence boundaries.
It works in chunks
Long recordings are processed in thirty-second slices with a small overlap, so the model never has to hold an entire hour of audio in memory at once. That is why a long meeting takes proportionally longer but works the same way as a short one, and why the tool can transcribe a file far larger than your browser tab could otherwise hold.
Why accuracy varies
The model is only as good as the recording it receives. Clear, close-mic speech at a steady volume transcribes best. Background noise competes with the voice in the spectrogram, and when two people speak at once their patterns overlap โ the model has to untangle them, and sometimes it cannot. Accents matter because the training data contains some pronunciations far more often than others; a voice the model has seen rarely is a voice it guesses about more.
No magic, just maths
Whisper does not understand your meeting. It does not know who the decision-makers are, which topic matters most, or whether someone was joking. It produces a text rendering of the sounds it was given. That is exactly why it is safe to run locally: it is a pattern-matcher on a spectrogram, not a listener with an opinion about what was said.
Transcription is different from understanding
Because Whisper renders sounds into words without knowing what they mean, it can produce a perfect transcript of a meeting whose significance it has no grasp of. It cannot tell you which remark was the important one, which promise was casual and which was serious, or whether a decision was unanimous or forced. Those judgements still belong to the people who were in the room โ which is why the tool produces notes, not verdicts.
Language coverage comes from training data
Whisper was trained on a very large collection of speech in many languages, so it can transcribe non-English audio as well as English. Coverage is not even: languages that appear more in the training data tend to transcribe more accurately, and a language that appears rarely will produce more guesses. The multilingual model handles many languages at once; the English-only models focus on one and do it faster.
What makes a spectrogram hard to read
The spectrogram is a picture, but a messy one. When several sounds occupy the same time and frequency range at once, their patterns overlap and the model has to guess which pattern belongs to which speaker. Reverberation โ a voice bouncing off a hard-walled conference room โ smears sound across neighbouring time slices, and long-term reverberation is one of the hardest things for any transcription model to undo. A quieter, drier room is easier for the model to read than a noisy one, even when the speech is the same.