Speech to Text

Whether anyone is waiting

Audio processing covers the two paths audio can take into a model, the size budget, and how to split a long recording. This page is about the transcription path specifically: choosing a model, getting the words right, and knowing whether it worked.

The first decision is not quality. It is whether a person is waiting for the text.

Batch sends a complete file and waits. Right for recordings, voicemail, uploaded media. The latency is the length of the audio plus processing, and nobody notices because nobody is watching.

Streaming returns text while the audio is still arriving. OpenAI supports it with stream=true, emitting transcript.text.delta events as it goes and transcript.text.done at the end. Right for live captions, a voice interface, a meeting assistant showing text as people speak.

Streaming is a different application, not a faster call. You hold a connection open, you render partial text, and you handle that text changing under the reader as later audio resolves an earlier ambiguity. Budget for the interface work, not just the API change.

Three models, split by capability rather than tier

OpenAI's current line does not run from cheap to good. Each model exists for a different job:

The documentation is direct about the middle one: use it "only when you need to identify who speaks". Diarization is not a free upgrade.

The split has a consequence worth planning around. Timestamps live on whisper-1, speaker labels live on the diarize model, and general transcription lives on a third. A captioning pipeline that also needs speakers is two calls or a compromise, which audio processing covers in more detail.

The two parameters that fix most accuracy complaints

Before changing model, change these.

Tell it the language. An unnecessary guess is an unnecessary error. The parameter name differs by model, which is easy to get wrong in a config shared across both: gpt-transcribe takes languages, plural, accepting ISO 639-1 and 639-3 codes and regional Chinese variants; whisper-1 takes language, singular.

Give it your vocabulary. The prompt is where the words the model has never heard go: product names, internal acronyms, the surnames of people in the meeting, the drug or part numbers your domain runs on. whisper-1 accepts up to 224 tokens of it; gpt-transcribe accepts longer context.

Most reports that transcription is "bad" turn out to be a domain-vocabulary problem. A list of twenty proper nouns in the prompt usually moves accuracy more than any model change, and costs nothing.

Word error rate measures the wrong thing for most products

The standard measure is word error rate: the share of words wrong against a reference transcript. It is well defined, widely reported, and treats every word as equally important, which your product does not.

A transcript rendering "um" as "hmm" takes the same penalty as one getting the customer's account number wrong. If a downstream step extracts a handful of fields, measure those fields. A 12% word error rate with every order number correct is a working system. A 4% rate with the numbers wrong is not.

So build two numbers:

• Field accuracy, on whatever your pipeline extracts. This is the one that tracks user-visible quality.
• Word error rate, as a regression signal. It catches a model change or an audio-pipeline change that field accuracy might absorb.

Build the reference set from your own recordings. Accent, recording quality, crosstalk, background noise and domain vocabulary all move the number, and none of them appear in a published benchmark. Fifty real clips with hand-checked transcripts will tell you more than any leaderboard.

Where the transcript goes next

A transcript is rarely the deliverable. Something reads it afterwards, and that step inherits every error.

Two habits make that survivable. Keep the audio, so a disputed extraction can be checked against the source rather than argued about. And pass the transcript onward as text the next step is told is a transcript, so a garbled proper noun reads as noise rather than as a fact: a model handed a clean-looking string will treat it as clean.

Further reading

• OpenAI, Speech to text: the three models, streaming events, and the language and prompt parameters.
• Audio processing: the two paths, the size budget, splitting, timestamps and diarization.
• Whisper API: the provider page for that model family.
• Text to speech: the other direction.