Whisper API

The model you can also run yourself

Audio processing covers the paths audio takes into a model and the limits on getting it there. Speech to text covers choosing a transcription model and measuring whether it works.

Whisper is worth its own page for one reason the others do not cover: it is open weights. You can call it as a hosted endpoint, or download it and run it on your own hardware, and that choice has consequences the API-only models do not present.

Hosted, as whisper-1

Through OpenAI's API, Whisper is the model named whisper-1, and it has a specific niche now that newer transcription models exist.

It is the one that carries timestamps, translation and subtitle output. Newer models transcribe better and do not all do those things, so whisper-1 remains the choice for captioning work rather than being the legacy option. That is the opposite of what a version number suggests, and speech to text sets out the split across the current line.

Its prompt parameter accepts up to 224 tokens of vocabulary, and its language parameter is language, singular, where the newer model takes languages. A config shared across both will be wrong for one of them.

Self-hosted, and why you would

Running it yourself changes four things, and only one of them is cost.

The audio never leaves. This is the main reason teams do it. Recordings of customer calls, clinical conversations or interviews often cannot go to a third party, and no per-token discount addresses that. Context security covers the wider version of this question.

No file-size ceiling. The hosted endpoint has a 25 MB limit that forces splitting, which audio processing covers in detail. Locally the constraint is your own memory and patience.

The model is frozen. It transcribes the same way next year, which matters if you have to demonstrate a consistent process.

The cost curve inverts. Per-minute pricing becomes a GPU you have paid for whether it runs or not. As closed vs open source models argues, that only wins above a utilization line most workloads sit below.

Sizes are a real trade, not a tier list

The released variants run from small to large, and the choice is about the deployment rather than about quality alone.

The small end runs on a CPU or a modest GPU, fast and roughly real-time, and it makes noticeably more errors on accented speech, crosstalk and domain vocabulary. The large end needs a real GPU and is much better on exactly those cases.

The way to pick is not to read a benchmark. It is to take fifty clips of your own audio, run them through two sizes, and measure the fields your pipeline extracts rather than the word error rate, for the reasons speech to text gives. Many teams find a mid-size model is sufficient, and the ones who do not find it out in production.

The operational reality of running it

Three things that are easy to underestimate.

Throughput is a queue problem. One GPU transcribing one file at a time is not the same as an endpoint that absorbs bursts. You now own the queue, the retries and the back-pressure.

Long audio still needs splitting, just for memory rather than for a size limit, and the sentence-boundary problem is unchanged.

Nothing improves on its own. A hosted model gets better without you. A downloaded one is exactly as good in two years as it is today, which is the flip side of being frozen.

Choosing between the two

Hosted, unless the audio cannot leave your network, you have volume high enough to beat per-minute pricing, or you need a model you can pin. Those three cover almost every legitimate case for self-hosting, and none of them is "it will be cheaper", which is the reason most often given and the one least often true.

Further reading

• Speech to text: the model split, the parameters that fix accuracy, and what to measure.
• Audio processing: the size budget, splitting, timestamps and speakers.
• Self-hosted models: what running your own inference involves.
• Closed vs open source models: the licensing and control axis in general.