Text to Speech

The silence before the first word

Ask a speech API for a paragraph and it returns an audio file. Simple, and it puts a wait in front of your user proportional to the length of what you asked for.

For a notification that is fine. For a conversation it is the whole experience: a voice assistant that pauses for two seconds before every reply feels broken in a way that a chat interface showing a typing indicator does not.

Most of the engineering in speech output is about that gap.

The call, and why format is a latency decision

You send a model, the text, and a voice. You get audio back, MP3 by default.

Six formats are available: MP3, Opus, AAC, FLAC, WAV and PCM. The choice is not only about size. OpenAI recommends wav or pcm "for the fastest response times", because those are uncompressed and need no container-level decoding before playback begins.

So the rule of thumb runs opposite to the usual instinct. For files you store and serve later, use a compressed format. For audio you are about to play, take the uncompressed one and spend the bandwidth.

Streaming audio, and what it forces you to build

The API supports real-time streaming through chunked transfer encoding, so playback can begin before generation finishes.

Adopting it means building three things you did not need for a file.

A playback buffer. Audio arrives in chunks and plays continuously, so you need enough buffered to survive a slow chunk. Too little and playback stutters; too much and you have reintroduced the delay you were removing.

Failure handling after playback starts. A stream can fail halfway, which leaves you with a voice that stops mid-sentence. This is the same shape as the error that arrives after a successful response in a text stream, and it is worse here, because a truncated sentence in text is visibly truncated while audio stops with no visible edge at all.

A cancellation path. Users interrupt. When they do, you want synthesis to stop and billing to stop with it, which means the request has to be abortable rather than fire-and-forget.

Pairing speech synthesis with a streaming model response

Here is the part with no documented answer, because it sits between two systems.

Your model streams text in fragments that have nothing to do with sentences. A delta might be " the", or " refund win". Your speech API wants an utterance: enough text to produce natural prosody, because intonation depends on where the sentence is going.

Synthesize each fragment and you get robotic, disconnected speech. Wait for the whole response and you have given up streaming.

The practical middle is to buffer text until you have a natural boundary, then synthesize that unit while the next one accumulates. Sentence-ending punctuation is the obvious boundary, with a length cap so a long sentence does not stall playback, and a flush when the text stream ends.

Two details that bite. Abbreviations and decimals contain periods that are not sentence ends, so a naive split on . will chop "version 2.5" into two utterances. And the seam between units is audible if each is synthesized independently, because the voice resets its prosody, so shorter units mean smoother latency and choppier delivery.

This is a design reasoned from how the two APIs behave rather than a documented recipe, and it is worth testing with your own content before committing to a boundary rule.

Disclosure is a requirement, not a courtesy

OpenAI's usage policies "require you to provide a clear disclosure to end users that the TTS voice they are hearing is AI-generated and not a human voice."

That is an obligation attached to using the API, so build the disclosure into the product rather than treating it as a decision to revisit. Where it goes depends on the surface: a line in the interface, a statement at the start of a call, or something in onboarding for a returning user.

The same care applies to voice choice. Synthesizing a voice that resembles a specific real person raises consent questions the API terms do not settle for you, and the safe position is a provided voice rather than one built to sound like someone.

Further reading

• OpenAI, Text to speech: formats, the streaming support, voice options, and the disclosure requirement.
• Streaming responses: the text stream this has to align with.
• Speech to text: the other direction.