OpenAI-Compatible APIs

One schema everyone implements

POST /v1/chat/completions stopped being OpenAI's endpoint and became the industry's wire format. Local runtimes speak it, hosted competitors speak it, routers speak it, and inference servers speak it.

The practical effect is that a client written against the OpenAI SDK can often be pointed somewhere else by changing two things:

const client = new OpenAI({
  baseURL: 'http://localhost:11434/v1',
  apiKey: 'ollama',
})

That is genuinely useful. It is also the reason the word "compatible" does more work than it should.

Compatible means a subset, and the projects say so

Ollama's documentation states the position plainly: "Ollama supports a subset of the OpenAI API." vLLM describes an HTTP server "compatible with many interfaces" and makes no broader guarantee.

Both publish what is missing, and the lists are specific.

Ollama, on chat completions, does not support tool_choice, logprobs, n, logit_bias or user. Image URLs are unsupported; only base64 works.

vLLM ignores user on chat completions and does not support suffix on completions. It also adds a parameter the OpenAI schema does not have, parallel_tool_calls, to cap a response at one tool call.

Read those two lists together and the shape of the problem appears. Fields do not fail; they fall away.

The failures are silent, and that is the point

Nothing above returns an error. An unsupported field is accepted and ignored, which is the compatible thing to do and the worst thing for you.

tool_choice is the example worth holding onto. On OpenAI it forces the model to call a specific tool. On Ollama it is unsupported, so a request that depends on a tool being called becomes a request that suggests one. Your code waits for a tool call that may never come, and the only symptom is an application that works against one backend and behaves oddly against another.

The same applies to n, to logprobs, and to anything else a library sets on your behalf. A client SDK that quietly sends user for telemetry is fine; a retry policy keyed on logprobs is not.

Where the ceiling is

Compatibility is strongest at the centre and weakest at the edges, and the edges are where the money and the behavior live.

The common path ports. Messages, temperature, max_tokens, streaming, basic tool declarations. If your application is a chat call with a system instruction, it will move.

Provider-specific features do not. Prompt caching semantics, thinking and how it is returned, safety settings, citations, server-side tools, stop reasons beyond the common set. These have no place in the shared schema, so a compatible endpoint either omits them or invents its own parameter, which is exactly what parallel_tool_calls is.

The newer endpoint is patchier. /v1/responses is being implemented too: vLLM lists it, and so does Ollama. But it is newer, less widely implemented, and the thing the ecosystem standardised on is still Chat Completions. OpenAI Responses API covers what you give up by staying on the older one.

Using it without being surprised

Name the fields you depend on. Write down which request parameters your behavior requires. That list is what you check against a new backend, and it is short.

Assert the behavior, not the response. A test that a call succeeds proves nothing, because an ignored field succeeds. Test that the tool was called, that the output parsed, that the run stopped for the reason you expected.

Check the stop reason across backends. It is the field most likely to differ in spelling or in meaning, and the one most likely to be load-bearing in your code.

Treat the base URL as a deployment choice, not a runtime one. Swapping providers per request sounds like flexibility and means your behavior depends on something your tests probably fix.

OpenRouter takes the same idea to its conclusion, one endpoint over many providers, and runs into the same boundaries in more detail.

Further reading

• Ollama, OpenAI compatibility: the supported and unsupported fields, stated as a subset.
• vLLM, Online serving: the endpoints implemented, the ignored parameters, and the added one.
• OpenAI Responses API: the endpoint the ecosystem has not finished copying.
• OpenRouter: one surface over many providers, and where the abstraction ends.