Transformers.js

Inference with no server

Transformers.js runs Hugging Face models in the browser. No backend, no API key, no request leaving the page.

The API mirrors the Python library closely enough to read at a glance:

import { pipeline } from '@huggingface/transformers'

const pipe = await pipeline('sentiment-analysis')
const out = await pipe('I love transformers!')
// [{ label: 'POSITIVE', score: 0.999817686 }]

The first argument is a task, which is the contract Hugging Face tasks describes. A second argument names a specific model:

const pipe = await pipeline('sentiment-analysis', 'Xenova/bert-base-multilingual-uncased-sentiment')

Models are cached in the browser after the first load.

Two backends, and the one you get by default

WASM runs on the CPU and is the default. WebGPU is opt-in with device: 'webgpu'.

Both sit on ONNX Runtime, so a model has to exist in ONNX form. That is the first thing to check against a candidate, and it is why so many models in Transformers.js examples come from the Xenova namespace: someone converted them.

WebGPU is where the speed is and the documentation is direct that the API remains experimental in many browsers. Treat it as an enhancement with a WASM fallback rather than a requirement, and test the fallback, because it is what a meaningful share of your users will run.

Quantization is not optional here

The dtype option takes fp32, fp16, q8 and q4. WebGPU defaults to fp32 and WASM defaults to q8.

Those defaults encode the constraint. A browser tab has a memory budget and a download the user is waiting on, so an unquantized model is often not a realistic option regardless of quality.

The trade is the one self-hosted models describes, with the dial turned further. Going to q4 may be the difference between a feature that loads in five seconds and one nobody waits for, and it costs accuracy you have to measure on your own task rather than assume.

Weigh the first load honestly

The cost nobody accounts for is the download. The model ships to every visitor before anything works, and it is measured in tens or hundreds of megabytes.

Two consequences. Pick the smallest model that clears your quality bar rather than the best one, since here model size is page weight. And design the loading state as a real part of the interface, because on a slow connection it is the first thing most users experience.

Caching means returning visitors pay once. First-time visitors decide whether there are returning ones.

What it is good for

Privacy-bound features. Text never leaves the page, which settles the question that security and privacy concerns frames. No processor, no retention terms, nothing to put in a data map.

Working offline, once the model is cached.

Removing per-request cost. Compute is the user's, so a feature that would be uneconomic at scale becomes free to you.

Latency for small tasks. Classification or embedding with no network round trip beats a hosted call, once loaded.

What it is not for

Anything needing a large model. The browser cannot hold one, and task coverage is narrower than the Python library: the documentation names Table QA, Mask Generation and Video Classification among the unsupported.

So check three things before committing: the task is supported, the model exists in ONNX, and the quantized size fits a page load. Failing any one of them points you back to a server, whether that is Ollama on a machine you run or a hosted API.

Further reading

• Hugging Face, Transformers.js: the pipeline API, backends, dtype options and unsupported tasks.
• Hugging Face tasks: the first argument, and what it guarantees.
• Gemma: small open models sized for this kind of deployment.
• Self-hosted models: the same quantization trade with more memory to spend.