Image Understanding

Counting the cost of a picture

Multimodal AI establishes that images are billable input tokens rather than attachments. This is the arithmetic, which is the part that decides what your feature costs.

Anthropic publishes the mechanism: Claude views images in patches rather than pixels, each patch a 28×28-pixel block called a visual token. An image costs

⌈width / 28⌉ × ⌈height / 28⌉ visual tokens

A 200×200 thumbnail is 64 tokens. A 1000×1000 image is 1,296. A 1092×1092 image is 1,521. The numbers grow with area, so doubling each edge quadruples the cost until a cap intervenes.

The cap, and the upgrade that triples your bill

Every model has a maximum native resolution, and larger images are downscaled before processing. On Claude there are two tiers:

Read the two right-hand columns together and the consequence appears. A 3840×2160 screenshot is downscaled to 1456×819 and costs 1,560 tokens on a standard model. The same screenshot on a high-resolution model is downscaled to 2576×1449 and costs 4,784.

Nothing in your code changed. You upgraded the model and image costs tripled. Anthropic states the ratio: high-resolution images can use roughly three times more visual tokens than the same image on a standard-tier model.

If you do not need the fidelity, downsample before sending. That is a decision to make deliberately, because the default is to send what you have.

Downscaling is silent, which is why small text disappears

The API enforces the cap by resizing rather than rejecting. A 4000-pixel-wide scan of an invoice arrives at the model as a 1456-pixel-wide scan of an invoice, and the eight-point print in the footer is now unreadable.

This is the mechanism behind most "the model misread my document" reports. The model read what it was given; what it was given was not what you sent.

Two ways out. Crop to the region that matters rather than sending the page, so the resize budget is spent on the part you care about. Or ask for an error instead of a resize: Claude accepts an oversized_image setting of error on the image block, which turns a silent degradation into a failed request you can see.

The rule that catches batch pipelines

Above 20 image blocks in one request, a stricter per-image dimension limit applies to every image in that request, and images over it are rejected with an invalid_request_error mentioning "many-image requests".

Three details make this easy to trip over:

• Images resent from earlier conversation turns count toward the 20.
• Images nested inside tool_result content count, which includes screenshots returned to a computer-use tool.
• On Bedrock and Google Cloud, PDF document blocks count too.

So a conversation that was fine at turn three starts failing at turn nine without anything new being sent. The documented fix is to keep each image under 2000 px, or keep the request to 20 blocks or fewer.

The hard ceilings sit further out: 100 images per request on 200k-context models, 600 on the rest, 8000×8000 px maximum, 10 MB per image base64 on the Claude API and 5 MB on Bedrock and Google Cloud. Request size limits of 32 MB are usually reached first.

Order and labelling change the answer

Two small things with measurable effect.

Images before text. Anthropic notes that Claude works best when images come before text, the same way long documents do. Images after text or interleaved still work; the ordering is a free improvement.

Label each one. When sending several, introduce them with Image 1:, Image 2:. It gives you something to name in the question and in later turns. The model retains earlier images, so a follow-up can ask about them without resending, which also keeps you under the 20-block threshold.

Two things it will refuse, and one it will get wrong confidently

Multimodal AI lists what vision models read badly. Three Claude-specific behaviours sit outside that list.

It will not name people in images, and refuses rather than failing.

It cannot tell whether an image was AI-generated, and may be wrong if asked. Do not build a detector on it.

It does not read image metadata. Nothing in EXIF reaches the model, so orientation, capture time and GPS have to be passed as text if you need them.

The pattern across all of the documented weaknesses is that vision is strong at description and weak at measurement. "What does this invoice say" works. "How many items are on it" is the question that comes back plausible and wrong. Ask for structured output so the answer lands in fields you can check against something.

Further reading

• Anthropic, Vision: the patch formula, the resolution tiers, the many-image rule, and the limitation list.
• Multimodal AI: how media reaches a request, and what vision models read badly.
• OpenAI Vision API: the same decisions expressed as a detail parameter.
• Context window: the budget these tokens come out of.