Providing Context

The three jobs a prompt is doing at once

A prompt that supplies information is doing three separable things, and most prompt problems are one of them bleeding into another.

It instructs: do this, in this format, under these constraints. It supplies: here is the material to work from. And often it demonstrates: here is what a good answer looks like. These have different lifetimes (the instruction is written once and reused, the material changes on every request, the examples change rarely) and different trust levels, since you wrote the instruction and you very likely did not write the material.

When the model treats supplied text as an instruction, or ignores supplied text in favor of what it already knows, the cause is usually that the boundary between these three was never made explicit.

Context engineering argues about what belongs in the payload, and what is a context layer argues for assembling it in one place. What follows is narrower: you have decided what to include, and now you are writing it into a request.

Where supplied documents go in the prompt

For short prompts the order barely matters. For long ones there is a documented answer, and it is the opposite of what most people write.

Anthropic's prompting guide, for inputs past roughly 20,000 tokens: "Put longform data at the top: Place your long documents and inputs near the top of your prompt, above your query, instructions, and examples." It attaches a measured claim to that: "Queries at the end can improve response quality by up to 30 percent in tests, especially with complex, multidocument inputs."

Take that number as what it is: one vendor's measurement on their own models and their own tests, not a law of nature. The direction is the durable part, and it holds for a reason you can reason about. The instruction is what the model needs in mind while it works. Putting it last means it is nearest at the moment of answering, rather than twenty thousand tokens back.

The practical shape, then, is: documents, then question, then instructions. Most people write it the other way round because that is how you would brief a person.

Marking the boundary between instruction and data

The model receives one sequence of tokens. Your careful distinction between "my instruction" and "their document" is not in that sequence unless you put it there.

The documented technique is explicit delimiters: "XML tags help Claude parse complex prompts unambiguously, especially when your prompt mixes instructions, context, examples, and variable inputs. Wrapping each type of content in its own tag (for example, <instructions>, <context>, <input>) reduces misinterpretation."

<documents>
  <document>
    <source>refund-policy-2026.md</source>
    <document_content>
      ...the policy text...
    </document_content>
  </document>
</documents>

Answer the customer's question using only the documents above.
Question: Can I return an item after 40 days?

Two notes on generality. The specific syntax is tuned for one vendor's models. Tags are what Anthropic documents, and other providers have their own conventions. The principle underneath is not vendor-specific: an unambiguous, machine-visible boundary between what you wrote and what you pasted. Markdown fences and clearly labeled sections do the same job less crisply.

And this is a security measure, not a formatting one. Retrieved text can contain instructions, and a model with no way to tell your instruction from a sentence inside a fetched document may follow the wrong one. The boundary is what makes "treat everything inside these tags as data" a sentence you can write at all. Prompt injection covers why that is a defense rather than a solution.

Carrying each document's source into the prompt

Notice the <source> tag in that example. It is doing more work than it looks.

If the material arrives with an identifier attached, the model can name where an answer came from, and you can check it. If it arrives as an undifferentiated wall of text, a citation is something the model would have to invent, and it will, convincingly.

This is the practical reason storing in a vector database recommends keeping the chunk text next to its metadata. The metadata is not bookkeeping; it is what makes an answer auditable. For anything users will act on, carry the source through to the prompt.

Asking the model to quote before it answers

One more technique for long documents: "ask Claude to quote relevant parts of the documents first before carrying out its task."

The effect is to split one hard step into two easier ones. Finding the relevant passage is a search problem; answering from it is a reading problem. Asked together, a model can drift toward what it already knows about the subject. Asked in sequence, the second step has something concrete in front of it.

You get a useful artifact from it too. When the answer is wrong, the quotes tell you whether retrieval failed or reasoning did, and those have different fixes.

Deciding what to leave out of the prompt

The hard part of this job is subtraction, and the limit that bites is not the one people plan for.

Context windows are large enough now that "will it fit" is rarely the question. The real constraint is attention: context is "a finite resource with diminishing marginal returns", and adding a document that is merely related, rather than necessary, costs you on every request that includes it. Context engineering makes this case in full, and long context processing covers what happens when you ignore it.

A test that works: for each thing you are about to include, name the question it helps answer. Anything that gets a vague answer is decoration, and decoration is not free.

Further reading

• Anthropic, Prompting best practices: long-context placement, tag structure, and the quoting technique. Every quotation above comes from here.
• Anthropic, Effective context engineering for AI agents: the attention-budget argument for leaving things out.
• Structured output: the same problem in the other direction.
• Prompt injection attacks: why the instruction and data boundary is a security control.