A request that looks like one prompt is usually four or five fetches, each with its own latency, freshness and trust level. Giving every source a token budget at assembly makes the total knowable before the request goes out.
Where the payload comes from
Context engineering sets out what a payload is made of. This page is about where each part comes from, which is a different question with different consequences: every source has a latency, a freshness, and a trust level, and the assembly step inherits all three.
A request that looks like one prompt is usually four or five fetches wearing a trenchcoat.
The five, and what each costs you
Static text you wrote. System instructions, policies, format rules. Free to fetch, zero latency, and the most trustworthy thing in the window. It is also the part that silently grows: nobody deletes a rule.
Conversation history. Free to fetch, already in hand, and the source that grows without bound. State and historical context covers what it costs to carry.
Retrieved documents. A vector or keyword search at request time. Adds real latency, and the content is only as fresh as your last index run. This is the source where "the model is wrong" most often means "the index is stale."
Tool results and live API calls. The freshest source and the slowest, since you are waiting on someone else's system. Also the least predictable in size: a query returning thirty rows and one returning thirty thousand look identical until they arrive.
User and account state. Profile, entitlements, preferences, tenant. Cheap, usually a single database read, and the one with the sharpest blast radius if you fetch the wrong row.
Freshness is a property of the source, not the request
The failure this prevents: a payload assembled from sources with different ages, presented to the model as a single consistent picture.
The retrieved document says the price is £40 because the index ran on Tuesday. The live API says £45. The model sees both, with nothing marking which is current, and picks one. It will sound equally confident either way.
Two habits help. Label the source of each block, so the model can be told which to prefer when they disagree. And know each source's staleness well enough to state it: an index rebuilt nightly is a different thing from one rebuilt monthly, and only one of them can answer a question about today.
Trust is also a property of the source
Two of the five carry text you did not write. Retrieved documents and tool results can contain anything, including instructions aimed at your model.
Context isolation makes the structural case for separating what you wrote from what you fetched, and prompt injection covers the attack. The assembly step is where the distinction has to be made, because after concatenation it no longer exists.
Marking provenance at assembly time costs a few tokens per block. Reconstructing it afterwards is impossible.
Assemble on a budget, not on availability
The tempting design fetches everything that might help and concatenates it. It produces a payload whose size depends on what the searches happened to return.
The alternative is to give each source a budget, in tokens, and enforce it at assembly. Retrieval gets 2,000 tokens and returns its best passages within that; history gets 4,000 and is summarized past it. The total is then knowable before the request goes out.
This also makes the tradeoff explicit rather than accidental. When retrieval and history compete, someone has decided who wins, instead of whichever one happened to be longer crowding out the other.
Fetch late, and only if needed
Not every source has to be read on every request.
A classifier or a cheap first pass can often decide which sources this request needs, so a question answerable from history does not trigger a retrieval, and a question about live state does not drag in three documents. This is the same just-in-time argument that external memory makes for notes: fetch on demand rather than loading up front.
The saving is latency as much as tokens. A source you did not query is a round trip you did not wait for.
Further reading
- Context engineering: what the payload is made of, and the decision behind each part.
- Context isolation: keeping sources separable once they are in the window.
- Retrieval process: the source with the most moving parts.
- Context failure modes: what happens when sources contradict each other.
Knowledge check
Question 1 of 3
Sign in to save your progress and pick up where you left off.