Pipelines are a declared graph, so branching a query into keyword and vector retrieval and merging the results is something you can draw and inspect. The structure pays at five stages and gets in the way at two.
Pipelines as an explicit graph
Haystack builds search and RAG systems out of components wired into a pipeline. Where LangChain composes a chain of calls and LlamaIndex organises around an index, Haystack's unit is a declared graph: you name the components and the connections between them.
That sounds like a stylistic difference and has a practical consequence. A graph you declared is a graph you can inspect, draw and reason about, which is the thing that gets hard once a retrieval pipeline has more than three stages.
What a pipeline is made of
The pieces are the ones any retrieval system has, given names and a common interface: converters that turn files into documents, splitters that chunk them, embedders, writers that put them in a store, retrievers that get them back, rankers that reorder, and generators that call a model.
Two pipelines, not one, which is the split what is RAG insists on. Indexing runs offline and query runs per request, and keeping them as separate declared graphs makes that separation structural rather than a convention you remember.
Where the shape pays
Branching and merging. A query pipeline that runs keyword and vector retrieval and merges the results is a normal thing to want, and semantic search says hybrid is what most production systems run. Expressing that as two branches into a joiner is clearer than the same thing written as nested calls.
Swapping a component. Changing the retriever or the reranker is a change to one node rather than to the code around it, which makes a comparison cheap to run.
Seeing the pipeline. You can look at the graph. When a result is wrong, knowing which node produced what narrows the search immediately, which is the split Ragas and evaluation metrics both argue for: retrieval failure and generation failure need different fixes.
Where it costs you
The ceremony is real. A single retrieval-augmented call expressed as a declared pipeline is more code than the same call written directly. The structure pays at five stages and gets in the way at two.
It is retrieval-shaped. If your system is mostly agent loops and tool calls, you are using a search framework for something else. LangChain or a provider SDK fits that better.
It is another abstraction over providers, with the same trade every framework here has: a common interface, a lag behind provider features, and a payload assembled by code you did not write. Log the assembled prompt for the reason LLM observability gives.
Choosing between the three
The frameworks in this section overlap enough that the distinction is about what you are building.
| If the system is | Reach for |
|---|---|
| A multi-stage search pipeline you will tune | Haystack |
| An index and queries against it | LlamaIndex |
| A chain of model calls with tools | LangChain |
| One retrieval-augmented call | the provider SDK |
None of these is a rule. The question underneath is whether your complexity is in the retrieval graph, the index, the call sequence, or nowhere yet, and adopting a framework before you have that complexity buys structure you do not need.
Further reading
- What is RAG: the two pipelines, and the stages that can ruin it.
- LlamaIndex and LangChain: the neighbouring frameworks.
- Semantic search: why hybrid is the common shape.
- Evaluation metrics: measuring the stages separately.
Knowledge check
Question 1 of 3
Sign in to save your progress and pick up where you left off.