LangChain

An abstraction over providers, and a framework on top

LangChain does two things that are worth separating, because teams adopt one and inherit the other.

The first is a common interface over model providers: one message format, one way to attach tools, one way to stream, so swapping a provider is a configuration change. LangChain for multimodal apps covers that layer, where content is a list of blocks rather than a string.

The second is a framework for composing calls, retrieval, tools and agent loops into pipelines.

LlamaIndex is the neighbouring choice and is organised around indexing and retrieval. LangChain is organised around the chain of calls.

What the abstraction is worth

Provider portability sounds like the selling point and is rarely why teams stay.

The honest version: OpenAI-compatible APIs get you most of the way for free, since many providers speak that dialect. What an abstraction adds is coverage of the providers that do not, and a single shape for the things compatibility layers drop.

The cost is that the common format cannot express everything. Provider-specific parameters either get passed through untyped or are unavailable, and you debug through a layer that owns the request you did not write.

Where a framework helps

Three things are tedious enough to be worth someone else's code.

Streaming and callbacks, consistently across providers, including token-level events into a UI.

Tool loops, where the framework handles dispatch, results and iteration rather than you writing the same loop again.

The integration surface, which is the real reason most teams arrive: loaders, vector store clients, and retrievers already written. Pulling a PDF loader and a vector store binding off the shelf beats writing both.

Where it costs more than it saves

The recurring complaint is not that it fails. It is that it obscures.

A chain that produces a bad answer has a prompt somewhere, and finding out exactly what reached the model takes longer through an abstraction that assembled it. That is the argument for LLM observability generally and it bites harder here, since the payload is constructed by code you did not write.

Two more, both about pace. The API has changed shape more than once, so tutorials age badly and a version upgrade can be real work. And the abstraction tracks provider features with a lag, so something released this month may be reachable only by dropping to the underlying client.

Choosing deliberately

Reach for LangChain when you want the integrations and the tool loop, when you are switching providers, or when a team benefits from one shape across several features.

Reach for LlamaIndex when the problem is retrieval-shaped and the index is the thing you are building.

Reach for the provider SDK when you call one provider, your pipeline is a few steps, and you would rather own the prompt assembly. For a single retrieval augmented call, the framework is more moving parts than the task has.

A middle path works well: provider SDK for the model calls, and individual libraries for the pieces you would otherwise write. You give up the unified interface and keep the thing you wanted.

If you adopt it, keep the escape hatch

Two habits make the decision reversible, which is what choosing the right model argues for generally.

Keep your prompts as your own artifacts rather than inline in chain definitions, so they survive a framework change. And log the fully assembled payload, not the chain's inputs, so you can always see what the model received.

Further reading

• LangChain for multimodal apps: the message format, and whether the abstraction is worth it.
• LlamaIndex: the retrieval-shaped alternative.
• OpenAI-compatible APIs: the portability you may already have.
• LLM observability: seeing what a chain sent.