PostHog

The question a trace cannot answer

Your traces show a support assistant answering in 1.8 seconds, using 2,400 tokens, with a valid schema and a plausible reply. Every span is green.

Did the feature work?

A trace records what your system did. It does not record that the user read the answer, closed the panel, and opened a support ticket anyway. The engineering question and the product question have separate answers, and the second one lives in your analytics.

What PostHog records for an LLM call

PostHog captures, in its own words, "the prompt, the response, the tokens, the cost, the latency, and the tools it reached for along the way." The model is the familiar one: traces containing spans and generations, across providers and frameworks.

So far this overlaps with any tracing tool. The difference is where those records land.

Why the events are the same events

The mechanism behind the join is one sentence in the docs: the observability layer "runs on the same events as the rest of PostHog."

An LLM generation is not stored in a separate system keyed by trace ID. It is an event on a person, on the same timeline as their page views, their clicks, and their sign-up. Which means "every trace comes with the user who triggered it, their session replay, and any exceptions the request threw."

You do not correlate anything. There is nothing to join, because the records were never apart.

Questions the joined records answer

The useful queries are the ones that cross the boundary.

Did a bad answer cause the abandonment? Find sessions where the assistant answered and the user opened a ticket within five minutes, then watch the replay and read the generation that preceded it. You see the answer and the reaction together.

Which answers do people act on? Compare generations followed by the user completing the task against generations followed by nothing. Latency and token counts will not separate those groups. Behavior does.

Where does cost concentrate? Cost per generation is a number your tracing tool has. Cost per user, or per converted user, needs both halves.

What did the exception do to the person? An error rate tells you how often requests failed. The session replay next to it tells you what the user saw and whether they tried again.

That last pair is the clearest case for keeping these together. A 2% error rate reads as acceptable on a dashboard and reads differently when you watch someone hit it twice and leave.

Deciding whether you want this tool

Start with the instrumentation model rather than the vendor. Tracing and logging covers what to record and why, and that applies whatever you use.

Then ask what your unanswered questions look like. If they are "which step was slow" and "why did this call fail," a tracing-first tool answers them well, and Langfuse is built for that. If your questions involve what the user did next, the join is the argument.

The usual caution about analytics applies with more force here. Prompts and completions carry whatever your users typed, so capturing them means customer conversations are in your analytics tool, readable by everyone with access. Decide retention and redaction before you turn content capture on, not after someone asks how long you keep it.

Further reading

• PostHog, AI observability: what is captured and how it attaches to a person.
• Tracing and logging: the vendor-neutral model underneath this.
• Cost and latency monitoring: the numbers worth watching per request.