Page 2 of 11
Listed inSelf-Hosted ModelsModels & Providerson
Sizing a GPU for a self-hosted model means budgeting for more than the weights. The KV cache grows with concurrency, and exhausting it preempts in-flight requests that are then recomputed from the start.
Listed inSecurity and Privacy ConcernsSafety & Ethicson
The industry checklist: injection, insecure output handling, supply chain, data leakage, and the rest — with mitigations.
Listed inSupabase VectorVector Databaseson
Vectors as a column, so the filter is a WHERE clause on columns you already have and a row and its embedding land in one transaction.
Listed inLLM ObservabilityObservabilityon
Chip Huyen assembles the whole reference architecture step by step — context, guardrails, routing, caching, observability.
Listed inWhat is a Context Layer?Context Engineeringon
One place in the codebase that decides what goes into every prompt. The limit that binds is the model's attention rather than the size of the context window, so the layer curates what it sends instead of filling the space available.
Listed inPrompt CachingHow LLMs Workon
How cache breakpoints work, what invalidates a prefix, and the latency and cost numbers to expect.
Listed inTracing & LoggingObservabilityon
One answer is many steps, and the step that broke is the one you did not log. What belongs on a span, and what should stay off it.
Listed inEmbedding Models on Hugging FaceEmbeddingson
Benchmark scores across retrieval, clustering, and reranking tasks. Useful for a shortlist, dangerous as a final answer.
Listed inSampling ParametersHow LLMs Workon
Temperature and top-p change how a token is drawn, not what the model knows. And temperature 0 is not a reproducibility guarantee.
LangChain's concept doc on the splitter family — what each one optimises for and when to reach for it.
Listed inHugging Face Inference SDKModels & Providerson
One client in front of hosted providers, dedicated endpoints and a local server, with a two-line diff from an OpenAI client.
Listed inIndexing EmbeddingsVector Databaseson
Qdrant's explainer on HNSW, payload indexes, and the quantization trade-offs behind recall and memory.
Listed inStoring in a Vector DBRAGon
A namespace is a hard partition, not a tag. Changing the embedding model invalidates every vector, so plan the rebuild before you need it.
Listed inPrompt EngineeringPrompt Engineeringon
A dense, well-cited tour of zero-shot, few-shot, chain-of-thought, and automatic prompt search.
Listed inContext EvaluationContext Engineeringon
Wrong answer or missing context? Precision has a reference-free variant you can run today; recall always needs ground truth.
Listed inRagasEvaluationon
Faithfulness, answer relevance, and context precision — how the metrics are computed and what they miss.
Listed inPostHogObservabilityon
A trace says what your system did. Whether the feature worked is a question about what the person did next, on the same events.
Listed inLangfuseObservabilityon
Open-source tracing, prompt management, and evaluation — including the data model behind a trace.
Listed inStreaming ResponsesHow LLMs Workon
The finish reason arrives late, and an error can land after a 200 with half an answer already on screen.
Listed inSentence TransformersEmbeddingson
The library most self-hosted embedding pipelines are built on: pooling, training, and inference.