|
| 1 | +# Glossary — Context Engineering |
| 2 | + |
| 3 | +Covers Lessons 1–9. Grouped by where each term first lands. |
| 4 | + |
| 5 | +## Foundations (L1–2) |
| 6 | +- **LLM (large language model)** — the reasoning model itself: the brain. Fixed weights, no memory |
| 7 | + between calls, no access to your data except what you place in its context window for that call. |
| 8 | +- **Context** — everything the model can see for a single call: system prompt + instructions + |
| 9 | + conversation history + retrieved data + tool definitions/results + the user's query. The model's |
| 10 | + *entire* world for that one inference. It has no other access to your data. |
| 11 | +- **Context window** — the model's fixed maximum number of tokens per call (input + output). A hard |
| 12 | + ceiling, like total RAM. Exceed it and content must be dropped, truncated, or summarized. |
| 13 | +- **Token** — the unit the model counts in. A sub-word chunk (~4 chars / ~0.75 words of English on |
| 14 | + average). Limits, latency, and cost are all measured in tokens, not characters or lines. |
| 15 | +- **Working context** — the curated subset actually assembled into the window for *this* turn: the |
| 16 | + few things relevant right now, not everything that exists. The output of context engineering. |
| 17 | +- **Context engineering** — selecting, retrieving, compressing, and assembling the window before the |
| 18 | + model runs. Decide what to put in the window (and what to leave out) so a fixed model produces the |
| 19 | + best answer it can. |
| 20 | +- **Context assembly** — the step that packs selected/retrieved content (plus prompt, history, |
| 21 | + tools) into the final window, within budget, in an order the model uses well. |
| 22 | +- **Context-as-query** — the mental model: in classic software the query (SQL) fetches from the DB; |
| 23 | + in AI the context-retrieval pipeline fetches what the LLM reasons over. The "query" moved from |
| 24 | + SQL into context assembly. |
| 25 | +- **Retrieval quality vs model quality** — past a baseline, *what you feed* the model usually moves |
| 26 | + answer quality more than *which model* you use. Right context + smaller model often beats wrong |
| 27 | + context + best model. |
| 28 | + |
| 29 | +## Selection & retrieval (L2) |
| 30 | +- **Selection** — choosing the smallest set of content that still contains the answer, within the |
| 31 | + token budget. Candidate generation → ranking → packing. |
| 32 | +- **Lexical / keyword search (BM25)** — exact-term matching with TF-IDF/BM25 scoring. Precise on |
| 33 | + identifiers and exact terms; blind to synonyms (vocabulary mismatch). |
| 34 | +- **Semantic / vector search** — encode query and chunks as embeddings, retrieve by similarity. |
| 35 | + Captures meaning and paraphrase; can return "topically similar but wrong"; weak on rare exact tokens. |
| 36 | +- **Hybrid search** — run lexical + semantic and fuse results (e.g. Reciprocal Rank Fusion). Covers |
| 37 | + exact *and* semantic; more infra to tune. |
| 38 | +- **Ranking** — ordering candidates by a cheap score; first stage optimizes **recall** (don't miss it). |
| 39 | +- **Re-ranking** — a second, expensive, accurate stage (e.g. a cross-encoder) that re-scores the |
| 40 | + top-N jointly with the query; optimizes **precision**. Can't recover a doc the first stage missed. |
| 41 | + |
| 42 | +## Codebase, chunking & RAG (L3) |
| 43 | +- **Repository indexing** — building a searchable representation of a codebase (embeddings and/or a |
| 44 | + symbol/dependency index) so relevant code can be retrieved. |
| 45 | +- **Read-on-demand (agentic) context** — fetching files live via tools (grep/glob/read) instead of a |
| 46 | + persistent embedding index. Always fresh; costs tool round-trips. (Claude Code's model.) |
| 47 | +- **Chunking** — splitting documents into retrievable units. Chunk on meaning/structure, not raw |
| 48 | + character count. |
| 49 | +- **Overlap** — sharing boundary text between chunks so a fact spanning a boundary survives. |
| 50 | +- **Context fragmentation** — a fact split across chunks so no single chunk is complete/retrievable. |
| 51 | +- **Contextual chunking/retrieval** — prepend a short doc/section-situating summary to each chunk |
| 52 | + before embedding (and index BM25 too) to cut retrieval failures. |
| 53 | +- **RAG (Retrieval-Augmented Generation)** — retrieve → assemble → LLM → answer. Grounds the model |
| 54 | + in your private or current data; only as good as its retrieval. |
| 55 | + |
| 56 | +## Memory, compression & failure modes (L4) |
| 57 | +- **Memory** — persisted state (derived facts, decisions) about an ongoing task or relationship. |
| 58 | + Reachable only by retrieving it back into context — the model never reads it directly. |
| 59 | +- **Knowledge base** — the corpus of source documents/facts; usually read-only reference material. |
| 60 | +- **Compression** — fitting more useful signal into the budget via **summarization** (condense), |
| 61 | + **distillation** (extract salient facts), pruning redundancy, and **sliding windows** (recent |
| 62 | + verbatim + rolling summary). Lossy — keep recent turns and decisions verbatim. |
| 63 | +- **The 5 failure modes** — **missing** (never retrieved), **wrong** (irrelevant retrieved), |
| 64 | + **outdated** (stale index), **conflicting** (sources disagree), **excessive** (too much → noise / |
| 65 | + lost-in-the-middle). Diagnose by symptom → cause → fix. |
| 66 | + |
| 67 | +## Evaluation & observability (L6) |
| 68 | +- **recall@k** — is at least one relevant doc in the top-k. The most important RAG retrieval metric; |
| 69 | + nothing downstream can use what wasn't retrieved. |
| 70 | +- **precision@k** — fraction of the top-k that are relevant. |
| 71 | +- **MRR (mean reciprocal rank)** — how high the first relevant result sits. |
| 72 | +- **nDCG** — graded, position-discounted relevance; rewards putting the best at the top. |
| 73 | +- **Faithfulness / groundedness** — is every claim in the answer supported by the retrieved context |
| 74 | + (detects hallucination). |
| 75 | +- **Context precision / recall** — are retrieved chunks relevant and well-ranked (precision); did |
| 76 | + retrieval capture everything the ideal answer needs (recall, needs ground truth). RAGAS vocabulary. |
| 77 | +- **Golden set** — a versioned eval set of (query, relevant doc IDs, ideal answer) for offline |
| 78 | + regression testing. |
| 79 | +- **Offline vs online eval** — offline = regression on the golden set before deploy; online = |
| 80 | + production signals (thumbs, deflection/escalation, citation-clicks, rephrase rate). |
| 81 | +- **LLM-as-judge** — using an LLM to score faithfulness/relevance at scale (needs calibration). |
| 82 | + |
| 83 | +## Pre-retrieval & advanced RAG (L7) |
| 84 | +- **Query rewriting** — clean/expand/disambiguate the raw query before retrieval. |
| 85 | +- **Multi-query** — generate several paraphrases, retrieve for each, union the results (boosts recall). |
| 86 | +- **HyDE (Hypothetical Document Embeddings)** — embed a generated hypothetical *answer* (not the |
| 87 | + question) to bridge the query↔document vocabulary gap. |
| 88 | +- **Query decomposition** — split a complex question into sub-queries, retrieve each, combine. |
| 89 | +- **Step-back prompting** — ask a broader question first to pull grounding, then the specific one. |
| 90 | +- **Routing** — classify the query and send it to the right index/datasource/tool. |
| 91 | +- **Parent-document / small-to-big** — match small precise chunks but return the larger parent for context. |
| 92 | +- **GraphRAG** — retrieve over an entity/knowledge graph (subgraphs + community summaries); wins on |
| 93 | + global "connect-the-dots across the corpus" questions. |
| 94 | +- **Agentic RAG** — the LLM decides whether/what/when to retrieve, iterating in a loop. |
| 95 | +- **Self-RAG / Corrective RAG (CRAG)** — the model critiques its own retrieval/answer; CRAG grades |
| 96 | + retrieved docs and falls back (e.g. web search) when quality is low. |
| 97 | + |
| 98 | +## Embeddings, indexing & cost (L8) |
| 99 | +- **Embedding** — a dense vector capturing meaning; near vectors ≈ similar meaning. |
| 100 | +- **Similarity metric** — cosine (most common), dot product, or Euclidean distance over embeddings. |
| 101 | +- **ANN (approximate nearest neighbor)** — trade a little recall for large speed at scale; exact |
| 102 | + k-NN is O(n) per query. |
| 103 | +- **HNSW / IVF / PQ** — vector index families: graph-based (fast, high recall, memory-heavy) / |
| 104 | + clustering (probe a few cells) / product quantization (compress vectors, lower recall). The |
| 105 | + recall ↔ latency ↔ memory trade-off. |
| 106 | +- **Metadata filtering** — pre/post-filtering candidates by attributes (date, type, tenant); also an |
| 107 | + access-control hook. |
| 108 | +- **Index freshness / invalidation** — keeping the index current (incremental updates, CDC, |
| 109 | + re-embedding on model change); a stale index *is* the "outdated context" failure mode. |
| 110 | + |
| 111 | +## Caching, ordering & security (L9) |
| 112 | +- **Prompt / KV caching** — reusing the processed state of a stable prefix across calls to cut cost |
| 113 | + and latency. Put stable content (system prompt, tools) first, volatile content (query, retrieved |
| 114 | + docs) last; any change busts the cache from that point on. |
| 115 | +- **Lost in the middle** — models attend most to the start (primacy) and end (recency) of the |
| 116 | + window, weakest in the middle. Order matters: put the query last and the best doc at an edge. |
| 117 | +- **Indirect prompt injection** — a malicious instruction hidden in *retrieved* content that the |
| 118 | + model may obey. Treat all retrieved content as untrusted data, never as instructions. |
| 119 | +- **Multi-tenant access control** — enforce row/document-level permissions in the *retrieval query* |
| 120 | + (scope by tenant/ACL before anything reaches the model). Never rely on the prompt to enforce access. |
0 commit comments