Context engineering: making AI more deterministic in production

Prod, not IDE session

Production context engineering is not compact/clear in Claude Code. It is designing everything that reaches the model when a business agent runs: system, tools, docs, history, policies. IDE session hygiene has its own article.

In 2025-2026 the consensus among teams running agents is clear: most failures are no longer weak models; they are context assembly failures. The prompt can be fine; what enters the window is wrong, noisy, or too long.

In prod the bottleneck is not only which model, but which context, on every request, in a replayable way. Without the third brick (control: allowlist, logs, HITL), you just have a POC / demo.

Four strategies: Write, Select, Compress, Isolate

A widely reused frame (LangChain formalization for agents) organizes the work in four moves:

Write: persist context outside the window (state machine, ticket store, file handoff, project memory). The window is finite; what must survive between calls is written elsewhere.

Select: retrieve only what is relevant (bounded RAG, filters, top-k, include rules). A whole Drive is not context.

Compress: summarize, truncate tool outputs, hybrid sliding window (last N turns raw + older summary). On long tasks, compress first.

Isolate: separate contexts per agent or sub-task (subagents). The orchestrator does not swallow the full research history.

What sits in a context window

Four practical pillars recur in agent guides: instructions (system and framing), retrieval (RAG / docs), memory (short-term conversation, long-term project), tools (function calling, increasingly via MCP).

Noisy tools (huge JSON, raw logs) rot context as fast as a long chat. We filter, summarize, cap tool response size.

If you wire several MCP servers, count tool-schema tokens before any user request. That entry cost is often underestimated.

Context rot and quality

LLMs lean on the start and end of context. The middle erodes. After a fill threshold, quality drops (same physics as the dump zone on coding agents).

In a business agent: invented prices, forgotten policy, obsolete docs cited. Countermeasure: versioned corpus, mandatory citations, max docs, state reset between tickets, isolate for heavy research.

Version prompts, templates, retrieval pipelines and tool definitions like code. An undated “small change” makes debug impossible.

Skills, routing, evaluation

Versioned skills and playbooks beat throwaway prompts. Explicit triggers when you want determinism.

If the agent serves multiple domains, add routing (even keyword rules at first) to cut bloat before investing in an LLM classifier.

Evaluate: 30-50 gold cases (input → output or action class). Measure tokens per ticket, failure rate, human edit rate. Without a gold set, every “improvement” is an opinion.

Prod checklist

1) Sources listed. 2) Write/Select/Compress/Isolate applied. 3) Tool size caps. 4) MCP schema tokens counted. 5) Versioning. 6) HITL on high-impact outputs. 7) Monthly budget. 8) Gold cases.

If any point is missing, you still optimize by feel. Fine in a spike, not in customer production.

If you want a context audit on a business agent, we can scope in 20-40 minutes.