This technical article from the Claude Code team at Anthropic reveals how prompt caching forms the architectural foundation of Claude Code, making an agentic product with long-running sessions economically viable. The fundamental principle: the cache works through prefix matching, where any modification to the prefix invalidates everything that follows. This single constraint dictates the design of the entire system.
The system prompt architecture follows a strict order: static content and tools first (global cache), then CLAUDE.MD (per-project cache), session context, and finally conversation messages. This hierarchy maximizes cache sharing across sessions. The team discovered that this order is surprisingly fragile: a simple timestamp, a non-deterministic tool order, or updating tool parameters is enough to break the cache.
Several counterintuitive rules emerge from their experience. First, never modify the system prompt for changing information (date, modified files) but instead use <system-reminder> tags in subsequent messages. Second, never switch models mid-session: at 100k tokens into an Opus conversation, using Haiku for a simple question costs more because the cache must be entirely rebuilt. The solution: subagents with a handoff message. Third, never add or remove tools, since they are part of the cached prefix.
Plan mode perfectly illustrates cache-constrained design. Instead of replacing tools with read-only versions (which would break the cache), Claude Code keeps all tools and uses EnterPlanMode/ExitPlanMode as additional tools, along with an explanatory system message. This design lets the model enter plan mode on its own without breaking the cache. Tool search applies the same principle: instead of removing unused MCP tools, lightweight stubs with defer_loading: true keep the prefix stable.
Compaction (summarization when the context window is exceeded) presents major pitfalls. The naive implementation, using a separate API call (different system prompt, no tools), incurs the full cost of all tokens. The solution: use exactly the same parameters as the parent conversation to reuse the cached prefix, paying only for the compaction prompt's tokens.
The article concludes that the cache hit rate must be monitored like uptime, with alerts and incidents declared when it drops. These patterns are now integrated directly into API Anthropic, allowing any agent developer to benefit from these optimizations.