Compaction

Compaction is the automatic summary that replaces a conversation's history when a session nears its context window limit. Thariq, an engineer at Anthropic, frames it as one answer to context rot: performance degrades as the window fills, and for the one-million-token model that degradation typically starts around 300-400k tokens depending on the task.

What separates compaction from simply clearing a session is who decides what survives. Compaction is automatic and potentially more exhaustive, but lossy: the model chooses what to keep. Thariq names the sharpest problem in that arrangement. Bad compactions happen when the model cannot anticipate where the work is heading, and it sits at its point of least intelligence precisely when it compacts.

The Claude Code team at Anthropic documented the cost side. A naive implementation issues a separate API call with a different system prompt and no tools, which breaks the cached prefix and pays full price on every token. Their fix keeps exactly the same parameters as the parent conversation, so only the compaction prompt's own tokens are billed. Those patterns are now folded into the API Anthropic.

Vivek Trivedy of LangChain places compaction next to offloading bulky tool results to the filesystem and Skills as progressive disclosure, all responses to the same pressure inside an agent harness.

Thariq expects these mechanisms to eventually be handled by Claude itself. For now, the summary a model writes about its own unfinished work is still something the user has to check.