Token Budget Wars
Viral X thread (**230.5K views**, May 28, 2026, 1:51 AM) by **Jaya Gupta** (@JayaGup10, investor — likely Foundation Capital, author of the *Context Graphs* framework) titled ***"Token Budget Wars"***. **Pivot thesis**: ***"Enterprise AI has moved from adoption to allocation"*** — phase 1 of enterprise AI proved that models can work; phase 2 will decide **how much of that work is worth it**. The new currency at the top of the enterprise is the **ability to quantify AI ROI**: *"show me the value"*. Canonical concept: ***marginal token utility*** = *"the business value created by each additional dollar of inference"* — the number that matters at scale, and that **most companies cannot see**. Timeline: **Claude shipped November 2025**, after the 2026 annual budgets were locked → as early as **Q1**, companies *"running multiples ahead of plan"* → inference stops being an experimentation line item and becomes a **recurring operating cost**. Shift from **experimentation (a few $100K) → infrastructure (seven figures, $1M+)**: at infrastructure scale, **technical variance produces material P&L swings — two runs of the same workflow on the same input can differ by 5-10× in token cost** with nothing visibly broken, *"a number the CFO has to explain to the CEO"*. **AI competes with labor**: 3 types of budget requests (replace outsourced work / replace internal work / generate revenue) → shift toward the ***cost of a completed outcome*** (cost per resolved ticket, processed claim, reviewed contract, completed invoice, avoided hire, retained customer, dollar of revenue moved). **BPO = the easiest baseline to benchmark against** (already priced in completed units); internal work is much harder (multi-skilled employees, diffuse gains, HR resistance to headcount reduction). **Why it's different from SaaS**: SaaS learned to treat usage as a proxy for value; AI breaks that proxy — *"the signal and the noise share the same unit"* (the token), *"SaaS usage told you the software had been adopted. AI usage tells you the meter is running. It doesn't tell you whether your company is cooking."* **Three causes of marginal token utility's invisibility**: (1) ***retry tails*** — tokens per resolved workflow ≈ **T/p**; going from 90% to 70% completion increases effective cost by ~**28%**, not 20%, because failures compound; (2) ***context inflation*** — inference cost ≈ **O(n²)** in context length (attention), doubling the context **quadruples** reasoning cost (over-retrieval: 50 docs when 5 would do); (3) ***routing*** — by default the most powerful model is used (basic classification run on a complex reasoning model); across millions of calls, the difference between routing easy tasks to a small model and sending everything to the frontier model = *"the difference between a manageable bill and a board-level problem."* **Sector split**: **software** companies = a **productivity measurement** problem (already instrumented: PRs, commits, deploys, incidents, cycle time, MTTR — tracks *"AI layoffs"*); **non-software** companies = a **transformation** problem (operational work: claims, underwriting, support, compliance reviews, supply chain exceptions, payment disputes — *right under audit, not just right on average*). **The missing layer = token-to-outcome attribution**: a conversion layer linking inference spend → work performed → business outcome, answering 3 questions (real cost including retries/corrections; which parts of the trace mattered vs. thrashing; did the work change the operating model). ***Measurement becomes memory***: linking a token to an outcome requires capturing **decision traces** (what the agent saw, retrieved, called, ignored, where it retried, when a human overrode it) — *"decision rationale is one of the most perishable assets in a company"* (lives in Slack, emails, escalation calls, people's heads). Agents **create** these traces; captured first to justify the spend, they become *"more valuable than the cost report"* → a **context graph** (*"although I am so tired of that word these days"*). **The allocation layer is the prize**: whoever owns token-to-outcome attribution makes the **allocation calls** (which workflows deserve more compute, which are capped, which move to cheaper models, which stay human, which replace BPO). Companies won't do this on their own — they'll **buy it as a transformation** (Fortune 500 playbook: McKinsey + Palantir alumni + top-down CEO, in the manner of ERP/BI/digital transformation, a *"program"* with an executive sponsor and infrastructure that becomes the **new source of truth**). Framed by **Charlie Munger**: *"show me the incentive and I will show you the outcome."* Organizational sub-thesis: the decades-old executive instinct that *big teams = big jobs/scope/power* → once intelligence becomes the **scarce resource**, the new marker is *"how much of it you're orchestrating."* Direct relevance to the **Cost Optimization / agentic FinOps positioning**: empirically confirms the levers (model routing, prompt caching, context hygiene, sub-agents) and shifts the KPI toward **cost per completed outcome**. Strong convergence with Bain's *cross-system labor* (execution data moat, Cursor), Ng's *No AI jobpocalypse* (pricing anchored on the replaced employee's salary), DORA ROI (cost per feature), Mensch/Mistral (electron→token), Ensarguet (economics of computation), Foundation Capital's *Context Graphs* (decision traces, same author), Wescale's *Token Burning*, BFM/Girard (token = value fuel).
**Jaya Gupta** (@JayaGup10) — investisseuse / VC. Très probablement **Foundation Capital** (le thread s'auto-réfère au cadre ***Context Graphs*** — *« ahem, context graph, although I am so tired of that word these days »* — concept porté par Foundation Capital, cf. fiche `bain-100b-saas-opportunity` qui cite *Foundation Capital — Context Graphs trillion-dollar opportunity, 2025-12-22*). Thread publié sur X le **28 mai 2026 à 1h51** · **230 · 5K vues** · format essai long en un seul post. Une réponse notable de **@tuning_engines** (*« DevSecFinOps for the Agentic Era »*) : *« Tokens will basically have to be managed like headcount […] model hierarchies too »*.