A Block Engineering benchmark post from August 6, 2026, signed by Atish Patel, about Buzz — the human + agent workspace launched on July 21 — asking a cost question: which agent team is the cheapest one that reliably succeeds?
By - **Atish Patel** — *« Building AI solutions @ Block »*// Source engineering.block.xyz ↗/Reading 2 min/.md// Auto-verified translation
A Block benchmark post signed by Atish Patel, published on August 6, 2026, extending the Buzz launch: since assembling an agent team there has become trivial, which one is the cheapest that reliably succeeds?
Vocabulary first. The post proposes four tiers: QuickBee (fast and cheap — builds, screenshots, tests, first-pass triage: GPT-5.6 Luna, DeepSeek V4 Flash, local models, run at high effort), WorkerBee (versatile, carries a full subset unsupervised: GPT-5.6 Terra, Gemini 3.6 Flash, open models), SmartBee (big picture, trade-offs, escalations: Claude Opus 5, Kimi K3, GPT-5.6 Sol, at medium effort), and the human, "the most expensive bee on the team, and the slowest. Also still the smartest". Two team forms: the permanent Hive, which remembers your preferences, and the disposable Swarm, which remembers the project and then disappears.
The solo result. On Terminal-Bench 2.1, raising the effort of a cheap model is the best buy: Luna goes from $1.61 / 57.3% (medium) to $4.98 / 75.0% (high). At the other end, Opus 5 at xhigh is the most expensive run ($140.63) and scores only 75.0%, having hit the timeout on 17 of 88 tasks through over-reasoning. Among the six best runs: a 5.5× price gap, an 8.9-pt score gap. Conclusion: "choosing between them is not a quality decision at all. It is a budget decision."
The team result, in two acts. On Terminal-Bench 2.1, twelve compositions were tested and none beat solo at equal cost — a short task doesn't have enough structure to divide. On Long-Horizon Terminal-Bench (44 multi-hour tasks, lead GPT-5.6 Sol, 3× the timeout), the reversal is clear: solo 15 tasks / 59.1%, +2 WorkerBees 20 / 71.5% — +12.4 pts, of which 11.4 come from additional completions. The team costs more per task, which pays off "when the alternative is a human picking up unfinished work".
The operating rule. Route worker escalations to a SmartBee coordinator rather than to the human: "every ambiguity becomes a notification" is the real failure mode. A Block engineer says they migrated over 2,000 apps with a Swarm (coordinator, 1–10 migrators, independent verifier), the coordinator storing human answers in memory.
Caveats: n=1 per task, no confidence interval, team costs unpublished, and an admission — "this might change if models are trained on better collaboration."
Key takeaways
Date / source.August 6, 2026, engineering.block.xyz, signed by Atish Patel. Measurements on Harbor, real Buzz agents on a live relay, one attempt per task, no retry, prices as of 2026-07-30.
Key framing. the question asked is not "which is the best team" but "which is the cheapest one that reliably succeeds". All findings follow from that. ### Negative result on short tasks On Terminal-Bench 2.1, twelve compositions — pairs, triads, cheap swarms under a frontier lead — pitted against the solo agent each was built around: none beat it at equal cost. The stated reason is structural: a task that finishes in minutes doesn't have enough structure to divide, and "More agents mostly buys you the cost of explaining it twice." Resulting rule: don't assemble a team for short, well-specified work. Empirical counterweight to the launch post [[longwell-block-buzz-workspace-agents-nostr-2026-07-21]]. ### The horizon reverses the result Long-Horizon Terminal-Bench, 44 tasks, lead GPT-5.6 Sol at high effort for all rosters: | Roster | Tasks finished /44 | Score | |---|---|---| | SmartBee solo | 15 | 59.1% | | + 2 QuickBees | 19 | 64.1% | | + 1 QuickBee + 1 WorkerBee | 19 | 69.5% | | + 2 WorkerBees | 20 | 71.5% | +12.4 points, of which 11.4 are additional completions: the team doesn't do better, it finishes the job. Three caveats: runs at 3× the timeout (solo included), team cost unpublished, n=1 per task. Decision criterion proposed by the post: the team is worth its extra cost "when the alternative is a human picking up unfinished work". ### The diminishing-returns ceiling of price Solo on Terminal-Bench 2.1, Opus 5 at xhigh effort = $140.63 for 75.0%, the most expensive run, trailing six runs between $20.08 and $109.82 (79.5% to 88.4%). Stated cause: over-reasoning, with 17 of 88 tasks timing out. This is first and foremost a wall-clock artifact — a property of the model × harness × timeout combination, not a measure of raw capability. What remains actionable: "Paying more stops helping, then starts hurting", and a SmartBee at medium effort suffices for most tasks. Among the six best runs: a 5.5× price gap for an 8.9-point score gap, which the post treats as a tie at this sample size. "Everything from Terra at medium effort upward is the same agent as far as these tasks can tell. Which is good news, because it means choosing between them is not a quality decision at all. It is a budget decision." Scope limited to this benchmark, these prices, and this harness. ### Reasoning effort: the trade-off flips by tier On a cheap model, raising effort is the best buy: Luna medium = $1.61 / 57.3% → Luna high = $4.98 / 75.0%. On a frontier model, raising effort costs more and degrades. | Tier | Recommended effort | Work | Cited examples | |---|---|---|---| | QuickBee | max / xhigh / high | builds, screenshots, test suite, first-pass triage | GPT-5.6 Luna, DeepSeek V4 Flash, local models | | WorkerBee | high / xhigh | a full end-to-end subset, unsupervised | GPT-5.6 Terra, Gemini 3.6 Flash, open models | | SmartBee | medium | big picture, trade-offs, absorbing escalations | Claude Opus 5, Kimi K3, GPT-5.6 Sol | | Human | — | "The most expensive bee on the team, and the slowest. Also still the smartest." | — | The inversion point depends on local timeouts: retest before generalizing. ### Hive or Swarm: what should the team remember?
Hive. — a permanent team of named agents, each with a role and a memory of your preferences. Cumulative argument: "The second time it reviews your peer's code it knows which nits you wave off. The tenth time, briefing it is faster than briefing a person."
Swarm. — a disposable team for a project with a start and an end (migration, framework upgrade, large refactor), which accumulates a memory of the project and is then deleted. Choice criterion: is the subject to remember you, or the project? Assumed prerequisite: the agent is a seat, not a session — name, persona, memory, its own presence in the channel. ### Escalation topology Diagnosis: "The failure mode of agent tooling is not that the work is bad, it is that every ambiguity becomes a notification." The SmartBee coordinator absorbs the routine (flaky test, ambiguous import, moved config) and writes human answers to memory, so the Swarm becomes cheaper to supervise over time. "The human stops being middleware and goes back to being the last reviewer." Qualifying question to ask of any multi-agent tool: where do escalations go, and does the tool learn from the answers? ### Migration testimonial Leigh Maddock, Engineer @ Block: "I migrated over 2000 apps/projects using Buzz and a Swarm of agents", with 1 coordinator, 1 to 10 parallel migrators, and 1 independent verifier, the coordinator handling the majority of escalations. Testimonial, not a measurement: no duration, no cost, no failure rate, no definition of "migrated". The pattern remains transferable — the same shape as the one-shot minions described in [[gray-stripe-minions-coding-agents-part1-2026-02-09]], with the addition of the independent verifier and escalation memory. ### Ready-to-use compositions | Case | Composition | |---|---| | PR review | SmartBee that reviews the PR + QuickBee that builds locally and produces screenshots | | Flaky test triage | QuickBee that reruns tests and gathers evidence + SmartBee that decides | | Short work | a single agent, possibly a QuickBee on related but different tasks | Common denominator: the expensive tier reads and decides, the cheap tier executes and gathers evidence. ### Commercial positioning Buzz accepts Claude Code and Codex subscriptions, open models, and local models: "You are not locked into one provider or forced to give every job to the most expensive model out of laziness." The proposed default composition is explicitly tri-vendor. The thesis "the best model isn't always the right one" is true and commercially useful to whoever sells the part rather than the model. See also [[paymentsdive-block-dorsey-pricing-ia-2026-08-06]]. ### Experimental conditions, to copy if the exercise is repeated Harbor; real Buzz agents on a live relay ("None of the results below were measured on a stripped-down test rig"); one attempt per task, no retry, with timeouts; LHTB at 3× the timeout, solo included; prices as of 2026-07-30; Kimi K3 and DeepSeek V4 Flash don't support medium effort. No confidence interval, n=1 — the post itself acknowledges the tie among six runs as a sample-size effect. Treat these figures as orders of magnitude.
Key figures
none of the twelve team compositions tested on Terminal-Bench 2.1 outperformed the equivalent solo agent on a quality-to-price basis
Claude Opus 5 at xhigh effort is the most expensive solo run on Terminal-Bench 2.1 at $140.63 for 75.0%, among six billed runs ranging from $20.08 to $109.82
beyond a certain tier, models are indistinguishable on these tasks, so choosing between them is no longer a quality decision but a budget decision
— Block
a short task doesn't have enough structure to be divided, and adding agents merely buys the cost of explaining it twice
— Block
plus de 2 000 apps et projets ont été migrés chez Block avec un Swarm composé d'un coordinateur, de 1 à 10 migrateurs parallèles et d'un vérificateur indépendant
— Leigh Maddock
The knowledge graph extracted from this fiche — 14 entities, 28 relations.
In this graph :QuickBee · WorkerBee · SmartBee · Hive · Swarm · escalade agent-vers-agent · Terminal-Bench 2.1 · Long-Horizon Terminal-Bench · Harbor · Claude Opus 5 · GPT-5.6 Luna · Buzz · Atish Patel · Leigh Maddock