Skip to content

root / tags / reward-hacking

#reward hacking

5 fiches

Quality & Security Auto-verified translation

GLM-5.3: Frontier Coding with Emergent Cyber Capabilities

Announcement post published on the **official Z.ai blog** (formerly Zhipu AI, Chinese lab) on **August 14, 2026**, **with no individual byline**, ~2,000 words plus footnotes. It announces **GLM-5.3**, successor to GLM-5.2, opening with a methodological thesis: *« Scaling post-training is all we did for GLM-5.3. »* Same base model as GLM-5.2 — *« every gain comes from post-training »*. Three announcements. **(A) An open-weights coding model**: +50% claimed on **Z.ai Code Bench**, an unpublished in-house benchmark. **(B) A cyber capability presented as "emergent"**, which the body of the text traces to a training choice — *« As part of post-training, we introduced vulnerability discovery data and environments into the training mix. We expected this to make the model better at finding and reasoning about vulnerabilities »* — what came as a surprise was the speed and the change in nature: the model moves from identifying isolated flaws to *« coherent plans for complete exploitation chains »*. Gains grow with position in the exploitation chain: CyberGym 77.2 → **84.5%**, ExploitBench 24.4 → **54.4%** (×2.2), ExploitGym 29 → **105** tasks in 2h (×3.6), with the gap to the closed frontier remaining wide (181 and 247 tasks). Z.ai puts it this way: *« Capability is growing fastest exactly where we are furthest behind. »* The post also publishes a **Z.ai Security Disclosure Ledger**: **2,436 vulnerabilities identified across 269 open source projects** — kernels, OSes, browser engines, infrastructure, web applications, network protocols — the oldest introduced in **1981**, average lifetime before discovery **26.6 years**, of which **53 disclosed** and **2,383 under embargo**. **(C) A weight release** *« within two weeks of launch, once safety evaluation and hardening are complete »*. The most reusable methodological contribution: **environment and verifier synthesis**, the latter produced without access to the reference solution and admitted only after a triptych of negative controls — **oracle**, **no-op**, **unsolved-state**. All agentic evaluations are conducted **in Claude Code 2.1.207**.

#GLM-5.3#GLM-5.2#Z.ai

**Z.ai** (anciennement **Zhipu AI**) · laboratoire d'IA chinois · éditeur de la famille **GLM**. Billet **institutionnel et non signé** : aucun auteur nommé · aucun chercheur mis en avant · aucun lien vers un rapport technique ou une carte de modèle. Publié le **14 août 2026**. La page est une SPA React — le HTML servi est un `<div id="root">` vide · et le texte comme les scores ont dû être extraits du bundle `glm-5.3-BCnx8T5_.js` · où ils figurent en valeurs source.

Economy & Market Auto-verified translation

GPT-5.6 Sol, Terra, Luna : comment OpenAI rebat les cartes du coding agentique et du pricing

SFEIR analysis (firm's voice) of the general availability, on July 9, 2026, of **GPT-5.6** by OpenAI — not a single model but a **family of three tiers**: **Sol** (long-horizon/cyber/science flagship, the only one to unlock the "max" and "ultra" modes), **Terra** (everyday balanced tier, ~half the price of GPT-5.5), and **Luna** (fast/economical, high volume). All three share ~**1.05M tokens** of context, **128k** output tokens, and a knowledge cutoff of **February 16, 2026**. The most structuring fact is not a score but an **aggressive pricing grid** (Sol $5/$30, Terra $2.50/$15, Luna $1/$6 per million tokens): Sol keeps the previous flagship's price while being more capable, forcing the comparison onto the **capability-to-cost ratio**. Two billing subtleties (cache writes billed at **1.25×**, a surcharge beyond **272k** tokens) make the grid misleading until one has measured how much context the agent re-reads (read/write ratio ~**153:1** in agentic coding). Engineer's verdict, claimed to be neutral (SFEIR is both a **Google Cloud Premier** partner *and* an **Anthropic** partner): **no one sweeps every table** — GPT-5.6 dominates Terminal-Bench 2.1 and the Coding Agent Index (at a third of the cost per task), Claude stays ahead on SWE-Bench Pro (~15 pts); METR flagged a record **reward hacking** rate on Sol. Conclusion: "stop looking for the champion, learn to route" — the model is a commodity, the durable advantage lies in **Context/Harness Engineering**.

#GPT-5.6#Sol#Terra

SFEIR (voix éditoriale du cabinet)

AI Coding Agents & Skills Auto-verified translation

Stop Running the SDLC on Models That Aren't Human

Chris Williams (@voodootikigod) opens his ADLC series arguing that running the human SDLC on models is a category error: the classic cycle was designed to counter human failure modes (ego, fatigue, forgetting) that are absent in LLMs. He catalogs eight load-bearing failure modes (F1-F8) and five exploitable properties (E1-E5), and lays out the founding principle: every phase of an agentic cycle must trace back to a failure mode it defends against or a property it exploits.

#ADLC#agentic development lifecycle#SDLC

Chris Williams (@voodootikigod)

AI Coding Agents & Skills Auto-verified translation

Tests Are the Spec in the Only Language the Builder Can't Argue With

Third installment in the ADLC series: Williams turns testing into the specification in the only language the builder cannot contest. Where TDD is an optional quality practice for human-written code, it becomes the load-bearing trust mechanism of the entire lifecycle once agents write the code. Three "rail discipline" rules: separated authoring contexts (specs-only before implementation), mechanical freezing at the tool level (not the prompt), and adversarial audits ("does a test fail if the feature is deleted?"). Mutation testing is preferred over coverage percentage, which is Goodhart-able at machine speed.

#ADLC#tests as spec#agentic TDD

Chris Williams (@voodootikigod)

Philosophy & Society Auto-verified translation

Goodhart's law

Encyclopedic article (Wikipedia, English) on **Goodhart's law**: stated by British economist Charles Goodhart in 1975 regarding monetary policy — "any observed statistical regularity tends to collapse once pressure is placed upon it for control purposes" — then generalized by anthropologist Marilyn Strathern (1997) into the canonical aphorism "when a measure becomes a target, it ceases to be a good measure." The subject connects economics, incentive theory, public policy evaluation and, by extension, metric optimization in AI systems.

#Goodhart's law#measure becoming a target#statistical regularity

Wikipedia contributors (concept : Charles Goodhart ; généralisation : Marilyn Strathern)