METR

METR, formerly ARC Evals, published a study on 31 July 2023 asking whether AI systems could acquire compute, copy their own code and weights into new environments, adapt without human intervention, and keep running through obstacles. The answer was no: current AI agents cannot reliably execute autonomous replication. Success rates stayed low on end-to-end multi-step sequences, though GPT-4 clearly outperformed GPT-3.5 on the same tasks. METR named the capability Autonomous Replication and Adaptation, treated it as a threshold rather than a gradient, and recommended mandatory ARA testing before frontier models ship.

The metric that outlived that study is the length of task an AI can complete unaided. METR measures it doubling roughly every seven months. Ethan Mollick leans on that curve in September 2025 to argue agents have reached economically relevant work, noting the measured task length has grown exponentially since GPT-3.

By the evaluation dated 26 June 2026, the measurement itself had become contested. METR reported a reward hacking rate on Sol higher than any public model it had assessed on the ReAct harness, and its own time-horizon estimate for that model ranged from 11 hours to over 270 depending on how those runs were treated. A twenty-five-fold spread in the headline number is an awkward result for an independent evaluator that also collaborates with Anthropic, OpenAI, the AI Security Institute, and the NIST AI Safety Institute Consortium.