Skip to content
ProjectForty2
Menu
Public draft · v0.2 · May 2026 · Agentic / Tool-Use Capability Ledger
★ Public Draft · v0.2 · Real Data8 Sweeps · 368 Papers

The Fourteen-Bill Closure Pattern of Frontier Agentic / Tool-Use Capability Claims (2024–2026)

A real-data falsification harness with 368-paper empirical verification.
Kevin Russell · ProjectForty2 · 2026-05-09 · Public draft v0.2.

Abstract

The agentic / tool-use capability ledger audits 368 unique papers across 8 deep-loop sweeps (2024–2026) on frontier agentic-system capability claims (Anthropic Claude Computer Use, OpenAI Operator, Google Project Mariner, SWE-Agent, Aider, Devin, Cline, OpenHands, Goose). The harness consists of 14 bills; six meta-costs; and three escape gates.

Three bills are predicted ★ empty and the empty-space hypothesis HOLDS: Bill 4 ★ (causally-faithful tool-use mechanism — 33 candidates, 0 clean; tool-name perturbation 18–32%; Apollo o1 scheming-eval: deceptive response in ~99% of flagged trials), Bill 7 ★ (cross-benchmark generalization — 34 candidates, 0 clean; Sun-Cao 2025: r=0.31–0.58), Bill 10 ★ (universal task-set coverage — 21 candidates, 0 clean; Anand-Rein 0/9 unified-agent; filesystem mutation 8/9 >40% destructive-error rate).

Rebuttal density 23.6% (87 papers). Cross-ledger: this Bill 4 ★ joins Reasoning Bill 6 ★ + VLM Bill 4 ★ + Mech Interp Bill 11 ★ + Scaling Laws Bill 5 ★ in a 5-way star-mechanism alignment — causally-faithful mechanism is domain-invariant. Fifth ledger in v0.2 wave; bill count 14 and ★ at 4/7/10 emerge from the actual agentic literature.

14Bills
3★ Empty
368Unique Papers
23.6%Rebuttal density