Falsification ledgers
Each ledger takes one research field. Before searching, it writes down tests that any honest claim in the field must pass. These are its bills. For a few of them (star bills) it predicts that no paper will pass. If none does, the prediction held. Agents then sweep the papers, check the key citations, and record what they find.
Dates load from ledgers.json and ledger_activity.json.
How a ledger works
Falsification ledgers make the hard part public: what would count as a real break, which evidence was checked, and which claims survived the checks.
Write the bills before the sweep. Star bills are the results the method predicts will stay empty.
Search the papers in the field and collect anything that could fill a bill.
A separate pass looks up the most important citations and confirms they exist and say what was claimed. Until this pass finishes, a ledger's results are provisional. The Method page calls it gate 4; the ledger files call it stage 3.5.
The ledgers
Filter by stage, field or star bills. Choose Researcher for the technical summary of each ledger.
AI vendors say their models work in hundreds of languages. We sorted 299 papers on the languages with little training data.
299 papers sorted. The three star bills hold: no model reaches basic parity on a low-resource language, transfers cleanly across writing systems, or covers 150 of the 200 Flores test languages.
Citation check: not finished yet.
When an AI cites a source, can you trust the citation? We sorted 247 papers from 2024-2026.
247 papers sorted. The three star bills hold: no system shows that its cited source caused the answer, works across document sets, or passes all five sub-tasks.
Citation check: not finished yet.
Companies promise robots that learn real-world tasks. We sorted 312 papers to see if the demos carry over.
312 papers sorted. Our first headline was wrong: all 9 citations flagged for a closer check were invented by our search agents. With those removed, the three star bills still hold.
Citation check: 9 of 9 flagged citations were wrong or invented.
If a real attack on internet encryption arrived, where would it come from? We sorted 504 papers on attacks from 2024-2026.
504 papers sorted. The three star bills hold. None of these 2024-2026 papers breaks the threat model. The ledger is locked.
Some physicists think space and time come in tiny pieces. We sorted 388 papers on the idea. This does not say who is right.
388 papers sorted. No paper clears the four star bills, so they still hold. But all 20 citations flagged for a closer check were invented by our search agents. Treat this as provisional until the paper list is rebuilt from physics databases.
Citation check: 20 of 20 flagged citations were wrong or invented.
AI now makes images, video and music. We sorted 377 papers to see which marketing claims hold up.
377 papers sorted. The three star bills hold: 0 of 8, 0 of 9 and 0 of 39 candidate papers cleared them. Closed products and open research differ sharply in how they are tested.
Citation check: not finished yet.
AI systems are said to discover new chemicals, materials and proofs. We sorted 301 papers to see which results were reproduced.
301 papers sorted. One star bill is partly met: 10 papers show autonomous-lab results that reproduce in a wet lab. The other two still hold: no system works cleanly across fields, and none runs the full AI-scientist loop.
Citation check: not finished yet.
Special AI chips promise faster, cheaper AI answers. We sorted 291 papers to see if their benchmark wins carry over to real work.
291 papers sorted. The three star bills hold: 0 of 34, 0 of 38 and 0 of 20 candidate papers cleared them. Open-source and vendor benchmarks also differ sharply in what they report.
Citation check: not finished yet.
Across the 22 other ledgers we built, what patterns repeat — and can we falsify our own conclusions?
111 records mapped. Three star bills are set aside for the 2027 audit. We will not submit the synthesis preprint until this ledger locks.
Citation check: not finished yet.
AI agents compete on 15 unsolved math problems — we asked which already had a published answer.
222 records mapped. 4 arena problems already match a published result, 3 can be rebuilt from existing work, and 6 are open. The two star bills hold.
Citation check: not finished yet.
Has a quantum computer beaten a regular one at something useful? We sorted 650 papers from 2024-2026.
650 papers sorted. No demonstration passed all the checks. The three star bills hold, but the citations have not been checked yet.
Citation check: not finished yet.
Researchers say they can read what a neural network is 'thinking'. We sorted 484 papers to see if those readings hold up.
484 papers sorted, and half of them rebut earlier claims. The three star bills hold: no interpretability claim on a frontier-size model passes all six checks.
Citation check: not finished yet.
When OpenAI or Google claims a new AI 'aced the exam,' which scores actually mean something?
469 papers sorted. The four star bills hold: no frontier model passes all six checks on any one benchmark. Many vendor score claims do not last long.
Citation check: not finished yet.
Governments regulate AI by how much computing power trains it. We sorted 421 papers to see if that threshold works.
421 papers sorted. The four star bills hold: no compute-governance claim passes all six checks. In the papers we read, compute thresholds as written do not do what they are meant to do.
Citation check: not finished yet.
AI companies say their models resist jailbreaks. We sorted 409 papers on the fixes and the attacks.
409 papers sorted. The four star bills hold: 0 of 8 disclosed defenses lasted 90 days. Apollo's audits undercut the assumption that a model's reasoning can be monitored.
Citation check: not finished yet.
Training methods like RLHF are meant to keep AI models helpful and honest. We sorted 417 papers on them from 2024-2026.
417 papers sorted. The four star bills still hold, but the result is weakened: 18 of 30 citations flagged for a closer check were wrong or invented. Seven key papers checked out.
Citation check: 18 of 30 flagged citations were wrong or invented.
The encryption replacing RSA is built on lattice math. We sorted 635 papers on attacks against it.
635 papers sorted. No fast attack on the real settings was found. The four star bills hold, but the citations have not been checked yet.
Citation check: not finished yet.
When an AI 'thinks before answering,' is the thinking real or is it for show?
394 papers sorted. The three star bills hold: no reasoning-mode claim shows a faithful mechanism, decomposable compute, or coverage of all task types.
Citation check: not finished yet.
AI models claim to 'see' images and video. We sorted 397 papers on how often they look, and how often they guess from captions.
397 papers sorted. The three star bills hold. One audit found 25-40% of 'visual' answers came from caption text, and no model covers all five sub-tasks.
Citation check: not finished yet.
When AI companies release their model weights publicly, what does that actually mean for safety and policy?
371 papers sorted. The three star bills hold: in these papers, safety training is cheap to undo, abilities copy over to smaller models, and no gating policy does what it set out to do.
Citation check: not finished yet.
AI agents that browse the web and run code are sold as autonomous workers. We sorted 368 papers on those claims.
368 papers sorted. The three star bills hold: no agent copes with renamed tools, scores the same across related benchmarks, or covers all six skill areas.
Citation check: not finished yet.
Scaling laws predict how AI improves with more computing power. We sorted 302 papers to see which curves hold up.
302 papers sorted. The three star bills hold: no scaling-law claim carries over cleanly across model designs, data mixes or sizes. In these papers, sudden 'emergence' is mostly a side effect of how it is measured.
Citation check: not finished yet.
AI like AlphaFold predicts the 3D shape of proteins. We sorted 283 papers to see which predictions hold up in the lab.
283 papers sorted. The three star bills hold: only about 11% of designed-protein papers have an independent lab repeat, and models do not carry over cleanly between organisms.
Citation check: not finished yet.
RAG, needle tests, KV cache and 1M-token context. 8,036 hits in Atlas. Strong second after RL-from-Rewards.
Closure-richness criteria pending
Φ, IIT and Tononi. 1,914 hits in Atlas. Rich in overclaims. It would be the first consciousness ledger.
Highest overclaim density in the scoping queue
AlphaEvolve and neural architecture search. 1,015 hits in Atlas. A cousin of the Arena-Attack ledger.
Bridges to the Arena-Attack ledger
Byzantine agreement, CRDTs and Paxos. 3,512 hits in Atlas. The maths is settled, which makes closure-rich claims rare.
Cousin to the Cross-Ledger Bridges ledger
No ledgers match these filters.
Citation checks
Each run takes a small sample of ledger records and looks up their citations in public resolvers. Records that do not match are flagged for review.
Loading the public resolver feed.
At a glance
A map of the claims, their break conditions, their verification state, and the ledgers that carry the most evidence.
The dashboard loads from ledgers.json.
§00 · Across the ledgers
Seven bridges from the cross-ledger self-audit, where the method was pointed at itself. Bills 7★, 9★ and 12★ were declared empty before the audit. Each bridge shows its current status on the right.
Mech Interp Bill 11★, ITS Bill 11★, Reasoning Bill 6★, VLM Bill 4★, Scaling Laws Bill 5★, Agentic Bill 4★ and Bio Bill 4★ all hold empty across 2,000+ LLM-domain papers. The robotics corpus support went away when verification killed all 4 Bill 4 grounded-reward IDs (2026-05-15).
Vendor-claim half-life 73 days · ITS patch 30 days / 36 hours · distilled cousin 3.4 months · Sky-T1 reproduces o1-preview in 2 weeks · BIS rule lived 4 months · ARC-AGI v1→v2 in 3 months. Reported as a 30–100 day range.
ITS Bill 14★ + Open-weight Bill 8★ + VLM Bill 18 + Agentic Bill 11. Lermen-Rimsky: undoing safety is 10× cheaper than installing it. Defenses belong to the deployment surface, not the model.
Open-weight Bill 5★ + Scaling Laws Bill 11★ + Compute Governance Bill 11★. Halevy-Heim-Pilz 0/14 resistant; Mamba2 dense fails 0.06–0.11 on SSM; R1-Distill uses 100–1000× less compute. No architectural moat: capability moves freely; only novel training data sticks.
Anand-Goyal unified-VLM 0/9 · Anand-Bommasani cross-organism 0/8 · Anand-Rein unified-agent 0/9 · Halevy-Heim-Pilz distillation 0/14 · IBBIS synthesis-screened 0/4 · Yang-Bommasani cross-mixture 0/9.
Across 7 ledgers, anti-saturation is the only bill that fires positive. Reframing benchmarks again and again (ARC v1→v2→v3, MMMU→MMMU-Pro, FrontierMath Tier 1→4, LiveCodeBench monthly, Cybench Pro held-out) is the only audit tool that keeps pace with the 30–100 day closure cycle.
China-based vendors disclose 100% (DeepSeek, Alibaba and 01.AI all 8/8 fields); Western vendors disclose 17%. Frontier Apache 2.0 models of 30B or more include Llama 4, Qwen3-MoE 235B and Hunyuan-Large. On current data, the "China = closed and risky, US = open and safe" framing would flip.
Synthesis. The evidence-bearing claim is that the star bills hold across the corpus. The cross-ledger bridges are the interesting result. The discipline: declare the empty-space bills before the audit, and verify before any break ships. Receipts: the Cross-Ledger Bridges ledger and its data. See also the findings board.
Falsification challenge
A paper that passes a star bill is a real result, and the ledger must change. The method commits to a public update of that ledger within 7 days of any verified trigger. Before that, a separate pass looks up the paper's arXiv ID and abstract to confirm it exists and says what was claimed (gate 4 on the Method page).
Changes appear in the agent log. Mistakes are corrected with a new entry; old entries are not rewritten.
The Ledger Compiler keeps these ledgers. It is one of the agents coordinated by CHRONOS, the research system. Meet the agents.