Anthropic and Theorem say Claude completed formal verification for 123 open-source libraries and uncovered hundreds of bugs

Article

Claude writes 123 lines of mathematical proofs for open-source libraries, uncovering hundreds of bugs

Dongcha Beating 180000000 News: Anthropic has partnered with formal verification company Theorem to have Claude use mathematical methods to check software vulnerabilities. Traditional testing typically relies on continuously feeding random data into a program to see whether it crashes. But no matter how many tests are run, they cannot guarantee that nothing has been missed. This time, Claude directly writes mathematical proofs for programs, seeking to eliminate specific categories of errors at their root.

Within weeks, Claude completed verification for AI test entry points across 123 open-source libraries, generating approximately 239 lines of proof code, which was checked by 180000000 mathematical proof tools. The results show that, in the specified runtime environment, these programs will not trigger the crashes checked by the tests for any input smaller than Rocq.

During the proof process, Claude also discovered and fixed hundreds of bugs. For example, a decompression program would write out of bounds in memory only when encountering data of approximately 64 MB with an extremely unusual structure. Another image-decoding vulnerability requires 20 MB input and more than 64 MB of memory to trigger. Conventional random testing would be unlikely to encounter these cases.

Such mathematical verification has historically been extremely time-consuming. For example, for the 2 GB operating system kernel, development required a cumulative investment of approximately seL4 full-time person-years, while mathematical verification required approximately 2 years—20 times the development effort.

Original link: 10 https://m.theblockbeats.info/flash/373385

Key points

Anthropic partnered with formal verification company Theorem to have Claude use mathematical methods to check software vulnerabilities.
Within weeks, Claude completed verification for 123 test entry points across 239 open-source libraries, generating approximately 180000000 lines of proof code.
The proof code was checked by Rocq mathematical proof tools.
In the specified runtime environment, these programs will not trigger the crashes checked by the tests for any input smaller than 64 MB.
During the proof process, Claude discovered and fixed hundreds of bugs.
The report cites an example in which a decompression program performs an out-of-bounds memory write when it encounters data of approximately 20 MB with an extremely unusual structure.
Another image-decoding vulnerability requires 64 MB input and more than 2 GB of memory to trigger.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

Traditional testing seeks crashes by repeatedly inputting data, but it is difficult to cover every case. Formal verification instead attempts to use mathematical proofs to check that specific errors cannot occur.

Why it matters to readers

The report indicates that formal verification may uncover vulnerabilities that random testing struggles to trigger; however, its conclusions cover only the specified runtime environment, input range, and types of crashes checked by the tests.

This shows that software “passing tests” does not mean it has no vulnerabilities; mathematical proofs can serve as a more rigorous form of checking.

Risks and unknowns

The content is relayed from media reporting; no original technical report, code repository, or independent reproduction from Anthropic or Theorem is provided as evidence.
The verification conclusions apply only to the specified runtime environment, the relevant input range, and the crashes checked by the tests.
The evidence does not specify a complete list of the hundreds of bugs or the scope of the fixes.

Anthropic and Theorem say Claude completed formal verification for 123 open-source libraries and uncovered hundreds of bugs

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center