CertiK and Columbia University researchers release the technical results of TensorLean
BIBIBI
AT A GLANCE
According to reports, CertiK and researchers at Columbia University released TensorLean, which verified 83 GPU kernel instances and identified 38 bugs.
Article
CertiK and Columbia University researchers have released TensorLean, achieving the world's first formal verification of production-level AI operators for DeepSeek and FlashAttention
BlockBeats reports that on November 21, CertiK announced the technical achievement of TensorLean jointly developed with Columbia University researchers, completing the world's first formal verification of production-level AI operators. This breakthrough enables mathematical proofs to be applied to real-world AI computation code that has undergone complex performance optimizations, providing model manufacturers with stricter correctness guarantees when improving training speed and reducing inference costs.
The research team evaluated 149 compiled instances within GPU repositories and verified through formal proof that 83 of them conform to the corresponding Lean specifications. This study covered GPU architectures across different generations and involved DeepSeek's DeepGEMM and the widely used FlashAttention operator library. The team discovered a total of 6 correctness issues across 38 libraries, including 2 in DeepGEMM and 23 in FlashAttention.
This research provides an example of applying formal verification to real-world AI computations. As large models continue to undergo performance optimization and hardware adaptation, underlying code also requires continuous adjustments. TensorLean allows developers to repeatedly verify correctness after code updates, helping reduce the risk of computational errors, rework during training, and wasted computing power.
According to BlockBeats, on November 21, CertiK announced the technical achievements of TensorLean jointly attained with researchers from Columbia University.
02
The research team evaluated 149 compiled GPU kernel instances and formally verified through proof that 83 of them conform to the corresponding Lean specifications.
03
Research covers GPU architectures across different generations, involving DeepSeek’s DeepGEMM and FlashAttention operator libraries.
04
The team found 38 critical issues in 6 libraries, including 2 in DeepGEMM and 23 in FlashAttention.
05
The report calls this achievement the first time in the world that a production-level AI operator has been formally verified, and states that TensorLean enables repeated verification of correctness after code updates.
AI-assisted interpretation
The following is analysis, separate from reported facts. Verify important claims independently.
This research uses mathematics to verify whether the performance-optimized AI underlying code meets the specified specifications. 83 instances have passed validation, meaning they comply with the corresponding Lean specifications, but it does not mean that all evaluated codes have achieved correctness assurance.
Why it matters to readers
According to reports, this verification method can provide a stricter guarantee of correctness for real AI computational code, and help reduce the risks of computational errors, retraining work, and compute waste due to code updates.
It can be understood as using mathematical methods to check whether AI computational code works according to specifications; when reading, it's necessary to distinguish between the number of evaluations, the number of passes, and the number of issues found.
Risks and unknowns
No research papers, proof details, or independent verification information are provided, so it is impossible to confirm the claim of “first in the world”.
The verification status and reasons for the remaining 66 evaluated instances are not specified, so it cannot be concluded that all of them contain errors.
The material does not specify the severity, actual impact, or repair status of the 38 correctness issues.
The verification conclusions are limited to the corresponding Lean specifications; the specification coverage of the materials is not specified.
Related Developments
Loading event timeline…
Related concepts
Formal verification
A verification method that expresses contract logic and expected behavior as mathematical statements and uses automated tools to check whether they are consistent.