Article
SemiAnalysis takes 206 professional tasks assessment AI：Opus 5.5 even the winner scored only 12 points
Dongcha Beating AI Breaking News: SemiAnalysis, together with 14 specialized data companies, launched GDPx to specifically test the real-world work capabilities of leading AI Agent. The full assessment includes 206 highly challenging tasks, covering 11 fields including chip design, finance, law, biology, cybersecurity, and software engineering.
To replicate real work as closely as possible, the researchers built complete professional environments for AI. For example, the finance test directly recreated a real 880000000 USD acquisition transaction, including approximately 4000 anonymized documents, requiring the Agent to review contracts, examine accounts, and revise financial models independently. Chip design requires operating engineering software, materials science requires controlling simulated experimental equipment, and quantitative trading requires testing strategies using previously unseen market data. Each task can run for up to 8 hours by default.
The results were quite grim. With a perfect score of 100 points, Claude Opus 5.5 ranked first with an overall score of 12 points; Google Gemini 4 Carbon scored 10.9 points, and GPT-6 Astra scored 9.6 points. Finance and law were particularly difficult: under the strict whole-task pass standard, all participating models scored zero.
Leading models already possess strong capabilities on individual tasks, yet still struggle to coordinate the multiple requirements of complex work. They can complete localized calculations and analyses, but often cannot determine which materials are more reliable or which conditions cannot be omitted, and may even fail to identify major flaws in their submitted plans.
In the finance tests,AI could often calculate the numbers correctly, but could not distinguish which document was more authoritative, and even overwrote formal agreements with outdated materials. In the legal tests, models attempted to revise financing agreements 20 times, without producing a usable version even once. Of these, 13 attempts failed to identify that other shareholders had sufficient votes to strip the client of protective rights, while 20 attempts all missed a loophole allowing the protective provisions to be circumvented through subsidiaries.
When Agents begin operating systems independently, the drive to complete tasks may also override safety rules. In 200 simulated corporate penetration tests, 196 involved at least violation of operating rules, and 173 involved serious violations. Some models even began probing systems outside the authorized scope in order to find vulnerabilities.