Article
AA Major overhaul of the Intelligence Index: Claude takes the lead, while DeepSeek and GLM trail behind 26 points
Beating AI breaking news,AI evaluation firm Artificial Analysis has significantly revised its Intelligence Index leaderboard. The new version adds 132 programming tasks from real open-source projects. The code written by AI must not only pass tests, but also meet the standard that project maintainers would be willing to merge. It also adds 70 research tasks, requiring AI to actually complete the research workflow in a terminal. To reduce the impact of models having seen the questions in advance, the new version's 60%scores are weighted based on private test sets or hidden answers.
Anthropic and OpenAI still dominate the top of the leaderboard. Claude Opus 5.5 scores 58 points to take first place, while GPT-6 Astra and GPT-6.1 Sol score 57 points and 55 points, respectively. Kimi K3.1 Pro scores 46 points, but its weights have not yet been released.
The open-weight DeepSeek V4.1 Flash and GLM-5.3 each score only 32 points, trailing the leader by a full 26 points and even falling below Claude Haiku 5.5 at 36 points.
In the newly added research tasks,GPT-6 Astra achieves a completion rate of 63%,Kimi K3.1 Pro records 34%,DeepSeek V4.1 Flash manages only 13%.
Although GPT-6.1 Sol scores 5.5 points lower than Claude Opus 3, based on publicly listed API prices, the estimated average cost per comprehensive evaluation task is only 1.27 USD, compared with 10.04 USD, nearly 8 times as much.
The new version's questions are also more demanding overall, with the median output Token for each task rising from Token in the previous version to 35000 in the new version.
Original link 59000 https://m.theblockbeats.info/flash/374127