Artificial Analysis evaluation shows Qwen4-Max comprehensive score of 43 points, lower cost than Kimi but weaker programming performance
BIBIBI
AT A GLANCE
According to reports, Qwen4-Max's comprehensive score is only 3 points behind Kimi, with a cost per task about 11% lower, but it still significantly lags in some programming tests.
Article
Qwen4-Max closes in on Kimi: 3 points behind overall, 11% lower cost, but programming gap remains significant
Dongcha Beating AI flash news: Independent evaluation agency Artificial Analysis has released the official scores for Qwen4-Max. At the highest xhigh reasoning tier, the model achieved a comprehensive intelligence index of 43 points, up 9 points from the previous generation Qwen3.8-Max's 34 points, and 3 points behind Kimi K3.1 Pro's 46 points.
Qwen4-Max surpassed its predecessor in all 9 evaluations. The accuracy rate on the high-difficulty knowledge reasoning test HLE-Diamond increased from 24% to 45%, and the enterprise software automation test improved by 17 percentage points. In AA-Briefcase, which simulates real business document processing, Qwen4-Max scored 1686 Elo, slightly higher than Kimi K3.1 Pro's 1674.
Programming capability remains a weakness. In the Terminal-Bench 4.0 test, Qwen4-Max scored 48%, significantly behind Kimi K3.1 Pro's 67%. In FrontierCode 2.0, which assesses real open-source project development capabilities, both scored 26%, behind Tencent Hy4's 31%.
In terms of cost, Qwen4-Max's average cost per task is 3.27 USD, down from the previous generation's 10.02 USD, a decrease of about 67%, and about 11% cheaper than Kimi K3.1 Pro's 3.68 USD.
In addition to API price reductions, the model itself has also reduced Token consumption. Qwen4-Max outputs an average of about 105000 Tokens per task, a 39% decrease from the previous generation's 171000. However, Kimi K3.1 Pro only requires about 72000 Tokens.
Report cites Artificial Analysis evaluation: at the highest xhigh reasoning tier, Qwen4-Max's comprehensive intelligence index is 43 points, previous generation Qwen3.8-Max is 34 points, and Kimi K3.1 Pro is 46 points.
02
Qwen4-Max surpassed its predecessor in all 9 evaluations; HLE-Diamond accuracy increased from 24% to 45%, and enterprise software automation test improved by 17 percentage points.
03
In the AA-Briefcase test, Qwen4-Max scored 1686 Elo, while Kimi K3.1 Pro scored 1674.
04
In Terminal-Bench 4.0, Qwen4-Max scored 48%, while Kimi K3.1 Pro scored 67%; in FrontierCode 2.0, both scored 26%, while Tencent Hy4 scored 31%.
05
Qwen4-Max's average cost per task is 3.27 USD, down about 67% from the previous generation's 10.02 USD, and about 11% lower than Kimi K3.1 Pro's 3.68 USD.
06
Qwen4-Max outputs an average of about 105000 Tokens per task, a 39% decrease from the previous generation's 171000; Kimi K3.1 Pro is about 72000 Tokens.
AI-assisted interpretation
The following is analysis, separate from reported facts. Verify important claims independently.
This report compares the models' scores and task costs in different tests. Qwen4-Max has improved comprehensively over its predecessor, with a comprehensive score close to Kimi, and slightly higher than Kimi in the business document processing test; but the programming results are inconsistent: significantly behind in one test and tied in another. Therefore, the closeness in comprehensive scores cannot be interpreted as closeness in all capabilities.
Why it matters to readers
The material shows both capability and cost differences: Qwen4-Max has a lower cost per task, but its programming performance still has shortcomings. When choosing a model, it is necessary to consider specific tasks rather than just looking at the overall ranking.
If you are concerned about document processing, you can pay attention to the AA-Briefcase results; if you are concerned about programming, you should look at the two programming tests separately. The cost per task in the article is evaluation data and cannot be directly regarded as your actual usage bill.
Risks and unknowns
The current material is a paraphrase and does not provide the original evaluation page of Artificial Analysis. The scores and costs have not been independently verified.
The material does not explain the complete testing methods, cost calculation standards, and actual usage conditions, so it is impossible to confirm whether these results can be applied to specific businesses.
Related Developments
Loading event timeline…
The material does not provide user adoption, revenue, or market share data, so the commercialization impact cannot be confirmed based on this.
Related concepts
Comprehensive intelligence index
This term is not in the glossary yet. Browse related concepts in the glossary.