Hermes model leaderboard launches, Opus 5.5 ranks first, while Astra ranks second but is the most expensive

Article

Beating AI news flash: Nous Research has given Hermes Agent launched a model leaderboard Hermes Index, primarily for comparing how different models perform in Hermes. All models use the same Hermes Agent runtime framework, and models supporting adjustable reasoning intensity are uniformly set to high. The leaderboard score is the average of four tests—Hermes Bench, Terminal-Bench 4, Terminal-Bench Science, and SkillsBench—while also tracking the average cost per task.

Hermes Bench is a new test developed by Nous Research itself, consisting of 150 tasks across 25 types of scenarios, covering Skills, research, charts and creative work, memory, tool calling, and safety.Agent directly handles the workspace and real files, and is ultimately scored based on the generated files, task status, and tool-use records.

The first edition tested 14 models. Claude Opus 5.5 ranked first with a score of 63.31,GPT 6 Astra ranked second with 56.25 points, and Claude Sonnet 5.5 ranked third with 53.14 points. Astra costs 11.61 USD on average per task, the highest on the leaderboard. Opus 5.5 scored 4.99 USD，Sonnet 5.5 scored 2.82 USD; some data for both are still officially marked as provisional.

Among the low-cost models, DeepSeek V4.1 Flash scored 36.91 points, with an average cost per task of 0.259 USD；GPT 6 Luna scored 33.89 points, at 0.141 USD per task. Both entered the cost-performance Pareto frontier, meaning that no model is both cheaper and higher-scoring.

Original article link https://m.theblockbeats.info/flash/370534

Key points

Nous Research has Hermes Agent launched a model leaderboard Hermes Index。
All models use the same Hermes Agent runtime framework; models supporting adjustable reasoning intensity are uniformly set to high.
The leaderboard score is the average of Hermes Bench, Terminal-Bench 4, Terminal-Bench Science, and SkillsBench, and the average cost per task is also reported.
Hermes Bench contains 150 tasks and 25 types of scenarios.
The first edition tested 14 models; Claude Opus 5.5 scored 63.31 points to rank first,GPT 6 Astra scoring 56.25 points to rank second, and Claude Sonnet 5.5 scoring 53.14 points to rank third.
GPT 6 Astra's average cost per task is 11.61 USD, the highest on the leaderboard; Claude Opus 5.5 costs 4.99 USD，Claude Sonnet 5.5 costs 2.82 USD。
DeepSeek V4.1 Flash scored 36.91 points, averaging 0.259 USD；GPT 6 Luna scored 33.89 points, averaging 0.141 USD。
DeepSeek V4.1 Flash and GPT 6 Luna entered the cost-performance Pareto frontier.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

This leaderboard compares different models under the same Agent framework and four tests, while showing task scores and average costs. The results indicate that the highest-scoring model is not necessarily the least expensive; low-cost models may also have an advantageous combination of price and score.

Why it matters to readers

The leaderboard provides both performance and cost per task, allowing an initial comparison of models' results and usage costs in Hermes Agent tasks.

Beginners can start with two metrics: the score represents test performance on the leaderboard, while average cost per task represents the average expense of completing one task; the Pareto frontier means there is no model that is both cheaper and higher-scoring.

Risks and unknowns

Claude Opus 5.5 and Claude Sonnet 5.5 still have some data officially marked as provisional.
No subsequent retest results were provided.
The evidence does not provide results from other independent tests corroborating this ranking.

Hermes model leaderboard launches, Opus 5.5 ranks first, while Astra ranks second but is the most expensive

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center