Report says Alma currently ranks in the top in the ongoing Agent evaluation

AT A GLANCE

According to the report,Alma ranks third in both pass rate and cost per success in Runta's ongoing Agent evaluation, currently outperforming Claude Code.

Article

Independent developer Yetone's Alma breaks into Agent the top, beating Claude Code on both pass rate and cost

Dongcha Beating AI News:Alma, developed under the leadership of independent developer Yetone, ranked third in both task pass rate and cost per success in the latest AI Agent comparison, outperforming well-known tools including Claude Code and OpenCode.

The evaluation was launched by AI Agent infrastructure company Runta. In the current leaderboard snapshot,Alma has a task pass rate of 72.7%, behind only Codex CLI at 77.9%and Pi at 72.8%. Claude Code stands at 70.0%。

Alma not only has a higher pass rate but is also cheaper per completed task. Its average cost per successfully completed problem is approximately 0.80 USD, compared with 0.94 USD for Claude Code, approximately 16%lower.

This Frontier Harness v1.1 examines a specific question: How much can performance differ when the same large language model is used with a different Agent execution framework?

The evaluation covers 21 Harness configurations, 5 large language models and 62 tasks, each repeated 4 times, for a planned total of 26,040 runs. All tools run in a standardized cloud environment and are compared on task success rate,Token cost and execution speed.Alma is a AI Agent focused on local operation and long-term memory. It can directly read and modify code, execute terminal commands and search the web, and also supports Skills and MCP. It can switch freely between underlying models and does not depend on any single model provider.

Recently,Alma also integrated Herdr. Coding Agent such as Claude Code, Codex and Pi can continue to run in Herdr, while Alma brings them together in a single chat interface for viewing execution progress, answering questions and approving actions. Users can even remotely manage multiple Agent on their computers via iPhone.

The evaluation is still ongoing, and rankings may change. The models tested are limited to open-source models such as GLM, DeepSeek, Kimi and Muse; the closed-source GPT and Claude model families were not covered.

Source: https://m.theblockbeats.info/flash/376710

Key points

The report says Alma was developed under the leadership of independent developer Yetone, and the evaluation was launched by Runta.
In the current leaderboard snapshot,Alma has a task pass rate of 72.7%，Codex CLI is at 77.9%, Pi at 72.8%, and Claude Code at 70.0%。
Alma The average cost per successfully completed problem is approximately 0.80 USD, compared with 0.94 USD for Claude Code; the report says the cost is approximately 16%。
Frontier Harness v1.1 covers 21 Harness configurations, 5 large language models and 62 tasks, each repeated 4 times, for a planned total of 26,040 runs; all tools operate in a standardized cloud environment.
The original article says Alma focuses on local operation and long-term memory, and supports reading and modifying code, executing terminal commands, searching the web,Skills、MCP and switching underlying models.
The original article says Alma has integrated Herdr, enabling centralized management of multiple coding Agent and supporting remote management via iPhone.
The evaluation is ongoing and rankings may change; the models tested are limited to open-source models such as GLM, DeepSeek, Kimi and Muse, and do not include the closed-source GPT and Claude model families.

AI-assisted interpretation

The following is analysis, separate from reported facts. Verify important claims independently.

The evaluation examines how task performance differs when the same large language model is used with different Agent execution frameworks. Based on the current data reported,Alma passes a higher proportion of tasks and has a lower average cost per success than Claude Code; this is not a comprehensive comparison of the Claude model family itself.

Why it matters to readers

The report identifies metrics to consider when choosing Agent tools: task pass rate and cost per success. However, the current rankings apply only within the scope of the evaluation described in the article.

A Alma can be understood as a tool that calls models and performs actions. When choosing one, do not rely on rankings alone; also consider the models used, the scope of the tasks and whether the evaluation is complete.

Risks and unknowns

The evidence comes from a media report, which did not provide a link to the original leaderboard; the data and feature descriptions have not been independently verified.
The evaluation is ongoing, and the current rankings do not represent the final results.
The evaluation did not cover the closed-source GPT and Claude model families, so it cannot be used to assess how the tools perform with those models.
The source does not specify the underlying model configuration associated with each tool's current data or the number of completed runs.

Report says Alma currently ranks in the top in the ongoing Agent evaluation

Article

AI-assisted interpretation

Why it matters to readers

Risks and unknowns

Related Developments

Related concepts

Keep following

Security Center