According to ChainCatcher, Alma, a local AI agent developed by independent developer Yetone, recorded a 72.7% task pass rate in a benchmark launched by AI agent infrastructure company Runta, versus 70.0% for Claude Code. Alma’s rate trailed Codex CLI at 77.9% and Pi at 72.8%. Alma averaged $0.80 per successfully completed task, about 16% less than Claude Code’s $0.94, and ranked third in both pass rate and successful-task cost.
The ongoing FrontierHarness v1.1 evaluation compares the same models across different agent frameworks. It covers 21 harness configurations, five models and 62 tasks, with each task repeated four times; 26,040 runs are planned, 72 times the scale of the previous version. Tools run in a standardized cloud environment and are assessed on task success, token cost and execution speed. The tested models include open-source GLM, DeepSeek, Kimi and Muse models; closed-source GPT and Claude models are not included.