Skip to main content
Evaluation sources/AutomationBench-AA
ParticipatingProfessional work

AutomationBench-AA

Artificial Analysis · Public model evaluation results from AutomationBench-AA.

Official evaluation
Source description

This source measures concrete delivery performance in Professional work; raw scores are shown separately from the normalized capability score.

Limitation: Different versions, reasoning settings, and test sets may change scores; read them with the data date.

How it is used

Participating sources affect capability scores; reference, cross-reference, and observing sources are for comparison and verification only.

Model results from this source

Gemini 4 Carbon0.7781660675.52026-11-11
Gemini 4 Argon0.7751346975.32026-11-05
Claude Fable 5.20.7642233874.72026-11-10
MiniMax M3.1 Flash0.725218172.62026-11-05
Claude Sonnet 5.50.7176882172.22026-11-05
Muse Wonder Full0.713307572.02026-10-18
Muse Wonder Full0.713307572.02026-10-17
Muse Wonder Full0.713307572.02026-11-05
Claude Opus 5.50.6953868171.02026-11-05
DeepSeek V4.1 Flash0.6889097870.72026-11-05
DeepSeek V4.1 Flash 10170.6849296570.42026-11-05
GPT-6 Astra0.6849174870.42026-11-05
Muse Wonder0.6809455170.22026-10-18
Muse Wonder0.6809455170.22026-11-05
Muse Wonder0.6809455170.22026-10-17
Ling-3.1-flash-10190.6754660769.92026-11-05
Grok 4.60.6699736869.62026-11-05
DeepSeek V4.1 Flash 10170.6664095469.42026-10-31
GPT-6.1 Sol0.6657372169.42026-11-05
Grok 4.70.6555835168.82026-11-05
MiMo V2.6 Flash0.6405168368.02026-10-18
MiMo V2.6 Flash0.6405168368.02026-11-05
MiMo V2.6 Flash0.6405168368.02026-10-17
GLM 5.30.6220286567.02026-11-05
Ling-3.1-flash0.6174373466.82026-11-05
Ling-3.1-flash0.6174373466.82026-10-18
Ling-3.1-flash0.6174373466.82026-10-17
GPT-6 Sol0.6169423566.72026-11-05
Gemini 3.8 Flash0.6088997466.32026-11-05
GLM-5.3 Flash0.6036862866.02026-10-18
GLM-5.3 Flash0.6036862866.02026-11-05
GLM-5.3 Flash0.6036862866.02026-10-17
GPT-6.1 Terra0.6016762365.92026-11-05
GPT-5.6 Sol0.600811565.92026-11-05
Mistral Large 40.5990136265.82026-10-28
GPT-5.6 Terra0.5964995865.62026-11-05
Claude Fable 5.10.5937591765.52026-11-05
MiMo V2.6 Pro0.5862340965.12026-11-05
Kimi K30.5827297864.92026-11-05
Grok 4.50.579348664.72026-11-05
Muse Spark 1.30.5785343864.72026-11-05
Mistral Large 40.5783010564.62026-11-05
DeepSeek V4 Pro 08130.5671165564.02026-11-05
Claude Opus 50.5657355664.02026-11-05
DeepSeek V4 Pro0.5628968663.82026-11-05
Claude Fable 50.5407130162.62026-11-05
DeepSeek V4 Flash 07310.5396771962.52026-11-05
GPT-6 Luna0.531681462.12026-11-05
Gemini 3.6 Flash0.5301669562.02026-11-05
Step 5 Preview0.5095659460.92026-11-05
GPT-5.6 Luna0.5020861860.52026-11-05
Qwen3.8 Max0.4923210460.02026-11-05
GPT-5.50.4730056658.92026-11-05
Claude Opus 4.80.4559343758.02026-11-05
Muse Spark 1.20.4061337855.32026-11-05
Muse Spark 1.10.3884891654.32026-11-05
Claude Sonnet 50.3651267653.02026-11-05
Claude Haiku 5.50.359538952.72026-11-05
Gemini 3.1 Pro Preview0.3542771852.52026-11-05
GLM-5.20.2839897748.62026-11-05
GPT-5.4 mini0.2620488747.42026-11-05
Gemini 3.5 Flash Lite0.2501127146.82026-11-05
Kimi K2.7 Code0.2445430546.52026-11-05
Qwen3.7 Max0.229768945.72026-11-05
MiniMax-M30.2125125844.82026-11-05
GLM-5.10.2030835544.22026-11-05
Claude Sonnet 4.60.2009807444.12026-11-05
MiMo-V2.50.1842951243.22026-11-05
Hy30.1840912843.22026-11-05
Nex-N2-Pro0.1762911742.82026-11-05
Qwen3.7 Plus0.1739122142.72026-11-05
Claude Sonnet 4.50.1407574340.82026-11-05
MiMo-V2.5-Pro0.1370361740.62026-11-05
Kimi K2.60.130207640.32026-11-05
Qwen3.6 27B0.1104522439.22026-11-05
Qwen3.5 397B-A17B0.068866736.92026-11-05
Gemini 3.1 Flash Lite0.0679618236.92026-11-05
GPT-5 mini0.0648746636.72026-11-05
Mistral Medium 3.50.0629964936.62026-11-05
Qwen3.5 122B-A10B0.0548111636.22026-11-05
Qwen3.6 35B-A3B0.0520856636.02026-11-05
Inkling0.049848935.92026-11-05
Inkling Small0.0481422835.82026-11-05
LongCat-2.00.0453391335.72026-11-05
MiniMax-M2.70.044352935.62026-11-05
Nemotron 3 Super 120B A12B0.0377903135.32026-11-05
Claude Haiku 4.50.0324863135.02026-11-05
Ling-3.0-flash (free)0.0321964934.92026-11-05
Nemotron 3 Ultra 550B A55B0.029863534.82026-11-05
DeepSeek V3.1 Terminus0.0270627334.72026-11-05
Gemini 2.5 Pro0.0217494634.42026-11-05
Command A Plus0.0181045434.22026-11-05
Mercury 20.0167977734.12026-11-05
Mistral Large 30.014974734.02026-11-05
North Mini Code0.0149128634.02026-11-05
Grok 4.30.0137596533.92026-11-05
Trinity Large Thinking0.0130692333.92026-11-05
Qwen3 Coder Next0.0105935533.82026-11-05
Qwen3.5 9B0.0104738733.82026-11-05
Nemotron 3 Nano 30B A3B0.0097146933.72026-11-05
Qwen3 32B0.0097112933.72026-11-05
Ring-2.6-1T0.0071786633.62026-11-05
Gemma 4 12B Base0.0068557233.62026-11-05
Ministral 3 3B0.0066586633.62026-11-05
Ministral 3 14B0.0055121333.52026-11-05
o3-mini0.0053018333.52026-10-11
o3 Mini High0.0053018333.52026-11-05
Ministral 3 8B0.0051674933.52026-11-05
Solar Pro 30.0049734233.52026-11-05
Qwen3 8B0.0033260733.42026-11-05
DeepSeek V3 03240.0027298633.32026-11-05
DeepSeek-R10.0024185333.32026-11-05
Llama 4 Maverick0.0021219233.32026-11-05
gpt-oss-120b0.001990733.32026-11-05
gpt-oss-20b0.0019111733.32026-11-05
Qwen3 14B0.0019111733.32026-11-05
Llama 4 Scout0.0019111733.32026-11-05

Frequently asked questions