Evaluation sources/ProgramBench

ParticipatingCoding

ProgramBench

Vals AI · Public model evaluation results from ProgramBench.

Official evaluation

Source description

This source measures concrete delivery performance in Coding; raw scores are shown separately from the normalized capability score.

Limitation: Different versions, reasoning settings, and test sets may change scores; read them with the data date.

How it is used

Participating sources affect capability scores; reference, cross-reference, and observing sources are for comparison and verification only.

Model results from this source


Claude Fable 5.2	19	100.0	2026-11-10
Claude Opus 5.5	18.5	100.0	2026-11-05
Claude Fable 5.1	7	71.5	2026-11-05
Claude Sonnet 5.5	6.5	69.5	2026-11-05
GPT-6 Astra	5.5	65.4	2026-11-05
Claude Opus 5	3	55.4	2026-11-05
GPT-6.1 Sol	3	55.4	2026-11-05
Claude Haiku 5.5	2.5	53.4	2026-11-05
Gemini 4 Argon	2.5	53.4	2026-11-05
Muse Spark 1.3	2.5	53.4	2026-11-05
Claude Fable 5	2	51.4	2026-11-05
Kimi K3	2	51.4	2026-11-05
GPT-6 Sol	2	51.4	2026-11-05
GPT-5.6 Sol	1.5	49.4	2026-11-05
GLM 5.3	1.5	49.4	2026-11-05
Claude Opus 4.8	1	47.3	2026-11-05
Gemini 3.8 Flash	1	47.3	2026-11-05
Grok 4.6	1	47.3	2026-11-05
Claude Sonnet 4.6	0.5	45.3	2026-11-05
DeepSeek V4.1 Flash	0.5	45.3	2026-11-05
Muse Spark 1.2	0.5	45.3	2026-11-05
GPT-5.4	0.5	45.3	2026-11-05
GPT-5.5	0.5	45.3	2026-11-05
GPT-5.6 Terra	0.5	45.3	2026-11-05

Page 1 of 3

Frequently asked questions