Artificial Analysis closes its benchmarks — and measures open models falling further behind
From the AI digest for Oct 28, 2026
Sources Ethan Mollick on X (via steipete) The Verge ActuIA
What happened: In a post relayed by Ethan Mollick late on Tuesday, October 27 (~11:00 PM ET), Artificial Analysis moved its model index to v5 and switched to closed benchmarks, dropping the public GDPval-AA suite. On the new scoreboard the leading open-weights entry — Kimi K3.1 Pro, whose weights have not yet been released — scores 46, twelve points behind the overall leader, compared with a three-point gap in v4.3. Among models with released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) score highest at 32. ActuIA's write-up the next evening (Wednesday, October 28, ~6:00 PM ET) puts numbers on the reset for models already scored: twelve of them fell 12–16 points on a board now weighted 60% toward private tests — among them Mistral Large 4 ("Le Chonk"), down from 38 to 25, now below the 32s of DeepSeek and GLM.
What's genuinely new: two things at once. The measured open-versus-frontier gap widens sharply (three points to twelve), and the scoreboard itself becomes unverifiable — closed benchmarks cannot be independently re-run the way GDPval-AA could, a caveat Mollick states plainly ("take with a grain of salt"). He adds the absence the numbers imply: no Fable-class open model has appeared. Published coverage beyond the initial relay now quantifies how steep the switch is for previously scored models: Mistral Large 4's thirteen-point fall puts the European open-weights flagship behind DeepSeek and GLM on the new board. On Wednesday, October 28, the leader got a name: Google announced Gemini 4 Carbon as the #1 model on the v5.0 index at 59, with Claude Opus 5.5 and GPT-6 Astra trailing — thirteen points above the best open-weights score if the vendor's claim stands on the same board.
Watch: whether other independent evaluators also move behind closed benches, which would remove the main third-party check on vendor benchmark claims; a full published v5 table naming the other eleven models that lost 12–16 points; the actual release of Kimi K3.1 Pro's weights; whether any released model regains ground in a later v5 revision.