Artificial Analysis closes its benchmarks — and measures open models falling further behind
From the AI digest for Oct 28, 2026
What happened: In a post relayed by Ethan Mollick late on Monday, October 27 (~11:00 PM ET), Artificial Analysis moved its model index to v5 and switched to closed benchmarks, dropping the public GDPval-AA suite. On the new scoreboard the leading open-weights entry — Kimi K3.1 Pro, whose weights have not yet been released — scores 46, twelve points behind the overall leader, compared with a three-point gap in v4.3. Among models with released weights, DeepSeek V4.1 Flash 1017 (max) and GLM-5.3 (max) score highest at 32.
What's genuinely new: two things at once. The measured open-versus-frontier gap widens sharply (three points to twelve), and the scoreboard itself becomes unverifiable — closed benchmarks cannot be independently re-run the way GDPval-AA could, a caveat Mollick states plainly ("take with a grain of salt"). He adds the absence the numbers imply: no Fable-class open model has appeared. For the released-weights tier, the 32 scores for DeepSeek V4.1 Flash and GLM-5.3 are the first v5 datapoints for the open-weights models this storyline tracks.
Watch: whether other independent evaluators also move behind closed benches, which would remove the main third-party check on vendor benchmark claims; the actual release of Kimi K3.1 Pro's weights; whether any released model regains ground in a later v5 revision.