Perplexity: decision-model benchmark claims, EVE, and the Computer drip
From the AI digest for 2026-10-10
What happened: Arav's feed: pplx-decider-v1.1-27b is on OpenRouter (multimodal — text/JSON/images; returns typed decisions; output free); DecisionBench #1 claimed twice (93.9% acc/949 cases/534ms median → refined Oct 9: 94.5% on 1,071 cases, $0.017/1,000 decisions — "ranks first on DecisionBench, and lowest cost too"); a clinical-decision claim (643/669, "best quality and lowest cost for clinical decisions"); and EVE — an inference stack demoing decider "4× faster than JEV" (128 questions across a lease and an offer letter in one request per document; Millionaire speed-run demo earlier). Computer side: a website editor with queued element-level changes, thumbnail rail, CAD/FreeCAD export, its own brand-identity build. Plus: pplx-embed-v2-late open-sourced (multi-vector embeddings, 9B + 0.6B, shared text/image space).
What's genuinely new: the distribution race is now three-way at the decision layer (OpenRouter carrying Perplexity's model; Microsoft's Foundry and soon-OpenRouter carrying Microsoft-Decision-1 — see today's ai digest), and the benchmark layer is consolidating on shared named boards (DecisionBench, HF Decision Index 0.3) — all vendor-run, still no independent runner. EVE as a demo stack is new — an unknown-in-our-record component; provenance unclear.
Watch: the never-run independent comparison (now Jev vs d1 vs Decisions API vs pplx-decider vs Clef vs Decision-1); whether EVE ships as a product; clinical-claim verification (health-adjacent claims need the primary).
Links: Arav (feed; several RTs) · DecisionBench claim (Oct 8)