Artificial Analysis details AA-Omniscience v1.1, with a smaller dataset and a new public set

Artificial Analysis published the details of AA-Omniscience v1.1, its knowledge and hallucination benchmark, which it updated on October 27 as part of v5 of its Intelligence Index. The benchmark's Net Score rewards correct answers, penalizes wrong ones and treats abstaining as neutral.

What changed, according to Artificial Analysis:

• The private dataset shrinks from 6,000 to 3,000 questions: public and saturated questions are removed, errors and ambiguities fixed, and new questions added. The questions still span 6 domains and 42 topics.
• A new 300-question public set is on Hugging Face, calibrated to the difficulty of the private set. It says public and private scores correlate at 0.99 (Pearson and Spearman) for Net Score.

Its own measurements:

• Gemini 4 Carbon (high) leads with a Net Score of 54, followed by Claude Opus 5.5 (48) and Claude Fable 5.1 (46). Muse Spark Open (xhigh) is the leading open-weights model, 28 points behind Gemini 4 Carbon.
• Rank order is largely preserved against v1 (0.98 Spearman correlation), but scores average about 1.9 points lower. MiniMax M3.1 Flash drops 22 points (16 to -6) and MiMo V2.6 Pro 13 points (8 to -5).
• Claude Fable 5.1 has the highest accuracy at 68%, ahead of Claude Opus 5.5 at 66%. Gemini 4 Argon (high) has the lowest hallucination rate, 4%; the median hallucination rate falls 1.4 points.