Arena open-sources two web-development datasets from its Interaction-as-a-Judge study

Arena is releasing two datasets: ArenaWebdevPreferences, with 6,390 app pairs carrying human preference labels, and ArenaWebdevCorrectness, with 189 game versions and 362 human-verified requirements.

They come from Arena's Interaction-as-a-Judge study. Arena reports these results:

• Computer-use agents that interact with an app detect more errors than judges that only read the code: 71.1% against 63.0%.
• Used as a reinforcement-learning reward, the approach lifts Qwen-3.5-9B's win rate by 6.7 points, about 50 Arena points.
• Iterative repair fixes 75.5% of broken requirements, against 55.4% with a code-only judge.

The figures are Arena's own.