September 2026Research

Benchmarking Discovery

dig.bench: uncover hidden game rules by experimenting

A grid of black and white tiles, each showing a fragment of a DIG puzzle in bold monospace type, such as rewrite rules, strings of letters and underscores, and arrows, with the @ sign and arrows picked out in green.

dig.bench is a benchmark of scientific discovery.

Each of its 70 games has hidden rules, and the only way to learn them is to experiment: press a button, see what changes, make a guess and test it. Everything is in text, so there are no visual confounds. People and AI models play the same games, see the same screen and get the same number of moves.

The games come in 7 tiers of difficulty. None of them are easy, but people can crack even the hardest.

You can play the 21 public games, and run any model against them through the API, at digbench.ai.

_→oo2_______1___

Move 40 of 200

Level 1 of P-16. An example playthrough. Play it yourself here.

Leaderboard

The strongest model we tested (up to August 2026), Opus 5, beat 50 of the 70 games. People beat all 70. Giving Opus 5 extra tools for coding and memory didn't make it any better on the two hardest tiers. Switch to Tier breakdown to see each tier on its own.

Summary

Evaluating the frontier of discovery is critically important. At Prefrontal we continually test the discovery capabilities of frontier models and AI scientists.

Citation

@misc{battleday2026dig,
  title  = {DiG-bench: Discovery in Games},
  author = {Ruairidh M. Battleday and Kai Sandbrink and
            Jimi Cullen-Drohan and Zihan Yan and Timothy Muller and
            Clare Maguire and Ales Kubicek and Fraser Greenlee-Scott
            and Sukrit Sumant and Tri Dao and Jürgen Schmidhuber
            and Michal Valko and Joshua Tenenbaum and
            Thomas L. Griffiths and Zeb Kurth-Nelson and
            James C.R. Whittington},
  year   = {2026},
  eprint = {2608.12593},
  archivePrefix = {arXiv},
  primaryClass  = {cs.AI},
  url    = {https://arxiv.org/abs/2608.12593}
}