See the puzzle.
Each task provides one image, a title when available, and permitted auxiliary text. Reference answers stay separate.
A visual puzzle benchmark for multimodal agents.
From what an
image shows to what it really means.
Real puzzles. Hidden rules. One answer to discover.
A collection of visual riddles where symbols, language and everyday knowledge meet. Each puzzle asks an agent to discover the rule before finding the answer.
Start with the image. Look for a connection.
Reveal a hint
when you need a different perspective.
The image contains the clues. Can you work out the hidden rule?
Current annotation; review is ongoing.
Try another title, puzzle ID or collection.
17 selected examples from the working collection. Original puzzle text is preserved; English titles are editorial labels.
39 puzzles. Four model–agent configurations.
A preliminary view
of visual puzzle solving.
| Model / agent | Reasoning | Solved | Time / task |
|---|
Pilot results, not scores on all 932 puzzles. Historical runs use different native harnesses, time budgets and prompt revisions, so this is an exploratory comparison. Timeouts count as unsolved.
No-hint accuracy is the number of correct final answers divided by 39. The GPT and Doubao runs used difficulty-based limits of 10, 20 or 30 minutes; DeepSeek used a uniform 20-minute limit.
Hint retests were run only on baseline failures: GPT-6 Sol solved 12 of 21 retested tasks, and DeepSeek Flash solved 17 of 34. Combined coverage is (baseline correct + retest correct) / 39. It is not accuracy from a full 39-task hinted run.
GPT-6.2 Sol and Doubao do not have hint-retest results in this snapshot. Doubao recorded 17 timeouts and DeepSeek recorded 4 in the no-hint pilot.
A controlled task package, a fresh agent session, and a final answer you can check.
Each task provides one image, a title when available, and permitted auxiliary text. Reference answers stay separate.
Agents keep their native tools. Background research is permitted; searching for the puzzle’s existing answer is prohibited.
Final answers are matched against accepted references. Failed baseline tasks may be retested with mechanism hints in fresh sessions.
Search restrictions are prompt-based and require trace review. They do not by themselves prove that an agent avoided existing solutions.
Explore the preview data and inspect the pilot results.
Research preview · Full dataset release details are not yet available.
The example data could not be loaded. Please reload the page.