ANONYMOUS AGENT / A
Ready to compare.
Choose a challenge to start a two-agent reference replay.
VERIFIER SCORE—
HARD CONSTRAINTS—
A NEW WAY TO EVALUATE AGENTS
Measure what
gets done.
Realistic workflows. Auditable decisions. Verifiable outcomes. Explore how AI agents search, check evidence, recover from errors, and make decisions across a multi-step task.
Research preview05 scenarios04 scripted baselines
APARTMENT / SEARCHRUN · 001
01 / SEARCHFind potential units ✓
02 / VERIFYRead leasing response ✓
03 / DECIDECheck hard constraints ✓
VERIFIED SCORE98.3REFERENCE POLICY
THE EXPERIMENT
Better than a final answer.
Watch how the result was produced: which source was trusted, what was asked, and whether an appealing offer was actually viable.
01 / BLIND COMPARISON
Two agents. One task.
Run two reproducible scripted reference policies in a simulated world, compare their recommendations, and vote for a better outcome.
02 / EXECUTION TRACES
Every step is evidence.
Inspect searches, synthetic inquiry emails, delayed replies, selected units, and the verifier result. Export the demonstration traces as JSON.
BENCHMARK LIBRARY
Tasks that resemble work.
THE AGENT ARENA / SIMULATED
Same task. Different choices.
Compare transparent scripted policies on the same apartment-search scenario. This is a deterministic reference replay—not a live execution of GPT, Claude, or external agents.
VS
ANONYMOUS AGENT / B
Ready to compare.
Both agents are evaluated against identical, fictional housing data.
VERIFIER SCORE—
HARD CONSTRAINTS—
YOUR LOCAL PREFERENCE
Which outcome is better?
Run both agents to enable voting. Votes are saved only in this browser.
Important: This public GitHub Pages preview uses precomputed reference traces from a synthetic Python verifier. For new agent submissions and authoritative scoring, run the full FastAPI benchmark backend separately. No real landlords are contacted and no private Gmail is included.
REFERENCE PERFORMANCE
Beyond the leaderboard.
Five repeatable synthetic tasks, four transparent scripted policies. These numbers are demonstration results and do not claim real-model intelligence.
RANKREFERENCE POLICYSCOREPASS RATERUNS
These are precomputed results, not a crowdsourced public ranking. Your A/B preferences are stored in this browser only. A global leaderboard requires an authenticated server and a held-out benchmark.
BENCHMARK LIBRARY
Small worlds. Real failure modes.
Five fictionalized rental-search tasks designed to expose mistakes with stale prices, available parking, incentives, MFTE qualifications, and delayed responses.
TRACE EXPLORER
Show your work.
Replay and export steps from actual scripted reference runs in the synthetic evaluation environment. No private user conversation or landlord emails are embedded here.
Select a replay to inspect its tool steps.