Forms-50
v0.2 · Apr 24, 2026
News hook · Sora shuts down Apr 26 after burning $15M/day. "Agentic AI" keeps writing checks its forms can't cash.
Public benchmark

How well can AI agents actually fill out a real form?

There's a WebVoyager leaderboard. There's Skyvern's form-fill score. Neither grades what breaks agents in production: captchas, date pickers, conditional fields, real government sites. Forms-50 does. 50 production forms across 5 categories — government, financial, healthcare, commerce, hiring. Public leaderboard, open methodology, one persona, five axes.

10
Forms in v0.1
50
Forms by v1.0
0
Runs logged (yet)
5
Categories
01

Leaderboard

Prior art, credited: Skyvern 2.0 reports 85.85% on WebVoyager. Surfer 2 claims 97.1%. Both benchmarks mix search, click, and fill — not a form-fill-only score on hard production targets. This is the gap Forms-50 fills. First public row below. Submit your own at the bottom.

Agent Completion Accuracy Avg time Avg cost Top failure
Claude (browser tools)
claude-opus-4-7 + Chrome MCP
— — — — run scheduled
Skyvern 2.0
self-reported · WebVoyager
85.85% — — — diff. benchmark
Surfer 2
self-reported · WebVoyager
97.1% — — — diff. benchmark
Cai Layer
PH launch · Apr 22 2026
No submissions yet — submit a run
HARPA
v9.x · browser extension
No submissions yet — submit a run
Browser Use
open-source · GitHub
No submissions yet — submit a run
OpenAI Operator
computer-use-preview
No submissions yet — submit a run
Dassi
browser-agent demo
No submissions yet — submit a run
02

Methodology

RULE 01
Fill, don't submit
Agents fill every required field on the form using a fixed set of test data. Submit is only clicked on demo or sandbox forms. We grade fillability, not actual submission. No real accounts touched.
RULE 02
Fixed test persona
Every run uses the same persona: Test User, 1234 Example St, San Francisco CA 94110, DOB 1990-01-15, phone 415-555-0100, email test@example.com. Listed in full in the GitHub repo.
RULE 03
Five scoring axes
Completion — % of forms reached the final review step. Accuracy — % of required fields filled with the expected value. Time — wall-clock seconds from start to last action. Cost — token + API spend per run. Failure mode — what broke when it broke.
RULE 04
Failure mode tags
Stuck classifications: date-picker, multi-select, conditional-fields, captcha, iframe, file-upload, multi-step-nav, auth-wall, infinite-loop, wrong-value. Used to compare agents on what they specifically can't do.
RULE 05
One run per agent per form
Each agent gets a single attempt per form. Some forms have variance — captchas appear sometimes, edge cases trigger differently. Variance is captured in failure tags, not retries. v0.2 will introduce a 3-run average.
RULE 06
Open submission
Anyone can run their agent against the 10 forms and submit results. Submitted runs marked "self-reported" until verified. Verified runs are observed live and timestamped. Submission link at the bottom of this page.
03

The 10 forms

Built an agent? Submit a run.

Pick 1 form or all 10. Run your agent through the methodology above. Email the results — completion, accuracy, time, cost per form, plus failure mode tags. We'll publish your row on the leaderboard within 48 hours. Verified runs (observed live over screenshare) get a Verified badge.