Submit Your Agent for Evaluation
Thank you! Your submission has been received.
Thank you! Your submission has been received!
Oops! Something went wrong while submitting the form.

Assessing the Real State of Web Agents

Online Mind2Web puts frontier agents through 300 real tasks on 136 live websites.
See the leaderboard
Paper (arXiv)
Github
Frontier agents measured on the benchmark
Most web agents look great in a demo. On real websites, the numbers tell a different story.
When the Online Mind2Web team ran five frontier agents on live sites under human evaluation, most scored far below what they reported on older benchmarks. Operator reached 61.3%. Everyone else sat near 30%, and none beat SeeAct, a simple agent from January 2024, at 30.7%.

The gap traces back to how older benchmarks were built. On WebVoyager, a naive agent that only googles and clicks one link scored 51%, so many tasks never tested real navigation at all. Online Mind2Web was built to remove that shortcut.
Read full analysis

What’s In The Benchmark

300 live tasks across 136 real-world websites.
Live, Online Setting
Tasks run on real websites, not cached pages or sandbox replicas. Agents explore freely instead of following a fixed reference trajectory.
Curated for Validity
Built from Mind2Web and Mind2Web-Live, plus 75 new tasks. Invalid, drifted, ambiguous, and CAPTCHA-blocked tasks were removed.
Graded by Difficulty
Tasks are split by reference length: easy (5 steps or fewer), medium (6 to 10), and hard (11+). Performance drops as tasks get longer.
Submit your Agent for Evaluation

The Leaderboard

 Live agent rankings from the Online Mind2Web team. Human evaluation is the primary metric.
Agent
easy
medium
hard
avg Score
Navigator
n1.5-latest
Yutori
100.0%
96.5%
96.1%
97.3%
Navigator
n1-preview-11-2025
Yutori
90.1%
76.2%
71.1%
78.7%
Google Computer Use
Gemini 2.5 Computer Use
Google DeepMind
77.1%
71.3%
55.4%
69.0%
Operator
OpenAI Computer-Using Agent
OpenAI
83.1%
58.0%
43.2%
61.3%
ACT-1-20250814
o3-2025-04-16 and
Claude-sonnet-4-20250514
Enhans
81.9%
54.5%
35.1%
57.3%
Navigator
n1.5-latest
Yutori
easy
100.0%
medium
96.5%
hard
96.1%
avg Score
97.3%
Navigator
n1-preview-11-2025
Yutori
easy
90.1%
medium
76.2%
hard
71.1%
avg Score
78.7%
Google Computer Use
Google Computer Use
Google DeepMind
easy
77.1%
medium
71.3%
hard
55.4%
avg Score
69.0%
Operator
OpenAI Computer-Using Agent
OpenAI
easy
83.1%
medium
58.0%
hard
43.2%
avg Score
61.3%
ACT-1-20250814
o3-2025-04-16 and Claude-sonnet-4-20250514
Enhans
easy
81.9%
medium
54.5%
hard
35.1%
avg Score
57.3%
full live leaderboard

What A Leaderboard
Submission Needs

Each agent run is a trajectory: the actions, screenshots, and steps behind a result. A leaderboard submission goes through both evaluations below.
WebJudge
Step 1
Start with WebJudge, the benchmark's automatic evaluator. It's an LLM-as-a-judge that reaches about 85% agreement with human judgment, higher than prior methods like Autonomous Evaluation, AgentTrek, and WebVoyager. It runs in three steps: identify the key points a task requires, select the key screenshots from the trajectory, then judge the outcome. It's free, fast, and how every submission starts.
Human evaluation
Step 2
Then add human evaluation from Careerflow. Trained reviewers confirm task completion directly, the higher bar that WebJudge is built to approximate but still runs 6 to 10% off. A leaderboard submission needs both, and the human eval is what makes your result one people can trust.
To be considered for the leaderboard, you submit both: the WebJudge auto-eval and the human evaluation from Careerflow.

Careerflow Is The Official Human Evaluation Partner

 Live agent rankings from the Online Mind2Web team. Human evaluation is the primary metric.
Human eval is the reference standard, and it is also the part that does not scale on its own. Careerflow runs it for teams submitting to the leaderboard. You get a verified, publishable human-eval score on your agent's full trajectory set, reviewed by an unbiased third party with no stake in the outcome.
An overall evaluation report on your submission
The supporting evidence and metadata behind each judgment
A clear view of how your agent performed across the benchmark
What you get back

How The Review Works

Every trajectory is judged by more than one reviewer, then verified. The process is built to keep the result consistent and defensible.
Independent review
Each task is evaluated by multiple trained reviewers, so no single judgment decides the outcome.
QA verification
A quality pass resolves disagreements and re-checks anything uncertain.
Final review
Results are confirmed and
locked, with evidence attached to every call.
Reporting
You get your results. We return your overall evaluation report, then submit your result to the Online Mind2Web leaderboard for you.

How Blink Proved
 Its Benchmark Score

A real benchmark success story powered by trusted human evaluation
“Benchmarks require humans to confirm that the thing you're claiming your browser use agent can do, it can in fact do. We chose to use Careerflow. They were absolutely amazing at this.”
Ricardo Spagni
Co-founder - Blink
Blink
Blink, built by Nanite Foundry, set out to conquer Online Mind2Web and prove it was the strongest browser-use agent in its class. They chose Careerflow for the human evaluation behind the result.Careerflow built custom tooling for the project, ran every trajectory through a multi-stage review, and returned the metadata and suggestions the team needed to back their score in public. Blink reports a perfect score across all easy, medium, and hard tasks.
Read full case study

Trusted by Researchers & AI Engineers Worldwide

Human evaluation for reliable, production-ready AI agents.
Product Hunt Review
“We have labeled tens of datasets and shipped those metrics in production. Our customers use them to guardrail and observe their agents, and they trust that the metrics are high quality.”
Vatsal
Staff Data Scientist - Galileo
G2 Review
“Our automation gets better the more it learns from the cases it gets wrong, and Careerflow is the human validation layer in that loop. Their team gives us accurate, reviewed ground truth on exactly the invoices our models struggle with most: the handwritten ones, the low-light scans, the messy layouts. That feedback is what moved our accuracy on complex documents from 60% to 85%. They've become a real extension of our AP automation team.”
Krishna Janakiraman
CTO - Ottimate
G2 Review
“Benchmarks require humans to confirm that the thing you're claiming your browser use agent can do, it can in fact do. We chose to use Careerflow. They were absolutely amazing at this.”
Ricardo Spagni
Co-founder - Blink
Submit your Agent for Evaluation

Simple, Transparent Pricing

Full Set
Human evaluation of all 300 trajectories
$3,500
Subsequent Submissions
Per trajectory, after the full set
$10
.00
/ trajectory
Book a call
Your fee covers the human evaluation, and a portion is donated to the OSU NLP Group at The Ohio State University to support the research behind Online Mind2Web.

Have More Questions?

Here are some of the frequently asked questions from our customers
What is Online Mind2Web?

A benchmark for browser-use agents from the OSU NLP Group and UC Berkeley, with 300 real tasks across 136 live websites. Introduced in the COLM 2025 paper "An Illusion of Progress? Assessing the Current State of Web Agents."

Why human eval if WebJudge exists?

WebJudge reaches ~85% agreement with humans and is great for iteration, but it still runs 6 to 10% off the true success rate. Human eval is the reference standard for a published result.

Who runs the human evaluation?

Careerflow, the benchmark's official human evaluation partner.

What do I receive?

A per-task pass or fail, the evidence behind each judgment, and error analysis mapped to the paper's failure categories.

How is my data handled?

Confidential, with anonymized workflows and enterprise-grade security.

Do you handle evaluation beyond Mind2Web?

Yes. Careerflow's Human Data team handles human evaluation and labeling across agents, LLMs, and a wide range of domains.

Get your submission
human-evaluated

Book a call and we'll review your agent's full trajectory set and return a leaderboard-ready result. After you book: a short scoping call to confirm the trajectory set and timing, then evaluation begins. No commitment on the first call.
Book a call
Your Career Copilot. AI-assisted tools and resources to get hired at FAANG, top tech, and startup companies 10X faster.

Products

LinkedIn OptimizerWho's HiringResume ReviewJob TrackerResume BuilderCover Letter WriterAI Mock InterviewResume ATS CheckerBrowser ExtensionLinkedIn to ResumeCoach PortalAll Features

Free Tools

Free LinkedIn
ReviewLinkedIn Profile ChecklistHeadline GeneratorSummary GeneratorAI Post WriterJob Skill MatchJob Description Summarizer

Resources

Help Center
BlogResume TemplatesCommunityInterview GuidesJob Search TipsChatGPT GuidesCareer ResourcesHuman Data ResearchHow It WorksFAQGift Cards

For

For StudentsFor ExecutivesFor Career ChangersFor OrganizationsFor Career CoachesFor BootcampsFor UniversitiesFor OutplacementFor Workforce Development

Company

AboutPressContact UsPrivacy PolicyYour Privacy RightsTerms of ServiceBecome an AffiliateDo Not Sell or Share My Personal Information

Developers

API DocsTrust CenterIntegrations
Do Not Sell or Share My Personal Information
© 2026 Careerflow Inc • All Rights Reserved