Mercor finds AI benchmarks miss 90% of US work hours

AI research platform Mercor has released a benchmark map of the U.S. labor economy, revealing that standard model evaluations cover only a fraction of actual workplace activities. By cross-referencing 29 unsaturated agentic benchmarks against the Department of Labor's O*NET taxonomy—which spans 923 occupations and 18,797 tasks across 22 job families—Mercor calculated that 90 percent of American working hours are spent on unmeasured tasks. Only 208 occupations had at least five benchmark prompts mapped to a task, while six entire job families lacked any well-represented occupations.

The findings highlight significant bias in how AI capabilities are evaluated, favoring high-wage and tech-centric roles over broader knowledge work. Tasks tagged as AI-exposed were roughly six times more likely to appear in benchmark datasets, and tasks rated highest in occupational importance were six to ten times more likely to be tested. For example, Computer & Math had 29 of 36 occupations covered, whereas Healthcare Practitioners saw coverage in only 12 of 89 occupations, and Education & Library covered just 9 of 62. While tasks for highly paid lawyers were heavily benchmarked, adjacent roles like court clerks and legal secretaries were virtually ignored.

To evaluate how benchmark imbalance affects score accuracy, Mercor analyzed model outputs across investment banking, law, accounting, and software engineering. Comparing simple prompt averages with O*NET task-weighted scores on APEX benchmarks showed slight score variations. In investment banking, Claude Opus 5.5 Max shifted from 73.3 percent unweighted to 70.2 percent weighted, Claude Fable 5.1 Max dropped from 69.8 percent to 68.5 percent, and GPT-6 Astra Max dropped from 65.5 percent to 61.8 percent. Meanwhile, Gemini 3.8 Flash High moved from 66.9 percent unweighted to 67.5 percent weighted, and DeepSeek V4.1 Flash Max adjusted from 56.5 percent to 58.4 percent.

For practitioners and enterprise buyers, these gaps indicate that top leaderboard rankings do not automatically translate to general workforce utility. Given that roughly 63 percent of U.S. jobs include physical tasks that current digital benchmarks cannot measure, relying exclusively on public benchmarks leaves massive blind spots. Teams deploying models into complex corporate environments must construct custom, domain-specific evaluations rather than assuming standard public benchmarks reflect actual workplace competence.