Here's a question that should bother anyone deciding which AI tool to pay for: who checks whether these models are actually as good as their makers claim? The honest answer, right now, is almost nobody independent. The whole industry runs on self-reported grades. And a San Francisco startup called Vals, which just raised $40 million from Andreessen Horowitz and got a deep-dive profile in TechCrunch this week, has made it its business to change that.
01.Why AI benchmarks stopped working
If you've followed AI even casually, you've seen the pattern: a new model launches, posts a record score on some benchmark with a name like MMLU or HumanEval, and the internet goes wild. The problem is that these public tests have quietly broken down. The questions leak into the training data — so the model has effectively seen the exam paper before. Test sets get saturated, meaning every top model scores so high that the test no longer separates them. And in many cases, the company building the model also runs the benchmark.
Vals co-founder Rayan Krishnan, who is 25 and studied computer science at Stanford, put it simply: the academic benchmarks were "not keeping up" with how fast new models were arriving. Trillions of dollars are pouring into building AI, and comparatively little into measuring it.
02.What Vals does differently: tests that stay secret
Vals' approach has one non-negotiable rule: the test materials are kept private. Think of it like the CAT or the SAT — questions stay sealed, so you can't study the answers in advance. A model can't be trained on a test it has never seen.
The second difference matters even more. Vals doesn't test models on quiz-style trivia. It tests them on actual professional work — legal research, financial document analysis, engineering tasks, clinical questions — built with domain experts from reference institutions in each field. Instead of asking whether a legal AI can pass the bar exam, Vals checks whether it can perform real legal research and grades the work product like a senior partner would.
It's a quietly powerful idea. If you're a bank in Mumbai deciding between two enterprise AI subscriptions, a sealed, independent score on real analyst tasks is worth far more than a press-release leaderboard.
03.The numbers behind the $40M round
Founded in 2024 by Krishnan and Langston Nashold (who previously worked at Meta, NVIDIA and Hudson River Trading), Vals raised a seed round led by 8VC and Bloomberg Beta before closing a $40 million Series A at a $400 million valuation, led by Andreessen Horowitz. The company says revenue has already grown 8x compared with all of 2025, its customer base has doubled, and the team has tripled in six months.
Here's the part that tells you it's being taken seriously: Vals' results have been cited in the official model cards of OpenAI, Anthropic, Google, Meta and xAI — the labs effectively acknowledging it as a scorekeeper. It has also supported the US Department of Commerce and members of Congress on AI policy work.
04.The reality check: top models fail half of real finance work
Now for the number that made me sit up. On Vals' Finance Agent v2 benchmark — tasks like building relative-value models across peer companies and generating investment recommendations grounded in actual documents — the best AI model scored 52% in May 2026. Read that again: the most capable AI on the planet could not correctly complete roughly half the work a professional financial analyst is expected to handle.
That's the gap between what leaderboards advertise and what models deliver on real work. It's also exactly why independent evaluation is becoming an industry. If a tool scores brilliantly on MMLU but can't finish an analyst's Tuesday, the MMLU score is just marketing.
05.What Vals is building next
Alongside the funding, the company shipped a batch of new products. Vals Smith lets any team build a custom coding benchmark from their own GitHub repositories. Its Frontier Risk Benchmarks line includes an RSI Index built with CoreWeave and a new cybersecurity benchmark, ReverseEngBench, developed with researchers at Columbia, Tufts, UC Berkeley and UCLA. It has also started work on mental health evaluation.
Of course, one fair caveat: a16z now has a financial stake in the referee, and Vals' own methodology isn't fully public — that's rather the point of sealed tests. But the direction is right. Every serious market — credit, pharma, food safety — eventually built independent measurement. AI was never going to be the exception.
One more thing worth noticing: we've entered a season where every single week brings a model launch with big self-reported numbers — we wrote about Jev's wild speed claims just the other day, and we covered how enterprises are actually putting GPT-6 Astra to work. Independent scoring is arriving at exactly the right moment.
Key Takeaways
- Public AI benchmarks are breaking down — leaked into training data, saturated, and often self-graded
- Vals keeps its test sets private and grades AI on real professional work: law, finance, engineering, medicine
- $40M Series A led by a16z at a $400M valuation; revenue up 8x; results cited by OpenAI, Anthropic, Google, Meta and xAI
- Best model on the finance benchmark still scored just 52% — real capability gaps remain behind the hype
- New launches: Vals Smith for custom coding benchmarks, cyber and mental-health risk benchmarks
Frequently Asked Questions
Why are public AI benchmarks considered broken?
Public benchmarks like MMLU have three problems: their questions leak into training data, so models have effectively seen the exam paper before; the tests are saturated, with every top model scoring so high that they no longer separate anyone; and in many cases the company building the model also runs the benchmark. The industry largely runs on self-reported grades.
What makes Vals' approach different?
Vals keeps its test materials private — like the CAT or SAT, questions stay sealed, so a model cannot be trained on a test it has never seen. Instead of quiz-style trivia, it tests models on real professional work such as legal research, financial document analysis, engineering tasks and clinical questions, built with domain experts and graded like a senior professional would grade the work product.
Who backs Vals, and how good are top models on its tests?
Vals was founded in 2024 by Rayan Krishnan and Langston Nashold, raised a $40 million Series A at a $400 million valuation led by Andreessen Horowitz (after a seed round led by 8VC and Bloomberg Beta), and says revenue has grown 8x versus all of 2025. Its results are cited in the official model cards of OpenAI, Anthropic, Google, Meta and xAI. On its Finance Agent v2 benchmark, the best model still scored just 52% on a professional analyst's tasks.
What This Means for You
If you're a student or developer in India, this is a career signal: AI evaluation is becoming a real, fundable field — auditing models is a skill labs and enterprises will pay for. If you're a professional or running a business, the lesson is more immediate: don't pick your AI tools based on launch-day scorecards. Run your own small, private tests on your actual work — a model that tops a leaderboard can still fumble your use case. And while independent scorekeepers matter, the skill that pays you personally is knowing how to direct these models well: clear prompting, structured review, and cost control so experiments don't drain your budget. That's exactly what our GPT-6 Astra — Zero to Hero Guide teaches, and honestly, that's where I'd start before trusting any benchmark at face value.
By the way, if terms like agents and computer use still feel fuzzy, our GPT-6 Astra guide for beginners explains them all without the jargon.