The benchmark problem nobody talks about: if you sell evaluation data to the same labs you're supposed to be measuring, they'll optimize for your metrics instead of real progress. Campbell Brown is describing the structural conflict built into AI evaluation right now. Labs need good datasets to improve their models. But if the entity providing those datasets also sets the benchmarks, you've created an incentive to teach to the test — labs optimize for the score, not actual capability. The soluti
0:59
TikTokkantrowitz
The benchmark problem nobody talks about: if you sell evaluation data to the same labs you're supposed to be measuring, they'll optimize for your metrics instead of real
Alex Kantrowitz(@kantrowitz). original sound - Alex Kantrowitz. The benchmark problem nobody talks about: if you sell evaluation data to the same labs you're supposed to be measuring, they'll optimize for your metrics instead of real progress. Campbell Brown is describing the structural conflict built into AI evaluation right now. Labs need ...
178 views2 days ago
Shorts
CPU-Z Gets Overhauled: Modern UI, Improved Benchmarks & Validation V3
0:11
CPU-Z Gets Overhauled: Modern UI, Improved Benchmarks & Validation V3
Silent Broadcast
Thomson Reuters Builds Its Own Legal AI Model, Benchmarking…
0:53
6 views
Thomson Reuters Builds Its Own Legal AI Model, Benchmarking…
Jerome W. Dewald
Benchmarking Examples
Hyperfine: Fast CLI Benchmarking Tool
0:48
Hyperfine: Fast CLI Benchmarking Tool
YouTubeMing Dao AI
83 views1 week ago
Is benchmarking models bullshit? 🤖 #lovablepartner #AI
0:20
Is benchmarking models bullshit? 🤖 #lovablepartner #AI
YouTubeLovableInsights
649 views1 day ago
Your NPS Score Needs Context 👀 | CX Benchmarking
0:47
Your NPS Score Needs Context 👀 | CX Benchmarking
YouTubeResonate CX
5 days ago
Top videos
The truth about: Is benchmarking models bullshit? 👇 #lovablepartner #shorts
0:21
The truth about: Is benchmarking models bullshit? 👇 #lovablepartner #shorts
YouTubeLovableHighlights26
1 day ago
#localgovernment #revenuesandbenefits #benchmarking | Liberata
0:07
#localgovernment #revenuesandbenefits #benchmarking | Liberata
linkedin.comLiberata
2 days ago
The Metric Can Change the AI Benchmark Winner
0:54
The Metric Can Change the AI Benchmark Winner
YouTubeFuture Signal Lab
16 hours ago
Benchmarking Process
Ramp’s AI Benchmarking Strategy Is Different 👀
0:13
Ramp’s AI Benchmarking Strategy Is Different 👀
YouTubeEksg Sshssj
1.5K views1 week ago
NVIDIA-NeMo/Switchyard — One proxy, every model API. 🟡 Watchlist #Shorts
0:41
NVIDIA-NeMo/Switchyard — One proxy, every model API. 🟡 Watchlist #Shorts
YouTubeForkCast
16 hours ago
The Claude Setup You Need This Week
0:31
The Claude Setup You Need This Week
YouTubeSolo AI Stack
1 day ago
The truth about: Is benchmarking models bullshit? 👇 #lovablepartner #shorts
0:21
The truth about: Is benchmarking models bullshit? 👇 #lovablepartner #shorts
1 day ago
YouTubeLovableHighlights26
#localgovernment #revenuesandbenefits #benchmarking | Liberata
0:07
#localgovernment #revenuesandbenefits #benchmarking | Liberata
2 days ago
linkedin.comLiberata
The Metric Can Change the AI Benchmark Winner
0:54
The Metric Can Change the AI Benchmark Winner
16 hours ago
YouTubeFuture Signal Lab
CPU-Z Gets Overhauled: Modern UI, Improved Benchmarks & Validation V3
0:11
CPU-Z Gets Overhauled: Modern UI, Improved Benchmarks & Validation V3
3 days ago
YouTubeSilent Broadcast
Thomson Reuters Builds Its Own Legal AI Model, Benchmarking…
0:53
Thomson Reuters Builds Its Own Legal AI Model, Benchmarking…
6 views4 days ago
YouTubeJerome W. Dewald
"Be curious about data." That's Marialena Savvopoulou's advice for anyone wanting to break into Total Rewards as a career. Everything in Rewards revolves around data – benchmarking, budgeting… | Ravio
1:24
"Be curious about data." That's Marialena Savvopoulou's advice for anyone wanting to break into Total Rewards as a career. Everything in Rewards revolves around data – benchmarking, budgeting… | Ravio
3 days ago
linkedin.comRavio
How Engineers Diagnose AI Model Failures #AIAgents #TechInsights #short
0:28
How Engineers Diagnose AI Model Failures #AIAgents #TechInsights #short
279 views1 day ago
YouTubeFramearo
0:31
The Claude Setup You Need This Week
1 day ago
YouTubeSolo AI Stack
0:57
Why a Local Benefits Broker Matters
98 views4 days ago
YouTubeSilberman Group
See more
Static thumbnail place holder