AI labs report benchmark scores 6.4% higher than independent testers, on average

I've been building whatmodel.sh, an open source AI leaderboard, and wanted to see how a labs' own benchmark scores compare with independent testers. Using 1,629 published benchmark results, I matched 611 comparisons of 180 models on 103 benchmarks. AI labs reported scores 6.4% higher on averag

中文全文翻译正在补齐，可先查看下方英文原文。

I've been building whatmodel.sh, an open source AI leaderboard, and wanted to see how a labs' own benchmark scores compare with independent testers. Using 1,629 published benchmark results, I matched 611 comparisons of 180 models on 103 benchmarks. AI labs reported scores 6.4% higher on averag