Benchmarks are an unreliable measure of difficulty

Very interesting paper on the way that benchmark question difficulty and framing affect the observed improvement in scores for model training. “Our results show that many benchmarks implicitly overweight certain difficulty regions, so their leaderboard conclusions should be interpreted with ca

中文全文翻译正在补齐，可先查看下方英文原文。

Very interesting paper on the way that benchmark question difficulty and framing affect the observed improvement in scores for model training. “Our results show that many benchmarks implicitly overweight certain difficulty regions, so their leaderboard conclusions should be interpreted with ca