Google用新LLM更新Android Bench，但Gemini仍落后

英文标题：Google updates Android Bench with new LLMs, but Gemini still lags behind

Google更新了Android Bench基准测试，引入新的大语言模型后，Gemini在排行榜上跌至第五。Fable 5与GPT 5.5得分虽高但token成本超过130美元，而Gemini 3.1 Pro仅需87美元；本应低成本的Gemini 3.5 Flash因耗时长达28小时，每次运行成本高达165美元。在Google转向代理式开发的背景下，Android编码性能的落后成为痛点，可能促使公司通过收购应用源码等措施强化AI训练。新的Harbor框架旨在降低社区参与门槛，鼓励开发者贡献基准任务，共同推动Android Bench演进。

打开原文相关 3 · 机会 0

中文

Google更新后的排行榜显示Gemini跌至第五名。

图片来源：Google

Google更新后的排行榜显示Gemini跌至第五名。图片来源：Google

然而，Fable 5与GPT 5.5的运行成本也极其高昂，在100道问题、10次运行的基准测试中，它们消耗了超过130美元的token。Gemini 3.1 Pro的得分虽未如此亮眼，但运行该测试仅需87美元。Gemini 3.5 Flash本应以比其他模型更低的成本运行，却因为完成基准测试耗时过长而在排行榜上成本最高：每次运行165美元，耗时28小时。

对Google而言，Android编码性能的差距是一个问题，因为该公司正将许多项目转向代理式开发。显然，Google更希望Android开发者在工作流中使用自家工具，这可能正是Google据报道一直在向开发者购买应用源代码用于AI训练的原因。

社区协作

Android Bench预计将随时间演进，采用新的工作流程来测试模型。Google希望开发者愿意通过分享基准和开发任务来为Android Bench做出贡献。为使这一设想更可行，Google正在切换至Harbor框架。据该公司称，该测试沙箱让开发者能够轻松运行、评估并分享Android Bench的结果。

Google使用Harbor重新运行了所有过往测试，以获得LLM性能的新基线。因此，即使底层测试尚未改变（目前如此），先前报告的分数也出现了一些变动。历史数据将保留在线上存档中。

借助这一全新的、更简便的框架，开发者可以针对Android Bench运行自己的开发任务，并提交以供纳入官方测试。Android Bench GitHub 已更新了新的数据集以及参与说明。

English / 原文

Google’s updated leaderboard shows Gemini slipping to fifth place.

Credit: Google

Google’s updated leaderboard shows Gemini slipping to fifth place. Credit: Google

However, Fable 5 and GPT 5.5 also have extremely high operating costs, chewing through more than $130 in tokens for the 100-problem, 10-run benchmark. Gemini 3.1 Pro didn’t score as high, but it only costs $87 to run the test. Gemini 3.5 Flash, which is supposed to be cheaper to run than other models, has the highest cost on the leaderboard because it took so much longer to complete the benchmark: $165 per run and a 28-hour runtime.

The Android coding performance gap for Google’s models is a problem as the company shifts many of its projects toward agentic development. Obviously, Google would prefer that Android developers use Google’s tools in their workflows, which may be why Google has reportedly been offering to buy application source code from developers for AI training.

Community collaboration

Android Bench is supposed to evolve over time, adopting new workflows to test models. Google hopes that developers will want to contribute to Android Bench by sharing benchmarks and development tasks. To make that more feasible, Google is switching to the Harbor framework. According to the company, this testing sandbox makes it easy for developers to run, evaluate, and share results for Android Bench.

Google re-ran all its previous tests with Harbor to get a new baseline for LLM performance. So there has been some shift in the previously reported scores even though the underlying tests haven’t changed (yet). The historical data will remain online in an archive.

With the new, easier framework, developers can run their own development tasks against Android Bench and submit those for possible inclusion in the official test. The Android Bench GitHub has been updated with the new dataset and instructions on how to get involved.