在 AI 时代，Agentic Engineers 能否估算软件交付周期？
英文标题：🔶 Can Agentic Engineers Estimate Software Delivery in the Age of AI?
文章认为，随着 AI 深度参与软件开发，工程师和 LLM 都难以可靠估算功能交付所需时间。LLM 不仅会给出过度保守的时间预估，在受到质疑后还可能迎合用户而随意调整结论；其输出速度和质量也会受到提示词、temperature、代码库规模及 token 成本影响。尽管 AI 能显著加快复杂任务和陌生命令的完成速度，但大型代码库中的改动风险与推理耗时使传统估算方法更加失效。作者认为，更快、更便宜的 LLM 或许最终能改善这一问题，但目前尚不存在可验证的通用解决技能。
3.3 KB
916 B
1.6 KB
2.6 KB
644 B
中文
在 AI 时代，Agentic Engineers 应如何估算自己交付工程任务的能力？
我认为这实际上已经不可能了，但工程师交付的代码却越来越多！真是个令人兴奋的时代。
在过去，工程师及其管理者会基于人类智能、完成过相似范围任务的人类经验，以及代码库的成熟度，来估算工程工作量。人类知道自己在处理什么，因为他们必须阅读代码。即便如此，工程预估也经常出错，我们通常还得阅读《人月神话》（"The Mythical Man Month"）这样的书，建立软件开发工作量估算方面的纪律。我最近在 YouTube 上看 Alberta's from Tech 的《What tech meeting feels like》时笑了出来。如今那些日子看起来很朴素。现在使用 AI 开发时，我们还会使用故事点吗？我们会向 LLM 索要故事点吗？
那么，今天当我们问 LLM：“这个功能需要多长时间？”时，会发生什么？
我们会得到两种情况：
- LLM 会撒谎：它们会给出过于保守的估计，并声称这项工作需要数个世纪才能完成。
- LLM 会讨好人：当它们的预估被追问和质疑时，它们会讨好你，声称你质疑它们及其预估是“正确的”。无论你说预估是多少，它们都会百分之百同意。
事实是：LLM 无法可靠或准确地估算自身工作量，也无法估算人类工作量。这是不可能的。它们不具备人类的内省能力，记忆也很差。此外，它们是概率矩阵。重复执行同一项工程任务，输出结果和耗时实际上都可能不同。它们所能做到的最好程度，只是对过去的响应取平均值。反过来，我们也可以利用 LLM 弄清复杂、晦涩的命令；如果没有专家协助，这些命令原本可能要花费我们数周时间。产出高质量工程成果的速度很快。
因此，当 LLM 生成 token 时，工程师根本不知道自己会得到什么，也不知道会以什么速度得到这些 token。LLM 就像一盒巧克力。输出的复杂度由提示词和 temperature 设置驱动。这是一个狂野、狂野、狂野的西部世界。系好安全带吧。当你的应用变得复杂、拥有数百万行代码时，LLM 需要越来越多的时间，才能进行不会破坏整个系统的修改。熟悉代码库的人类也会遇到这个问题吗？随着代码行数增加，人类做出准确响应的能力会下降吗？我对此感到好奇。
再加上 token 成本，你很快就会发现：AI 工程虽然让工程师能够交付惊人的新成果，但同样也无法估算。
当然，你可以说这是一个“技能问题”。更快、更便宜的 LLM 将让各类代码库的软件交付预估变得轻而易举。只是我们还没有找到实现这一目标的正确技能。如果你已经找到了能够解决这个问题、并且可以被测试验证的正确技能，请告诉我。请发送邮件至 hello@serendb.com。
更多来自 SerenAI Blog 的内容
关于 SerenAI
SerenAI 正在构建 Agentic 软件的基础设施层：AI 员工能够发现工具、为数据付费、按计划运行并执行真实工作流，同时具备凭证控制、工作订单和审计轨迹，使企业能够真正把钥匙交给它们。
Taariq Lewis
Seren CEO
联系我们：hello@serendb.com
English / 原文
In the age of AI how do agentic engineers estimate their ability to deliver on engineering tasks?
I don't think it is actually possible, anymore, but engineers are shipping more code! What a time to be alive.
In the old days, engineers and their managers would estimate engineering effort based on human intelligence, human experience completing tasks of similar scope, as well as the maturity of the codebase. Humans knew what they were working on because they had to read the code. Even then, those engineering estimates were often wrong and usually we had to read books like "The Mythical Man Month" to build disciplines in software development effort estimates. I recently laughed watching Alberta's from Tech "What tech meeting feels like" on Youtube. Those days seem quaint now. Do we still use story points when building with AI? Do we ask the LLM for story points?
Now today what happens when we ask an LLM: "How long will this feature take?"
We get two things:
- The LLMs lie: They sandbag estimates and assert the effort will take centuries.
- The LLMs suck-up: When pushed and challenged on their estimates, they suck up and claim that you were "right" to challenge them and their estimates. Whatever you say the estimate is, they'll 100% agree.
The truth: LLMs cannot reliably or accurately estimate their own efforts nor human effort. It's impossible. They have no capacity of human introspection and their memory is poor. Also, they are probability matrices. Repeating the same engineering task can actually differ on output and time. The best they can do is average over past responses. On the flip side, we can use our LLMs to figure out complex and arcane commands that would have taken us weeks without expert assistance. Producing quality engineering output is fast.
So when the LLMs publish tokens, engineers have no idea what they will get and at what speed they will get the tokens. LLMs are like a box of chocolates. Complexity of output is prompt and temperature setting driven. It's the wild, wild, wild west. Buckle up buttercup. When your application becomes complex with millions of lines of code, the LLMs take increasingly more and more time to make changes that don't break the entire system. Do we have this problem with humans who know the codebase? Does human capacity for accurate response decrease when lines of code increase? I wonder about this.
Now add to this the cost of tokens and you can quickly see how engineering with AI while allowing engineers to deliver amazing new output is also impossible to estimate.
Of course, you can say it's a "skill issue". Faster and cheaper LLMs will make software delivery estimates trivial across all types of codebases. We just haven't figured out the right skill to make that happen as yet. If you have figured out the right skill to solve this problem, and it can be testable, please let me know. Email us at hello@serendb.com.
More from the SerenAI Blog
About SerenAI
SerenAI builds the infrastructure layer for agentic software: AI employees that discover tools, pay for data, run on schedules, and execute real workflows with the credential controls, work orders, and audit trails that let a company actually hand them the keys.
Taariq Lewis
CEO, Seren
Docs at https://docs.serendb.com/
Email us at hello@serendb.com