摘要：具备函数调用（Function Calling）能力的大语言模型（LLM）正成为现代 Agent 式 AI 系统的关键核心。然而，现有的部署通常将推理请求路由至性能强大的云端模型，从而产生大量的能源消耗与碳排放。为应对这一可持续性挑战，我们提出了一种碳感知路由框架，该框架跨越由异构硬件构成的三层边-云（Edge-Cloud）架构，协同调度边缘与云端 LLM 来分发函数调用查询。其核心是一个运行在统一语义-词汇嵌入空间中的轻量级 k-NN 预测器，用于评估各个边缘层级处理特定查询时的准确率、延迟和功耗。随后，系统将这些预测指标与实时电网碳强度相结合，把每个查询路由至能够成功执行该任务且碳排放最低的层级。在业界前沿的函数调用基准测试和 LLM 家族上的评估结果表明，我们的框架在达到云端级别准确率的同时，将实际运行碳排放平均降低了 $4\times$。

Abstract:Large Language Models (LLMs) with function-calling capabilities are becoming critical for modern agentic AI systems. Nevertheless, current deployments typically route inferences to powerful cloud-based models, incurring significant energy use and carbon emissions. We address this sustainability challenge with a carbon-aware routing framework that distributes function-calling queries across a three-tier edge-cloud architecture, combining edge and cloud LLMs on heterogeneous hardware. At its core, a lightweight k-NN predictor operating in a unified semantic-lexical embedding space estimates query-specific accuracy, delay, and power consumption on each edge tier. These predictions are then combined with real-time grid carbon intensity to route every query to the lowest-emission tier capable of executing it successfully. Evaluated on state-of-the-art function-calling benchmarks and LLM families, our framework matches cloud-level accuracy while reducing operational carbon emissions by $4\times$ on average.