Show HN：Needle2——面向手机、可穿戴设备、智能家居和机器人的 14MB Agentic LLM

将端侧 AI 带到 200 美元以下的设备： 近来，人们谈及边缘 AI 往往指向 Mac 和 PC，但边缘设备的主体其实是廉价硬件：全球联网 IoT 设备超过 210 亿台，而 PC 约为 15 亿台；在新兴市场，大多数手机的售价低于 200 美元。若将入门手机、树莓派、微控制器、可穿戴设备、Reachy Mini 等小型机器人以及联网家居设备计算在内，约五分之四的边缘设备售价低于 200 美元。Needle 的目标正是这类硬件：没有 GPU、没有 NPU，只有数百 MB 的 RAM。

函数调用与设备使用： 打开一盏灯并不需要前沿模型。手表、家庭和机器人都已将自身能力暴露为带类型参数的函数，因此唯一困难的部分，是将一句杂乱的自然语言映射到这些函数：调用哪个函数，以及传入哪些值。以这种方式界定问题后，它不需要世界知识，也不需要开放式文本生成；这正是为什么聊天需要数十亿参数，而该任务只需 4500 万参数。这一更小的任务定义，是后续一切设计所基于的赌注。

提取与结构化输出： Schema 就是接口，而同样的定义也适用于文档处理：一个 Schema 加上一段文本，会返回带类型的字段；枚举字段是分类器；数组字段则可在一次调用中收集一个列表。我们通过契约而非约定来强制执行这一点：每一轮都以调用封装形式回答，空调用代表拒绝，而由声明的 Schema 编译而成的字节级语法会约束每一个 token。语法负责承载语法结构，因此全部 4500 万参数都可用于选择函数，并将参数与用户的话语准确对应。

端云协作： 没有任何小模型能够覆盖所有需求，因此 Needle 不会猜测，而是明确说明：每次响应都会携带一个学习得到的置信度分数，偏离主题的请求会返回空调用。高于你的阈值就执行；低于阈值则重新询问或升级至云端。绝大多数设备请求都是常规控制，因此升级至云端的情况很少，默认路径仍然保持私密、即时且免费。

无损 2bit 量化： 小模型在后量化后往往会失效，因此我们从不进行后量化：Needle 2 从预训练到训练后阶段，均针对 Cactus Quants 进行训练，覆盖权重、激活值和 KV cache。你部署的 2bit 模型，就是实际训练过的模型。这正是为何 4500 万参数能够在我们的一组测试中毫无损失地装入 14MB。

协同设计的模型与推理： 每一项架构选择，都是先在目标硬件上完成基准测试，才获得其参数预算；交付物并非单独的权重，而是一对协同产物：一个无任何依赖的 C++ 二进制文件，会在启动时探测 CPU 并选择对应内核，模型、tokenizer 和语法编译器均封装在其中。一个制品可运行于 Cortex-M、x86 和 WebAssembly。无需安装，也无需下载。

在你的 Mac/PC 上微调： 每个产品都有自己的工具词汇，而 4500 万参数的模型足够小，可以在其运行环境中重新训练：代码仓库和 Python 包可在你自己的电脑上完成调优和测试，耗时从几分钟到数小时不等。部署一个能够使用你设备工具、而非泛用助手语言的 Needle。

Bringing On-Device AI to <$200 Devices: Edge AI has lately meant Macs and PCs, but the edge is mostly cheap hardware: over 21 billion connected IoT devices against roughly 1.5 billion PCs, and in emerging markets most phones ship under $200. Count budget phones, Raspberry Pis, microcontrollers, wearables, small robots like Reachy Mini, and connected home devices, and roughly four in five edge devices cost under $200. That is the hardware Needle targets: no GPU, no NPU, a few hundred MB of RAM.

Function Call & Device Use: Turning on a light does not need a frontier model. A watch, a home, a robot: each already exposes its abilities as functions with typed parameters, so the only hard part is mapping a messy sentence onto them: which function, with which values. Framed that way, the problem needs no world knowledge and no open-ended prose, which is why 45M parameters suffice where chat needs billions. That smaller formulation is the bet everything else follows from.

Extraction & Structured Outputs: The schema is the interface, and the same formulation covers documents: a schema plus a paragraph returns typed fields, an enum field is a classifier, an array field collects a list in one call. We enforce this with a contract, not a convention: every turn is answered with a call envelope, the empty call is the refusal, and a byte-level grammar compiled from the declared schemas constrains every token. The grammar carries the syntax, so all 45M parameters go to choosing functions and grounding arguments in the user's words.

Edge-Cloud Collaboration: No small model covers everything, so Needle says so instead of guessing: every response carries a learned confidence score, and off-topic requests return the empty call. Above your threshold, act; below it, re-ask or escalate to the cloud. Most device requests are routine control, so escalation stays rare and the default path stays private, instant, and free.

Lossless 2bit Quantization: Small models break under post-hoc quantization, so we never quantize post-hoc: Needle 2 trains against Cactus Quants from pretrain through post-train, weights, activations, and KV cache alike. The 2bit model you deploy is the model that was trained. That is what fits 45M parameters into 14MB with nothing lost on our battery.

Co-designed Model & Inference: Every architectural choice was benchmarked on the target hardware before it earned its parameters, and the deliverable is the pair, not the weights: a single dependency-free C++ binary that probes the CPU at startup and picks its kernels, with the model, tokenizer, and grammar compiler sealed inside. One artifact runs from Cortex-M to x86 to WebAssembly. There is nothing to install and nothing to download.

Fine-tune on your Mac/PC: Every product has its own tool vocabulary, and a 45M model is small enough to retrain where it runs: the repo and python package tune and test on your own computer in minutes to a few hours. Ship a Needle that speaks your device's tools, not a generic assistant.