单元内学习:已部署语言模型无需改变任何一个已存储比特即可学习新知识
英文标题:In-Cell Learning: Deployed Language Models Can Learn New Knowledge Without Changing a Single Stored Bit
论文提出“单元内学习”(In-Cell Learning)范式:在不改变已发布 4-bit 模型权重整数编码和共享缩放因子的前提下,将新知识写入量化器丢弃的权重区间中。作者实现的 CellFill 采用低秩更新,在 NF4、QAT、GPTQ 风格量化网格以及 Qwen3、Gemma 等 17 亿至 310 亿参数模型上,写入了模型可验证未知的真实事实语料中 83% 至 97% 的知识,并在最多 2.4×10^10 个受约束权重上实现零违规。该更新以独立“填充文件”(fill)保存,可通过相减撤销,并且多个写入者可共享同一个模型发布版本。实验显示,相较 PopQA 长尾知识检索,CellFill 能回答原模型无法回答问题中的 82% 至 90%,平均每题仅需 75 个 token,而检索方法需要 90 至 1008 个 token。
612 B
380 B
9.1 KB
32.7 KB
22.8 KB
24.2 KB
34.7 KB
中文
[提交于 2026 年 8 月 21 日(
),最后修订于 2026 年 8 月 24 日(当前版本,v2)]
标题:单元内学习:已部署语言模型无需改变任何一个已存储比特即可学习新知识
查看 Zifeng Liu 与另外 8 名作者撰写、题为《单元内学习:已部署语言模型无需改变任何一个已存储比特即可学习新知识》的论文 PDF。
摘要:已部署的语言模型是一个许多事物都依赖于它的文件——基准测试报告、认证、一批设备——而每一种向其传授新知识的方式都会产生一个不同的文件。我们表明,这并非必要。一个 4-bit 发布版本将每个权重存储为整数编码和共享缩放因子,并在存储值之间留下量化器会丢弃的区间;我们将“单元内学习”(in-cell learning)称为这样一种范式:新知识仅被写入该区间,因此对服务中的权重重新量化时,能够精确复现已发布的编码和缩放因子。该保证在整数域中针对每一个权重进行检查;更新是一个独立文件(fill),可通过相减撤销;而漂移则由网格在训练前确定的半径加以约束。CellFill 通过构造实现这一范式:在单元内部采用低秩位置,并基于供应商自身发布的 4-bit 模型进行训练。在这些发布版本上——NF4、QAT 和 GPTQ 风格的网格,以及 Qwen3 和 Gemma,模型规模从 17 亿到 310 亿参数——CellFill 写入了模型可验证地尚未知晓的真实事实语料中 83% 至 97% 的知识,并在多达 2.4 x 10^10 个受约束权重上实现零违规。我们证明这些知识可被实际使用:注入的药物能够在模型自身的两跳上限内与注入的成分进行组合;一个截止日期之后的软件库能够以 2.6 倍于随机概率的水平被用于代码中。与 PopQA 长尾知识上的检索方案相比,fill 能够回答已发布模型无法回答问题中的 82% 至 90%,每个问题使用 75 个 token,而检索方案需要 90 至 1008 个 token。多个写入者可以共享同一个发布版本,顺序更新只会消耗剩余空间中的固定比例;支撑每张表格的归档结果文件均位于此 HTTPS URL。
提交历史
来自:Zifeng Liu [
]
2026 年 8 月 21 日,周五,08:49:07 UTC(84 KB)
[v2]
2026 年 8 月 24 日,周一,08:51:42 UTC(143 KB)
English / 原文
[Submitted on 21 Aug 2026 (
), last revised 24 Aug 2026 (this version, v2)]
Title:In-Cell Learning: Deployed Language Models Can Learn New Knowledge Without Changing a Single Stored Bit
View a PDF of the paper titled In-Cell Learning: Deployed Language Models Can Learn New Knowledge Without Changing a Single Stored Bit, by Zifeng Liu and 8 other authors
Abstract:A deployed language model is a file that many things depend on - a benchmark report, a certification, a fleet of devices - and every way of teaching it something new produces a different file. We show that this is not necessary. A 4-bit release stores each weight as an integer code and a shared scale, and leaves between the stored values an interval the quantizer discards; we call in-cell learning the paradigm in which new knowledge is written only into that interval, so that re-quantizing the served weights reproduces the released codes and scales exactly. The guarantee is checked in the integer domain on every weight, the update is a separate file (a fill) that can be withdrawn by subtraction, and drift is bounded by radii the grid fixes before training. CellFill realizes the paradigm by construction: a low-rank position inside the cell, trained on the vendor's own 4-bit release. On those releases - NF4, QAT and GPTQ-style grids, Qwen3 and Gemma, 1.7B to 31B - CellFill writes 83 to 97 percent of a corpus of real facts the models verifiably did not know, with zero violations over up to 2.4 x 10^10 constrained weights. We demonstrate the knowledge is usable: injected drugs compose with injected ingredients at the model's own two-hop ceiling, and a post-cutoff software library is used in code at 2.6 times chance. Against retrieval on PopQA's long tail, the fill answers 82 to 90 percent of questions the released model could not, at 75 tokens per question against 90 to 1008. Several writers can share one release, sequential updates consume a constant fraction of the remaining room, and the archived result files behind every table are at this https URL.
Submission history
From: Zifeng Liu [
]
Fri, 21 Aug 2026 08:49:07 UTC (84 KB)
[v2]
Mon, 24 Aug 2026 08:51:42 UTC (143 KB)