测试 Gemini 3.5 Flash Lite 在家庭监控中的人体检测表现
英文标题：🔶 Testing Gemini 3.5 Flash Lite for human detection in home surveillance
作者基于 Frigate 和 Casa Segura 搭建家庭入侵检测系统，并测试 Gemini 3.5 Flash Lite 能否减少将宠物狗误判为人的误报（FP）。在包含 18 段真实摄像头片段的基准中，Gemini 3.5 Flash Lite 出现了 1 次误报，而生产环境中的 Gemini 3.1 Flash Lite 和 Gemini 3.1 Pro Preview 均未出现误报或漏报。Gemini 3.5 Flash Lite 的价格较 3.1 版本提高约 50%，延迟也略高，但并未带来更好的准确率。作者最终认为，与其切换到新模型，不如对同一模型进行二次确认查询，尽管这会增加延迟且方案并不完美。
中文
我构建了一套基于 Frigate 的定制家庭监控系统。Frigate 负责人体检测部分，并将结果输送给 Casa Segura 进行更精细的检测。这就像一个用于入侵检测的漏斗。Frigate 的算法基于 TensorFlow Lite 或 OpenVINO 等机器学习模型，使用 MobileNet 或 YOLO 等模型进行人体检测。目前，系统使用的是 Gemini-3.1-Flash-Lite，效果不错，但存在误报问题：我的狗会触发系统，被检测为人。我不想在凌晨 3 点听到警报。这正是降低 FP（误报）的主要目标。
最近，Gemini-3.5-Flash-Lite 发布了，价格上涨了 50%，我觉得这有些奇怪。我原本以为同一档位的模型应该延续相同定价。也许模型有了很大改进，这就是涨价的理由。我已经有点厌倦逐一阅读每张新模型发布说明卡了。
我确实很期待测试它，因为我一直在等待一个能够以零样本方式降低 FP 的模型。我有一个包含以往 FP 事件片段的小型基准集，全部来自真实环境；目前该基准由 kimi k3 管理（它由 _Claude Opus 4.5 及后续模型_构建）。
18 段摄像头片段，逐帧由“人工”核验（其中 10 段没有人类——白天/夜间的狗、躺着的狗、远处的狗；8 段包含人类，包括人与狗同时出现在画面中的场景）。
误报
当模型把狗误判为人时。
| 模型 | FP（30 次运行） | 漏检（24 次运行） |
|---|---|---|
| gemini-3.1-flash-lite（生产环境） | 0 | 0 |
| gemini-3.5-flash-lite（“全新且改进版”） | 1（夜间、远处的深色狗；置信度 0.90） | 0 |
| gemini-3.1-pro-preview（“pro”版） | 0 | 0 |
定价
Gemini 3.1 Pro 相当昂贵。3.1 仍比 3.5 便宜 40%。
| 模型 | 置信度 | 输入/输出价格（美元/百万 token） |
|---|---|---|
| gemini-3.1-flash-lite（生产环境） | 0.95–1.00 | $0.25 / $1.50 |
| gemini-3.5-flash-lite（“全新且改进版”） | 0.90–0.95 | $0.30 / $2.50 |
| gemini-3.1-pro-preview（“pro”版） | 0.95–1.00 | 约为 flash-lite 的 8 倍 |
按模型划分的延迟
Gemini 3.1 和 3.5 非常接近，但 3.1 胜出。Gemini 3.1 Pro 则确实很慢。
| 模型 | 延迟 | 成本（54 次调用） | 结论 |
|---|---|---|---|
| gemini-3.1-flash-lite（生产环境） | 约 2.9 秒（中位数） | 约 $0.04（估算） | 保留 |
| gemini-3.5-flash-lite（“全新且改进版”） | 约 3 秒 | $0.059（实测） | 淘汰 |
| gemini-3.1-pro-preview（“pro”版） | 约 10 秒（3.4 倍） | 约 $0.30（估算） | 淘汰：准确率相同，成本高 8 倍 |
每段片段运行 3 次，使用与生产环境相同的提示词和转码设置（6 秒、2 fps、768px），temperature 为 0.2。
据 Kimi 分析：3.5 的那次单独 FP 置信度为 0.90——这正是没有任何模型能完全消除的长尾幻觉。面对这种情况，更昂贵的模型并不能胜出；需要进行第二次查询。
结论
遗憾的是，这个新模型比我已有的 Gemini 3.1 Flash Lite 略差；真正解决误报问题的并不是更换模型，而是让同一个模型再次检查。我觉得这是一种带有技巧性的方案，并不完美，而且会增加延迟。
English / 原文
I’ve built a custom home surveillance system based around Frigate. Frigate does the person detection part and feeds Casa Segura for finer detection. It’s like a funnel for intrusion detection. Frigate’s algorithm is based on machine learning models such as TensorFlow Lite or OpenVINO, using models like MobileNet or YOLO for person detection. The system is urrently using Gemini-3.1-Flash-Lite which works well, but there is an problem with false positives, basically my dogs triggers the system detected as persons. I don’t want to hear an alarm at 3AM. That’s the main point on lowering the FP.
Recently Gemini-3.5-Flash-Lite was introduced, there is 50% bump in pricing, which I find it odd, I would expect that the same tier of model continue to have the same pricing. Maybe they improved the model a lot, and that’s the justification of the price bump. I’m a bit tired of going through reading every new card of a model introduction.
I was excited to test it because I’m waiting for a model that can lower the FP on zero shot. I’ve a small benchmark with clips from previous FP events, which are real world, currently the benchmark is managed by kimi k3 (it was build with Claude Opus 4.5 and succesors models).
18 camera clips, verified frame by frame by “hand” (10 without humans — dogs day/night, dog lying down, dog far away; 8 with humans, including person + dogs in scene).
FALSE POSITIVE
When the model mistakes a dog for a person.
| Model | FP (30 runs) | Miss (24 runs) |
|---|---|---|
| gemini-3.1-flash-lite (production) | 0 | 0 |
| gemini-3.5-flash-lite (the “new and improved”) | 1 (dark dog, far away, at night; conf 0.90) | 0 |
| gemini-3.1-pro-preview (the “pro”) | 0 | 0 |
Pricing
Gemini 3.1 Pro is pretty expensive. 3.1 is still 40% cheaper than 3.5.
| Model | Confidence | Price in/out ($/M) |
|---|---|---|
| gemini-3.1-flash-lite (production) | 0.95–1.00 | $0.25 / $1.50 |
| gemini-3.5-flash-lite (the “new and improved”) | 0.90–0.95 | $0.30 / $2.50 |
| gemini-3.1-pro-preview (the “pro”) | 0.95–1.00 | ~8× flash-lite |
Latency by model
Gemini 3.1 and 3.5 are very similar, but 3.1 wins. Gemini 3.1 Pro, is really slow.
| Model | Latency | Cost (54 calls) | Verdict |
|---|---|---|---|
| gemini-3.1-flash-lite (production) | ~2.9 s (median) | ~$0.04 (est.) | Kept |
| gemini-3.5-flash-lite (the “new and improved”) | ~3 s | $0.059 (measured) | Discarded |
| gemini-3.1-pro-preview (the “pro”) | ~10 s (3.4×) | ~$0.30 (est.) | Discarded: same accuracy, 8× cost |
3 runs per clip, same prompt and transcode as production (6s, 2 fps, 768px), temperature 0.2.
According to Kimi: The single FP of the 3.5 came with confidence 0.90 — exactly the kind of tail hallucination that no model eliminates. Against that, a more expensive model doesn’t win; a second query is needed.
Conclusion
Sadly the new model was marginally worse than the one I already had (Gemini 3.1 Flash Lite); what fixed the false positives wasn’t switching models, but asking the same model to check again. I feel that is hackish, not perfect and adds latency.