Launch HN：Screenpipe（YC S26）——记录你的工作并将其转化为 Agent

大家好，Hacker News，我是 Louis。我开发了 Screenpipe（https://screenpipe.com），这是一款在本地（仅本地！）记录屏幕和音频的应用，为 AI Agent 提供可搜索的记忆，使它们能够了解你看到过、说过和听到过的内容。这让自动化重复任务、将其转化为 SOP（标准操作流程）等工作变得更容易。

我制作了一段 HN 风格的演示视频：https://www.tella.tv/video/build-your-ai-second-brain-with-s...；此外还有一段视频/嵌入内容：营销视频。

我长期以来一直痴迷于这个方向。自 2020 年起，我就一直维护着一个“第二大脑”，其中存放日记、手写笔记、我听的音乐、正在进行的项目、与他人的对话、个人 CRM 等内容。早期我使用 ParlAI、数百个微调的 GPT2 模型以及 GPT3 对 RAG 做了许多实验（https://forum.obsidian.md/t/fine-tuning-openai-api-gpt3-on-y...）。后来，我开发了 Ava——首个 Obsidian AI 插件，它很快增长到数千名用户。之后，它演变为 Embedbase，一个帮助开发者构建 RAG 驱动 AI 应用的 API。

我从这一切中学到的是：要让模型按你期望的方式完成任务，它们必须拥有关于你在电脑上做什么的上下文，这一点至关重要。

早期有微调，但过程太痛苦；后来有了工具调用，AI 可以访问你使用的软件，但它依然不够自主，需要大量微观管理。之后 MCP 出现了，但它感觉过于静态，非技术用户也难以构建和使用 MCP。后来我们有了 skills。最近，我们看到 Karpathy 的 LLM 维护型 wiki、Garry 的 GBrain 等项目：一个 Agent 会渐进式地维护一组持久化 Markdown 页面。新来源会更新实体页面、强化或反驳既有论断，并改进一份会随时间累积增值的综合内容。我喜欢这种模式，但它仍然始于某个人主动选择和导入信息来源。AI 依然无法知道你和你的公司每天跨应用在做什么，而不仅仅是知道应用内部的活动。

当然，并非所有人都希望这样。但我希望！我希望 AI 知道我在做什么，永远不会再丢失记忆；我也希望它使用与人类相同的软件，而不需要痛苦的上下文切换。

我在 2024 年开始为自己开发 Screenpipe：一个用于记录屏幕并将这类上下文接入 AI 的 CLI。一位 HN 用户在 2024 年发布了它（https://news.ycombinator.com/item?id=41695840），那场讨论影响了产品的发展。其中最有价值的批评涉及录制同意、本地安全性、CPU 使用率、信噪比，以及 Agent 是否能够基于这些数据执行操作。

最朴素的实现方式，是持续录制视频并对每一帧运行 OCR。但这会产生重复数据，消耗大量资源（基本上会把你的电脑变成一个暖炉！），并且丢弃操作系统本已掌握的结构化信息。现在，Screenpipe 会改为监听应用切换、点击、输入停顿、滚动和空闲回退等事件。当出现有意义的变化时，它会将截图与同一时间戳下的操作系统无障碍树配对。当没有可用的结构化无障碍数据时，才使用 OCR。我们还会持续采集音频，识别说话人，并通过 Parakeet/Whisper 在本地转录，或者使用云端模型。

所有内容都会被索引到本地 SQLite 数据库、MP4 文件，有时也会存入 Markdown 文件。面向 AI 的 API 在 3030 端口开放给 Agent 使用，并提供身份验证、MCP 和 skills。

当 Screenpipe 运行一段时间后，你可以通过内置聊天、Claude、ChatGPT、Hermes、Openclaw 或任何 Agent 使用它来完成如下任务：

• 为当前聊天补充上下文，例如：“收集有关任务 X 的所有上下文”，从而以更少的提示词完成目标。

• 检索信息，例如：“找回我从早上 8 点到下午 4 点处理过的任务，列出已完成内容和剩余事项。”

• 为你的 Agent 创建并维护个人 wiki / 第二大脑：“每隔 1 小时，将我做的所有事情按项目、人员、任务和会议整理到我的 Obsidian vault 中，保存为 Markdown 文件和文件夹。”

• 创建自动化：每当我访问某人的 LinkedIn 个人主页时，更新我的 CRM。

• 发现自动化机会：查看我的团队本周完成的所有工作，并将其转化为自动化机会清单。

Screenpipe 数据保存在本地；不过我们也提供企业版，用于发现自动化机会，在该方案中由企业决定数据存放位置。我们构建了自己的 AI PII 模型来脱敏敏感信息，它可以在 Apple MLX 或 Windows DirectML 上本地运行；我们也支持面向低端设备的云端机密推理，不过我们的本地模型目标是占用少于 1% 的 CPU 和少于 400 MB 的内存。除浏览器无痕模式外，用户还可以设置需要过滤的应用、窗口和 URL。

我们还支持录制计划及其他隐私功能。

我们的大部分代码库采用 Rust、MLX 和 Onnx 编写；对于 Apple API，我们喜欢使用 cidre 或直接进行 C 调用；对于 Windows API，则使用 windows-rs。我们也以实验性方式支持 Linux。

我们提供桌面应用（https://screenpipe.com/how-to-install）和 CLI：

npx screenpipe record

你无需创建账号即可运行它。全部代码均以 source-available 形式提供，位于：

https://github.com/screenpipe/screenpipe

我们迈出了令人畏惧的一步，制定了自己的 Screenpipe Commercial License。我知道 HN 强烈偏好 OSI 开源许可证（MIT/Apache 等），但当企业能够免费将其用于商业用途时，我们找不到能够持续开发 Screenpipe 的可持续方式。因此，现在个人非商业、非营利、教育和研究用途免费，但商业使用需要许可证。

许可证变更前发布的版本仍以 MIT 许可证提供。我们提供免费层级和其他套餐，其中包括帮助企业发现自动化机会的 Enterprise 方案。

很希望听到大家的任何反馈、你们使用 Screenpipe 完成过的事情，或者希望我们加入的功能。

Hi Hacker News, I'm Louis. I built Screenpipe (

https://screenpipe.com

), an app that records your screen and audio locally (only!), and gives AI agents a searchable memory of what you've seen, said, and heard. This makes it easier to automate your repetitive tasks, turn them into SOPs (Standard Operating Procedure) and so on.

I made a HN-style demo video at https://www.tella.tv/video/build-your-ai-second-brain-with-s... and there’s a marketing video at https://www.youtube.com/watch?v=c1jV6E9pyug.

I’ve been obsessed with this for a long time. I’ve been maintaining a “second brain” since 2020, in which I would store journals, handwritten notes, music I listen to, projects I'm working on, conversations I have with people, personal CRM etc. I experimented a lot of RAG in the early days with ParlAI, hundreds of fine-tuned GPT2 models, and GPT3 (https://forum.obsidian.md/t/fine-tuning-openai-api-gpt3-on-y...). Later I built Ava, the first Obsidian AI plugin, which grew to a few thousands of users quickly. It then became Embedbase, an API to make it easier to build AI apps powered by RAG.

What I learned from all this is how important it is for the models to have context about what you’re doing on your computer, in order to get them to do what you want.

In the early days there was fine tuning but it was too much pain, then there was tool calling so that AI can access software you use but still kinda not autonomous enough. needing micro management. Then MCP came, but it felt too static, and non technical users struggled to build and use MCP. Then we got skills. Most recently we’ve seen Karpathy’s LLM-maintained wiki, Garry's GBrain, etc., where an agent incrementally maintains a persistent collection of Markdown pages. New sources update entity pages, strengthen or contradict existing claims, and improve a synthesis that compounds over time. I like this pattern, but it still begins with someone selecting and importing the sources. There is still no way AI can know what you and your company are doing every day, across apps, not just inside of apps.

Of course, not everyone wants this. But I do! I want AI to know what I'm doing and never lose memory ever again, and I want it to use the same software that humans do, without painful context switches.

I started building Screenpipe for myself in 2024 - a CLI to record your screen and plug this context into AI. An HN user posted it in 2024 (https://news.ycombinator.com/item?id=41695840) and that discussion influenced the product. The most useful criticism concerned recording consent, local security, CPU usage, signal-to-noise, and whether agents could act on top of the data.

The naive implementation started from continuously recording video and running OCR over every frame. But that creates duplicate data, consumes substantial resources (it basically turns your computer into a space heater!), and discards structure the operating system already knows. Screenpipe now instead listens for events such as app switches, clicks, typing pauses, scrolling, and idle fallbacks. When something meaningful changes, it pairs a screenshot with the operating system’s accessibility tree at the same timestamp. OCR is used when structured accessibility data is unavailable. We also capture audio continuously, identify speakers and transcribe locally through Parakeet/Whisper or using cloud models.

Everything is indexed in a local SQLite database, mp4 files, and sometimes md files. An AI friendly API on port 3030 is open for agents, with authentication and a MCP and skills.

Once Screenpipe has been up and running for a while, you can use it through our built-in chat, Claude, ChatGPT, Hermes, Openclaw, or any agent, to do things like:

- adding context to your current chat, e.g. "gather all context about task X", then requiring less prompts to achieve your goal

- retrieve information, e.g. "retrieve the tasks i was working on from 8 am to 4 pm, make a list of what got done and what's left"

- create and maintain a personal wiki / second brain for your agents: "every 1h organize everything i do in projects, people, tasks, meetings in my Obsidian vault as markdown files and folders"

- create automations: whenever i visit someone's profile on linkedin, update my crm

- find automation opportunities: look at everything my team has done this week and turn it into a list of automation opportunities

Screenpipe data is stored locally, though we also offer an enterprise plan to discover automation opportunities and for that the company decides where the data lives. We built our own AI PII model to redact sensitive information, it runs locally on Apple MLX or Windows DirectML, we also support cloud confidential inference for low end devices, although our local models are meant to use <1% CPU and <400 mb RAM. Users can set apps, windows, and urls to filter, in addition to browser incognito mode.

We also support recording schedules and other privacy features.

Most of our codebase is written in Rust, MLX, Onnx, we like cidre or direct C call for Apple APIs and windows-rs for Windows API. We also experimentally support Linux.

We have a desktop app (https://screenpipe.com/how-to-install) and a CLI:

  npx screenpipe record

You can run that without creating an account. All the code is source-available at

https://github.com/screenpipe/screenpipe

. We took the dreaded step of making our own Screenpipe Commercial License. I know HN strongly prefers OSI open source (MIT/Apache/etc.) but couldn’t find a sustainable way to keep developing Screenpipe while companies were using it commercially for free. So now personal non-commercial, nonprofit, educational, and research use is free, but commercial use requires a license.

Versions released before the license change remain available under MIT. We have a free tier, and other plans, including Enterprise which helps companies find automation opportunities.

Would love to hear any feedback, things you've done with screenpipe, or features you'd want