Show HN：Scry，一种采用拥塞定价的可编程互联网搜索
英文标题：🔶 Show HN: Scry, programmable internet search w/ congestion pricing
Scry 展示了如何通过 SQL、全文检索、关系连接和向量语义操作，对 Twitter、Hacker News、LessWrong、OpenAlex 与 Manifold 等互联网数据源进行可编程分析。文中给出多项实际查询示例，包括发掘被收藏多于点赞的 LLM 内容、衡量 LessWrong 对 AI 对齐议题的观点漂移、追踪“vibe coding”传播，以及分析共同关注、论文共引和评论互动。查询结果表明，系统可在数亿级记录上于数秒乃至毫秒级完成分析，并支持原生关系键、时间序列和向量检索。对开发者而言，Scry 的意义在于将传统搜索、数据仓库查询与嵌入式语义分析整合为可组合的查询谓词。
53.3 KB
40.9 KB
56.3 KB
61.6 KB
中文
本周哪些小账号发布了被人收藏、却没有获得太多点赞的 LLM 一手信息？
收藏是比点赞更强的信号，而且没有人能看到是谁收藏了内容。每个归档的推文修订版本都同时包含这两个计数，因此“收藏数高于点赞数”可以作为一个谓词；一个 token 列表将范围限定在编程 Agent 和推理领域，第二个列表排除诱导互动的内容，粉丝数上限则确保结果来自你可能从未听说过的账号。三天数据，读取 2.04 亿个修订版本，耗时 4.9 秒。
SELECT any(author_handle) AS handle, max(author_followers) AS followers,
max(like_count) AS likes, max(bookmark_count) AS saves,
round(saves / likes, 1) AS saves_per_like, substring(any(text), 1, 150) AS text
FROM twitter.tweets
WHERE bucket_date >= today() - 3
AND hasAnyTokens(search_text_lc, ['subagents', 'mcp', 'evals', 'harness', 'worktrees', 'agentic',
'finetuning', 'rlhf', 'tokenizer', 'inference', 'codex', 'opus', 'sonnet', 'cursor', 'devin', 'ralph'])
AND NOT hasAnyTokens(search_text_lc, ['giveaway', 'retweet', 'free', 'course', 'bookmark', 'link',
'below', 'join', 'pdf', 'earn', 'money', 'income', 'breaking', 'viral', 'comment', 'dm', 'follow'])
AND lang = 'en' AND NOT startsWith(text, 'RT @')
AND bookmark_count >= 150 AND bookmark_count >= like_count AND author_followers < 20000
GROUP BY tweet_id
ORDER BY saves_per_like DESC
LIMIT 10;
→ @navaneethvb 558 followers 205 likes 337 saves "A lot of my LLM inference posts here were explaining how…"
@LLMpsycho 1,169 followers 146 likes 242 saves "security-audit-skill is a Cloudflare coding agent skill…"
@syrgkanis 1,087 followers 157 likes 198 saves "…we recently released a fully automated agentic pipeline…"
@jaimintf 2,873 followers 335 likes 467 saves "before → after. right one used mcp + skills, both one-shotted."
LessWrong 在对齐议题上向哪个方向漂移：可实现，还是遥不可及？
将两个极点铸造成仅立场不同的平行句子，取两者之间的平衡轴，将每篇文章按其各个文本块投影的均值评分，再按季度对文章评分取平均，并关联真实的发帖时间戳。窗口函数以时代均值为中心校正每个季度，因为对比轴的零点只意味着与两句话等距；posts 是仅从 forums.posts 中统计的该季度 LessWrong 正式文章总数（其中也包含评论，在 LessWrong 上评论与文章约为十五比一；p.kind = 'post' 将评论排除），scored 是嵌入连接成功覆盖到的文章数，coverage 是两者的比率：只有当覆盖率接近 1 时才应解读观点漂移；未被评分的季度仍会保留其行，drift 为 NULL，而不会消失。一条查询即可返回一个社区的演变轨迹。
SELECT quarter,
if(scored = 0, NULL, lean - avgIf(lean, scored > 0) OVER ()) AS drift,
scored, posts, round(scored / posts, 3) AS coverage
FROM (
SELECT quarter, avg(post_lean) AS lean, count() AS scored
FROM (
SELECT e.target_key,
toStartOfQuarter(any(p.original_timestamp)) AS quarter,
avg(scry_cosine_similarity(e.embedding,
scry_contrast_axis_balanced(@achievable, @unreachable))) AS post_lean
FROM embeddings.chunks AS e
JOIN forums.posts AS p ON p.post_key = e.target_key
WHERE e.source = 'forum_posts' AND e.model_name = 'voyage-4-lite'
AND p.source = 'lesswrong' AND p.kind = 'post'
AND p.original_timestamp >= '2009-01-01'
GROUP BY e.target_key)
GROUP BY quarter) AS s
RIGHT JOIN (
SELECT toStartOfQuarter(original_timestamp) AS quarter, uniqExact(post_key) AS posts
FROM forums.posts
WHERE source = 'lesswrong' AND kind = 'post' AND original_timestamp >= '2009-01-01'
GROUP BY quarter) AS c USING (quarter)
ORDER BY quarter
LIMIT 100;
“vibe coding” 流行得有多快？
搜索语法在查询内部以单个谓词运行——这里作为聚合操作数，其分母是当月的全部条目数。Hacker News：截至 2025 年 1 月均为零；2 月有 34 次提及（首条内容于 2025-02-03 发布，并链接到 Karpathy 的帖子）；3 月为 494 次，4 月为 590 次；耗时 4 秒，测量日期为 2026-09-07。
SELECT toStartOfMonth(original_timestamp) AS month, count() AS items,
countIf(scry_lex('"vibe coding"')) AS mentions,
round(100000.0 * mentions / items, 1) AS per_100k
FROM hackernews.items
WHERE original_timestamp >= '2024-12-01'
GROUP BY month
ORDER BY month
LIMIT 36;
→ 2025-01 0 0.0 per 100k
2025-02 34 11.9
2025-03 494 169.4
2025-04 590 212.6
Elon Musk、Sam Altman 和 Eliezer Yudkowsky 都关注了谁？
关注关系图是一张表，因此“三人都关注的所有人”只需要一次带 HAVING 的 GROUP BY——这是 Twitter 在 17 年中从未提供过的功能。结果为 10 个账号，Vitalik 和 JD Vance 位居前列，Ilya、Hinton 和 Scott Alexander 排在更后面；读取 290 万行数据耗时 171 毫秒。
WITH common AS (
SELECT followee_id FROM twitter.following
WHERE follower_id IN (44196397, 1605, 2595244026) -- @elonmusk, @sama, @ESYudkowsky
GROUP BY followee_id HAVING uniqExact(follower_id) = 3)
SELECT argMax(handle, observed_on) AS handle, argMax(followers, observed_on) AS follower_count
FROM twitter.users
WHERE author_id IN (SELECT followee_id FROM common)
GROUP BY author_id
ORDER BY follower_count DESC
LIMIT 10;
→ VitalikButerin 7,866,454
JDVance 5,551,484
paulg 5,243,364
karpathy 4,165,859
demishassabis 1,865,207 … waitbutwhy, ilyasut, geoffreyhinton, slatestarcodex, ShaneLegg
哪些论文同时引用了 Scaling Laws 和 Chinchilla？
引用边是原生键，因此“相关工作”可变成一个集合交集：找出同时引用 Scaling Laws for Neural Language Models 与 Training Compute-Optimal Large Language Models 的每篇 OpenAlex 论文，并按这些论文自身的引用数排序。共有 195 篇共同引用论文，耗时 0.1 秒；测量日期为 2026-09-07，所用数据为 2026-06-26 的 OpenAlex 快照。
SELECT id, title, publication_year, cited_by_count
FROM openalex.works
WHERE id IN (
SELECT citing_work_id FROM openalex.cited_by
WHERE cited_work_id IN ('https://openalex.org/W3001279689', -- Scaling Laws, 2020
'https://openalex.org/W4225591000') -- Chinchilla, 2022
GROUP BY citing_work_id HAVING uniqExact(cited_work_id) = 2)
ORDER BY cited_by_count DESC
LIMIT 10;
→ Large language models encode clinical knowledge 2023 3,194
Can LLMs Transform Computational Social Science? 2023 429
ProGen2: boundaries of protein language models 2023 426
Hacker News 上谁一直在和谁争论？
对 parent_hn_id 进行自连接，可将每条回复与其所回应的评论配对；least/greatest 将配对变为无序，HAVING 则仅保留双方都至少回复对方三次的配对。每位常客都能感受到这些争执；这是首次有人能够对它们进行计数——三周数据，读取 4670 万行，耗时 1.6 秒。
SELECT least(a.original_author, b.original_author) AS one,
greatest(a.original_author, b.original_author) AS other,
count() AS replies, uniqExact(a.story_hn_id) AS threads
FROM hackernews.items AS a
INNER JOIN hackernews.items AS b ON a.parent_hn_id = b.hn_id
WHERE a.original_timestamp >= now() - INTERVAL 21 DAY
AND a.original_author != b.original_author
GROUP BY one, other
HAVING countIf(a.original_author = one) >= 3
AND countIf(a.original_author = other) >= 3
ORDER BY threads DESC, replies DESC
LIMIT 8;
→ JumpCrisscross ↔ toomuchtodo 11 replies 8 threads
HarHarVeryFunny ↔ famouswaffles 39 replies 6 threads
mitxela ↔ ranger_danger 20 replies 5 threads
WalterBright ↔ nrr 21 replies 4 threads
本周预测者改变了哪些看法？
每笔 Manifold 投注都记录了市场概率在投注前后的数值，因此一个市场的一周可由首个价格的 argMin 和最后一个价格的 argMax 得出，并关联其最新标题。筛选拥有 30 名或更多投注者、且不属于 Manifold 自身 unranked 主题的二元市场，再按群体观点变动幅度排序——新闻发生之前的新闻；2700 万笔投注，耗时 268 毫秒。
WITH moves AS (
SELECT contract_id,
argMin(prob_before, created_at_source) AS p_start,
argMax(prob_after, created_at_source) AS p_now,
uniqExact(user_id) AS bettors
FROM manifold.bets
WHERE created_at_source >= now() - INTERVAL 7 DAY
AND prob_before != prob_after AND answer_id IS NULL
GROUP BY contract_id HAVING bettors >= 30)
SELECT m.title, round(100 * p_start) AS was, round(100 * p_now) AS now, bettors
FROM moves
INNER JOIN (SELECT market_id, argMax(title, observed_on) AS title
FROM manifold.markets
WHERE market_id IN (SELECT contract_id FROM moves)
GROUP BY market_id
HAVING NOT has(JSONExtract(argMax(raw_market, observed_on),
'groupSlugs', 'Array(String)'), 'unranked')) AS m
ON m.market_id = moves.contract_id
ORDER BY abs(p_now - p_start) DESC
LIMIT 8;
→ Assassination attempt on a major AI-lab CEO by 2030? 30% → 99% 69 bettors
Will Trump interfere with Greenland? 80% → 17% 32
US average gas price $4.40 or more on September 21? 50% → 97% 59
All Millennium Prize Problems solved before EOY? 50% → 3% 31
Opus 5.2 released in September? 50% → 24% 93
文本
scry_lex('"exact phrase" -noise /regex/') 整套搜索语法作为一个谓词
a NEAR/50 b · "phrase"~3 · word~1 邻近度、词距和拼写容错，并通过索引裁剪
hasAllTokens(search_text_lc, […]) 对完整归档进行基于 token 索引的匹配——覆盖每一行，而非排序后的样本
toStartOfMonth · uniq · argMin 时间序列、去重作者、首次观测值
JOIN … ON post_key / hn_id / contract_id 跨关系使用源数据原生键
语义
POST /v1/scry/embed → @handle 从任意文本生成一个具名向量
scry_contrast_axis(@a, @b) 两个概念之间的轴
scry_project_onto · scry_debias_vector 保留或移除某一种语义成分
scry_seed_centroid([@x, @y, …]) 由示例构建的概念
scry_cosine_similarity · scry_handle_matrix 测量单个向量对，或一次测量所有向量对
scry_vector_topk_distance 沿上述任一方向对已建立索引的关系执行 ANN 排序
English / 原文
Which small accounts posted LLM alpha this week that people saved but didnât like?
A bookmark is a stronger signal than a like and nobody can see who left it. Every archived tweet revision has both counts, so âsaved more than likedâ is one predicate; a token list keeps it on coding agents and inference, a second one drops the giveaway bait, and a follower ceiling keeps it to accounts you have never heard of. Three days, 204 million revisions read, 4.9 s.
SELECT any(author_handle) AS handle, max(author_followers) AS followers,
max(like_count) AS likes, max(bookmark_count) AS saves,
round(saves / likes, 1) AS saves_per_like, substring(any(text), 1, 150) AS text
FROM twitter.tweets
WHERE bucket_date >= today() - 3
AND hasAnyTokens(search_text_lc, ['subagents', 'mcp', 'evals', 'harness', 'worktrees', 'agentic',
'finetuning', 'rlhf', 'tokenizer', 'inference', 'codex', 'opus', 'sonnet', 'cursor', 'devin', 'ralph'])
AND NOT hasAnyTokens(search_text_lc, ['giveaway', 'retweet', 'free', 'course', 'bookmark', 'link',
'below', 'join', 'pdf', 'earn', 'money', 'income', 'breaking', 'viral', 'comment', 'dm', 'follow'])
AND lang = 'en' AND NOT startsWith(text, 'RT @')
AND bookmark_count >= 150 AND bookmark_count >= like_count AND author_followers < 20000
GROUP BY tweet_id
ORDER BY saves_per_like DESC
LIMIT 10;
â @navaneethvb 558 followers 205 likes 337 saves "A lot of my LLM inference posts here were explaining howâ¦"
@LLMpsycho 1,169 followers 146 likes 242 saves "security-audit-skill is a Cloudflare coding agent skillâ¦"
@syrgkanis 1,087 followers 157 likes 198 saves "â¦we recently released a fully automated agentic pipelineâ¦"
@jaimintf 2,873 followers 335 likes 467 saves "before â after. right one used mcp + skills, both one-shotted."
Which way has LessWrong drifted on alignment: reachable, or out of reach?
Mint the two poles as parallel sentences that differ only in stance, take the balanced axis between them, score every post once as the mean of its chunksâ projections, and average the posts quarter by quarter, joined to real post timestamps. The window function centers each quarter on the era mean, because the zero of a contrast axis only means equidistant from the two sentences; posts is the census of the quarterâs LessWrong posts proper from forums.posts alone (it contains comments too, fifteen to a post on LessWrong; p.kind = 'post' leaves them out), scored is how many of them the embedding join reached, and coverage is their ratio: read a drift only where coverage is near 1, and an unscored quarter keeps its row with drift NULL instead of vanishing. One query returns the trajectory of a community.
SELECT quarter,
if(scored = 0, NULL, lean - avgIf(lean, scored > 0) OVER ()) AS drift,
scored, posts, round(scored / posts, 3) AS coverage
FROM (
SELECT quarter, avg(post_lean) AS lean, count() AS scored
FROM (
SELECT e.target_key,
toStartOfQuarter(any(p.original_timestamp)) AS quarter,
avg(scry_cosine_similarity(e.embedding,
scry_contrast_axis_balanced(@achievable, @unreachable))) AS post_lean
FROM embeddings.chunks AS e
JOIN forums.posts AS p ON p.post_key = e.target_key
WHERE e.source = 'forum_posts' AND e.model_name = 'voyage-4-lite'
AND p.source = 'lesswrong' AND p.kind = 'post'
AND p.original_timestamp >= '2009-01-01'
GROUP BY e.target_key)
GROUP BY quarter) AS s
RIGHT JOIN (
SELECT toStartOfQuarter(original_timestamp) AS quarter, uniqExact(post_key) AS posts
FROM forums.posts
WHERE source = 'lesswrong' AND kind = 'post' AND original_timestamp >= '2009-01-01'
GROUP BY quarter) AS c USING (quarter)
ORDER BY quarter
LIMIT 100;
How fast did âvibe codingâ catch on?
The search grammar runs inside the query as one predicate â here as an aggregate operand, with the monthâs whole item count as its denominator. Hacker News: zero through January 2025, 34 mentions in February (the first item, 2025-02-03, links Karpathyâs post), 494 in March, 590 in April; 4 s, measured 2026-09-07.
SELECT toStartOfMonth(original_timestamp) AS month, count() AS items,
countIf(scry_lex('"vibe coding"')) AS mentions,
round(100000.0 * mentions / items, 1) AS per_100k
FROM hackernews.items
WHERE original_timestamp >= '2024-12-01'
GROUP BY month
ORDER BY month
LIMIT 36;
â 2025-01 0 0.0 per 100k
2025-02 34 11.9
2025-03 494 169.4
2025-04 590 212.6
Who do Elon Musk, Sam Altman and Eliezer Yudkowsky all follow?
The follow graph is a table, so âeveryone all three of them followâ is one GROUP BY with a HAVING â a button Twitter never shipped in seventeen years. Ten accounts, Vitalik and JD Vance at the top, Ilya, Hinton and Scott Alexander further down, in 171 ms over 2.9 million rows read.
WITH common AS (
SELECT followee_id FROM twitter.following
WHERE follower_id IN (44196397, 1605, 2595244026) -- @elonmusk, @sama, @ESYudkowsky
GROUP BY followee_id HAVING uniqExact(follower_id) = 3)
SELECT argMax(handle, observed_on) AS handle, argMax(followers, observed_on) AS follower_count
FROM twitter.users
WHERE author_id IN (SELECT followee_id FROM common)
GROUP BY author_id
ORDER BY follower_count DESC
LIMIT 10;
â VitalikButerin 7,866,454
JDVance 5,551,484
paulg 5,243,364
karpathy 4,165,859
demishassabis 1,865,207 ⦠waitbutwhy, ilyasut, geoffreyhinton, slatestarcodex, ShaneLegg
Which papers cite both Scaling Laws and Chinchilla?
Citation edges are native keys, so ârelated workâ becomes a set intersection: every OpenAlex work that cites both Scaling Laws for Neural Language Models and Training Compute-Optimal Large Language Models , ranked by its own citation count. 195 shared citers, 0.1 s, measured 2026-09-07 against the 2026-06-26 OpenAlex snapshot.
SELECT id, title, publication_year, cited_by_count
FROM openalex.works
WHERE id IN (
SELECT citing_work_id FROM openalex.cited_by
WHERE cited_work_id IN ('https://openalex.org/W3001279689', -- Scaling Laws, 2020
'https://openalex.org/W4225591000') -- Chinchilla, 2022
GROUP BY citing_work_id HAVING uniqExact(cited_work_id) = 2)
ORDER BY cited_by_count DESC
LIMIT 10;
â Large language models encode clinical knowledge 2023 3,194
Can LLMs Transform Computational Social Science? 2023 429
ProGen2: boundaries of protein language models 2023 426
Who keeps arguing with whom on Hacker News?
A self-join on parent_hn_id pairs every reply with the comment it answers; least/greatest make the pair unordered, and the HAVING keeps only pairs where both people replied to the other at least three times. Every regular feels the feuds; this is the first time anyone could count them â three weeks, 46.7 million rows read, 1.6 s.
SELECT least(a.original_author, b.original_author) AS one,
greatest(a.original_author, b.original_author) AS other,
count() AS replies, uniqExact(a.story_hn_id) AS threads
FROM hackernews.items AS a
INNER JOIN hackernews.items AS b ON a.parent_hn_id = b.hn_id
WHERE a.original_timestamp >= now() - INTERVAL 21 DAY
AND a.original_author != b.original_author
GROUP BY one, other
HAVING countIf(a.original_author = one) >= 3
AND countIf(a.original_author = other) >= 3
ORDER BY threads DESC, replies DESC
LIMIT 8;
â JumpCrisscross â toomuchtodo 11 replies 8 threads
HarHarVeryFunny â famouswaffles 39 replies 6 threads
mitxela â ranger_danger 20 replies 5 threads
WalterBright â nrr 21 replies 4 threads
What did forecasters change their minds about this week?
Every Manifold bet has the market probability before and after it, so a marketâs week is argMin of the first price and argMax of the last, joined to its latest title. Binary markets with thirty or more bettors, outside Manifoldâs own unranked topic, ranked by how far the crowd moved â the news before the news, 27 million bets in 268 ms.
WITH moves AS (
SELECT contract_id,
argMin(prob_before, created_at_source) AS p_start,
argMax(prob_after, created_at_source) AS p_now,
uniqExact(user_id) AS bettors
FROM manifold.bets
WHERE created_at_source >= now() - INTERVAL 7 DAY
AND prob_before != prob_after AND answer_id IS NULL
GROUP BY contract_id HAVING bettors >= 30)
SELECT m.title, round(100 * p_start) AS was, round(100 * p_now) AS now, bettors
FROM moves
INNER JOIN (SELECT market_id, argMax(title, observed_on) AS title
FROM manifold.markets
WHERE market_id IN (SELECT contract_id FROM moves)
GROUP BY market_id
HAVING NOT has(JSONExtract(argMax(raw_market, observed_on),
'groupSlugs', 'Array(String)'), 'unranked')) AS m
ON m.market_id = moves.contract_id
ORDER BY abs(p_now - p_start) DESC
LIMIT 8;
â Assassination attempt on a major AI-lab CEO by 2030? 30% â 99% 69 bettors
Will Trump interfere with Greenland? 80% â 17% 32
US average gas price $4.40 or more on September 21? 50% â 97% 59
All Millennium Prize Problems solved before EOY? 50% â 3% 31
Opus 5.2 released in September? 50% â 24% 93
Text
scry_lex('"exact phrase" -noise /regex/')the whole search grammar as one predicate
a NEAR/50 b · "phrase"~3 · word~1proximity, slop, and typo tolerance, index-pruned
hasAllTokens(search_text_lc, [â¦])token-indexed match over full archives â every row, not a ranked sample
toStartOfMonth · uniq · argMintime series, distinct authors, first observations
JOIN ⦠ON post_key / hn_id / contract_idsource-native keys across relations
Meaning
POST /v1/scry/embed â @handlemint a named vector from any text
scry_contrast_axis(@a, @b)the axis between two ideas
scry_project_onto · scry_debias_vectorkeep, or remove, one component of meaning
scry_seed_centroid([@x, @y, â¦])a concept built from examples
scry_cosine_similarity · scry_handle_matrixmeasure one pair, or all pairs at once
scry_vector_topk_distanceANN-rank an indexed relation along any of the above