商业产业 Hacker News LLM 2026-09-22T13:01 正文 ok 1406 words

Show HN：JevBench——面向类型化决策模型的可复现基准测试

英文标题：🔶 Show HN: JevBench, a reproducible benchmark for typed decision models

JevBench 是一个面向类型化决策模型的可复现基准测试项目，原文展示了多种模型和系统的排名及分数结果。榜单中，Jev 1.13.0 以 74.4 分位列 JevBench Score 第一，SemIf 和 djev 分别以 73.1 分、73.0 分紧随其后。不同评估列中的名次与 JevBench Score 排名经常不一致，说明模型在不同任务或评估维度上的表现存在显著差异。对开发者而言，该榜单可用于比较决策模型、重排序器及相关系统在一致、可复现实验条件下的能力边界。

打开原文相关 2 · 机会 0

50.0 KB

中文

Jev 1.13.0| #1 74.4| #3 71.8（排名与 JevBench Score 不同）| #2 77.1（排名与 JevBench Score 不同）| #4 76.2（排名与 JevBench Score 不同）| #9 63.1（排名与 JevBench Score 不同）
SemIf| #2 73.1| #2 73.3| #3 75.5（排名与 JevBench Score 不同）| #2 77.3| #4 67.4（排名与 JevBench Score 不同）
djev| #3 73.0| #1 75.8（排名与 JevBench Score 不同）| #1 78.5（排名与 JevBench Score 不同）| #1 81.7（排名与 JevBench Score 不同）| #3 67.9
Winnow-12B Q8| #4 71.2| #4 71.0| #4 75.2| #5 75.3（排名与 JevBench Score 不同）| #10 63.1（排名与 JevBench Score 不同）
reflex 4B| #5 70.3| #6 68.8（排名与 JevBench Score 不同）| #5 73.1| #17 68.4（排名与 JevBench Score 不同）| #5 65.0
jqv| #6 68.6| #14 65.5（排名与 JevBench Score 不同）| #9 70.7（排名与 JevBench Score 不同）| #14 69.0（排名与 JevBench Score 不同）| #14 57.6（排名与 JevBench Score 不同）
decision-machine-1| #7 68.3| #9 67.6（排名与 JevBench Score 不同）| #22 65.4（排名与 JevBench Score 不同）| #3 76.8（排名与 JevBench Score 不同）| #11 61.7（排名与 JevBench Score 不同）
decider-35b-a3b| #8 67.6| #13 66.3（排名与 JevBench Score 不同）| #8 71.3| #10 71.7（排名与 JevBench Score 不同）| #18 56.9（排名与 JevBench Score 不同）
open-alternative-jev| #9 67.0| #7 68.3（排名与 JevBench Score 不同）| #17 66.6（排名与 JevBench Score 不同）| #6 74.0（排名与 JevBench Score 不同）| #8 64.7（排名与 JevBench Score 不同）
system-one-open| #10 66.6| #5 70.3（排名与 JevBench Score 不同）| #11 70.0（排名与 JevBench Score 不同）| #9 72.9（排名与 JevBench Score 不同）| #2 68.0（排名与 JevBench Score 不同）
OpenJev| #11 66.4| #11 66.9| #7 71.6（排名与 JevBench Score 不同）| #8 73.0（排名与 JevBench Score 不同）| #15 57.3（排名与 JevBench Score 不同）
SimpleJev Qwen3.8-27B| #12 66.3| #18 62.0（排名与 JevBench Score 不同）| #10 70.2（排名与 JevBench Score 不同）| #24 65.5（排名与 JevBench Score 不同）| #22 51.8（排名与 JevBench Score 不同）
ZeroEntropy zerank-2| #13 66.0| #15 62.8（排名与 JevBench Score 不同）| #29 62.9（排名与 JevBench Score 不同）| #15 68.8（排名与 JevBench Score 不同）| #16 57.2（排名与 JevBench Score 不同）
GPT-5.6 Luna| #14 65.9| #23 59.5（排名与 JevBench Score 不同）| #6 71.8（排名与 JevBench Score 不同）| #23 66.1（排名与 JevBench Score 不同）| #30 44.3（排名与 JevBench Score 不同）
openjev-sglang| #15 65.3| #19 61.7（排名与 JevBench Score 不同）| #12 69.6（排名与 JevBench Score 不同）| #18 67.4（排名与 JevBench Score 不同）| #24 50.0（排名与 JevBench Score 不同）
Qwen3-Reranker-4B| #16 63.8| #16 62.8| #27 63.3（排名与 JevBench Score 不同）| #16 68.7| #17 56.9（排名与 JevBench Score 不同）
reflex-27b| #17 63.3| #26 57.2（排名与 JevBench Score 不同）| #16 67.2（排名与 JevBench Score 不同）| #28 61.1（排名与 JevBench Score 不同）| #27 45.5（排名与 JevBench Score 不同）
LitJev| #18 62.7| #28 57.0（排名与 JevBench Score 不同）| #19 66.0（排名与 JevBench Score 不同）| #29 60.7（排名与 JevBench Score 不同）| #26 46.1（排名与 JevBench Score 不同）
kev 0.6B| #19 62.5| #12 66.8（排名与 JevBench Score 不同）| #30 60.4（排名与 JevBench Score 不同）| #13 70.2（排名与 JevBench Score 不同）| #1 70.4（排名与 JevBench Score 不同）
SimpleJev Qwen3.6-35B-A3B| #20 62.5| #21 61.0（排名与 JevBench Score 不同）| #14 67.8（排名与 JevBench Score 不同）| #21 66.3（排名与 JevBench Score 不同）| #23 50.6（排名与 JevBench Score 不同）
djev| #21 62.4| #30 54.6（排名与 JevBench Score 不同）| #25 63.9（排名与 JevBench Score 不同）| #27 62.1（排名与 JevBench Score 不同）| #34 41.1（排名与 JevBench Score 不同）
jev-local| #22 61.8| #22 59.6| #26 63.9（排名与 JevBench Score 不同）| #26 63.3（排名与 JevBench Score 不同）| #21 52.5（排名与 JevBench Score 不同）
decider-2b| #23 61.7| #8 67.7（排名与 JevBench Score 不同）| #23 65.1| #7 73.5（排名与 JevBench Score 不同）| #7 64.9（排名与 JevBench Score 不同）
Bespoke Nimble 9B| #24 60.5| #24 58.9| #20 65.9（排名与 JevBench Score 不同）| #22 66.2（排名与 JevBench Score 不同）| #25 47.0（排名与 JevBench Score 不同）
Gemini 3.1 Flash-Lite| #25 60.1| #25 57.6| #15 67.5（排名与 JevBench Score 不同）| #20 66.3（排名与 JevBench Score 不同）| #32 42.8（排名与 JevBench Score 不同）
OpenJev| #26 60.0| #27 57.1（排名与 JevBench Score 不同）| #13 67.9（排名与 JevBench Score 不同）| #25 64.0（排名与 JevBench Score 不同）| #31 42.8（排名与 JevBench Score 不同）
kev 4B| #27 59.7| #10 67.2（排名与 JevBench Score 不同）| #18 66.2（排名与 JevBench Score 不同）| #12 70.5（排名与 JevBench Score 不同）| #6 65.0（排名与 JevBench Score 不同）
DeepSeek V4.1 Flash| #28 57.5| #34 48.4（排名与 JevBench Score 不同）| #28 63.2| #33 56.6（排名与 JevBench Score 不同）| #40 31.7（排名与 JevBench Score 不同）
kev 8B| #29 56.4| #20 61.2（排名与 JevBench Score 不同）| #24 64.3（排名与 JevBench Score 不同）| #19 66.3（排名与 JevBench Score 不同）| #19 53.6（排名与 JevBench Score 不同）
Open-Jev 9B| #30 55.0| #32 52.4（排名与 JevBench Score 不同）| #31 59.3（排名与 JevBench Score 不同）| #30 59.5| #35 40.9（排名与 JevBench Score 不同）
system-one| #31 54.8| #17 62.6（排名与 JevBench Score 不同）| #21 65.6（排名与 JevBench Score 不同）| #11 70.6（排名与 JevBench Score 不同）| #20 53.1（排名与 JevBench Score 不同）
jeff| #32 54.4| #31 53.6（排名与 JevBench Score 不同）| #33 48.2（排名与 JevBench Score 不同）| #34 54.5（排名与 JevBench Score 不同）| #13 58.7（排名与 JevBench Score 不同）
Laya| #33 54.4| #29 55.0（排名与 JevBench Score 不同）| #34 47.7（排名与 JevBench Score 不同）| #32 56.8（排名与 JevBench Score 不同）| #12 61.4（排名与 JevBench Score 不同）
Open-Jev 2B| #34 51.3| #33 50.1（排名与 JevBench Score 不同）| #32 54.2（排名与 JevBench Score 不同）| #31 58.4（排名与 JevBench Score 不同）| #37 39.8（排名与 JevBench Score 不同）
OpenDecision| #35 40.6| #35 41.7| #35 35.2| #35 46.0| #28 44.9（排名与 JevBench Score 不同）
openJev Verdict 1.4| #36 38.9| #37 37.4（排名与 JevBench Score 不同）| #38 30.7（排名与 JevBench Score 不同）| #37 40.8（排名与 JevBench Score 不同）| #33 41.6（排名与 JevBench Score 不同）
openJev Verdict| #37 38.1| #36 40.1（排名与 JevBench Score 不同）| #36 33.3（排名与 JevBench Score 不同）| #36 43.3（排名与 JevBench Score 不同）| #29 44.7（排名与 JevBench Score 不同）
kev 0.5B| #38 33.2| #39 35.4（排名与 JevBench Score 不同）| #39 29.4（排名与 JevBench Score 不同）| #38 38.9| #38 38.7
GLiNER2 large| #39 29.6| #38 36.5（排名与 JevBench Score 不同）| #37 31.8（排名与 JevBench Score 不同）| #39 37.8| #36 40.5（排名与 JevBench Score 不同）
smalljev semantic-v9| #40 27.4| #41 26.9（排名与 JevBench Score 不同）| #41 22.6（排名与 JevBench Score 不同）| #41 31.4（排名与 JevBench Score 不同）| #41 27.6（排名与 JevBench Score 不同）
GLiNER2| #41 24.0| #40 30.3（排名与 JevBench Score 不同）| #40 24.6（排名与 JevBench Score 不同）| #40 32.6（排名与 JevBench Score 不同）| #39 34.6（排名与 JevBench Score 不同）
open-jev-deberta-v3-large| #42 23.1| #42 21.9| #42 17.8| #42 23.7| #42 24.9
GLiNER2.5 multi| #43 16.6| #43 16.4| #43 12.6| #43 18.0| #43 19.5
GLiNER2.5 small| #44 13.8| #44 14.4| #44 10.6| #44 16.5| #44 16.9
Mixedbread mxbai-rerank-base-v2| #45 0.8| #45 0.6| #45 0.3| #45 0.9| #45 0.8
BAAI bge-reranker-v2-m3| #46 0.7| #46 0.5| #46 0.3| #46 0.8| #46 0.7
Alibaba GTE Reranker ModernBERT-base| #47 0.3| #47 0.3| #47 0.1| #47 0.4| #47 0.4
Certo v1| #48 0.0| #48 0.0| #48 0.0| #48 0.0| #48 0.0

English / 原文

Jev 1.13.0| #1 74.4| #3 71.8 (rank differs from the JevBench Score)| #2 77.1 (rank differs from the JevBench Score)| #4 76.2 (rank differs from the JevBench Score)| #9 63.1 (rank differs from the JevBench Score)
SemIf| #2 73.1| #2 73.3| #3 75.5 (rank differs from the JevBench Score)| #2 77.3| #4 67.4 (rank differs from the JevBench Score)
djev| #3 73.0| #1 75.8 (rank differs from the JevBench Score)| #1 78.5 (rank differs from the JevBench Score)| #1 81.7 (rank differs from the JevBench Score)| #3 67.9
Winnow-12B Q8| #4 71.2| #4 71.0| #4 75.2| #5 75.3 (rank differs from the JevBench Score)| #10 63.1 (rank differs from the JevBench Score)
reflex 4B| #5 70.3| #6 68.8 (rank differs from the JevBench Score)| #5 73.1| #17 68.4 (rank differs from the JevBench Score)| #5 65.0
jqv| #6 68.6| #14 65.5 (rank differs from the JevBench Score)| #9 70.7 (rank differs from the JevBench Score)| #14 69.0 (rank differs from the JevBench Score)| #14 57.6 (rank differs from the JevBench Score)
decision-machine-1| #7 68.3| #9 67.6 (rank differs from the JevBench Score)| #22 65.4 (rank differs from the JevBench Score)| #3 76.8 (rank differs from the JevBench Score)| #11 61.7 (rank differs from the JevBench Score)
decider-35b-a3b| #8 67.6| #13 66.3 (rank differs from the JevBench Score)| #8 71.3| #10 71.7 (rank differs from the JevBench Score)| #18 56.9 (rank differs from the JevBench Score)
open-alternative-jev| #9 67.0| #7 68.3 (rank differs from the JevBench Score)| #17 66.6 (rank differs from the JevBench Score)| #6 74.0 (rank differs from the JevBench Score)| #8 64.7 (rank differs from the JevBench Score)
system-one-open| #10 66.6| #5 70.3 (rank differs from the JevBench Score)| #11 70.0 (rank differs from the JevBench Score)| #9 72.9 (rank differs from the JevBench Score)| #2 68.0 (rank differs from the JevBench Score)
OpenJev| #11 66.4| #11 66.9| #7 71.6 (rank differs from the JevBench Score)| #8 73.0 (rank differs from the JevBench Score)| #15 57.3 (rank differs from the JevBench Score)
SimpleJev Qwen3.8-27B| #12 66.3| #18 62.0 (rank differs from the JevBench Score)| #10 70.2 (rank differs from the JevBench Score)| #24 65.5 (rank differs from the JevBench Score)| #22 51.8 (rank differs from the JevBench Score)
ZeroEntropy zerank-2| #13 66.0| #15 62.8 (rank differs from the JevBench Score)| #29 62.9 (rank differs from the JevBench Score)| #15 68.8 (rank differs from the JevBench Score)| #16 57.2 (rank differs from the JevBench Score)
GPT-5.6 Luna| #14 65.9| #23 59.5 (rank differs from the JevBench Score)| #6 71.8 (rank differs from the JevBench Score)| #23 66.1 (rank differs from the JevBench Score)| #30 44.3 (rank differs from the JevBench Score)
openjev-sglang| #15 65.3| #19 61.7 (rank differs from the JevBench Score)| #12 69.6 (rank differs from the JevBench Score)| #18 67.4 (rank differs from the JevBench Score)| #24 50.0 (rank differs from the JevBench Score)
Qwen3-Reranker-4B| #16 63.8| #16 62.8| #27 63.3 (rank differs from the JevBench Score)| #16 68.7| #17 56.9 (rank differs from the JevBench Score)
reflex-27b| #17 63.3| #26 57.2 (rank differs from the JevBench Score)| #16 67.2 (rank differs from the JevBench Score)| #28 61.1 (rank differs from the JevBench Score)| #27 45.5 (rank differs from the JevBench Score)
LitJev| #18 62.7| #28 57.0 (rank differs from the JevBench Score)| #19 66.0 (rank differs from the JevBench Score)| #29 60.7 (rank differs from the JevBench Score)| #26 46.1 (rank differs from the JevBench Score)
kev 0.6B| #19 62.5| #12 66.8 (rank differs from the JevBench Score)| #30 60.4 (rank differs from the JevBench Score)| #13 70.2 (rank differs from the JevBench Score)| #1 70.4 (rank differs from the JevBench Score)
SimpleJev Qwen3.6-35B-A3B| #20 62.5| #21 61.0 (rank differs from the JevBench Score)| #14 67.8 (rank differs from the JevBench Score)| #21 66.3 (rank differs from the JevBench Score)| #23 50.6 (rank differs from the JevBench Score)
djev| #21 62.4| #30 54.6 (rank differs from the JevBench Score)| #25 63.9 (rank differs from the JevBench Score)| #27 62.1 (rank differs from the JevBench Score)| #34 41.1 (rank differs from the JevBench Score)
jev-local| #22 61.8| #22 59.6| #26 63.9 (rank differs from the JevBench Score)| #26 63.3 (rank differs from the JevBench Score)| #21 52.5 (rank differs from the JevBench Score)
decider-2b| #23 61.7| #8 67.7 (rank differs from the JevBench Score)| #23 65.1| #7 73.5 (rank differs from the JevBench Score)| #7 64.9 (rank differs from the JevBench Score)
Bespoke Nimble 9B| #24 60.5| #24 58.9| #20 65.9 (rank differs from the JevBench Score)| #22 66.2 (rank differs from the JevBench Score)| #25 47.0 (rank differs from the JevBench Score)
Gemini 3.1 Flash-Lite| #25 60.1| #25 57.6| #15 67.5 (rank differs from the JevBench Score)| #20 66.3 (rank differs from the JevBench Score)| #32 42.8 (rank differs from the JevBench Score)
OpenJev| #26 60.0| #27 57.1 (rank differs from the JevBench Score)| #13 67.9 (rank differs from the JevBench Score)| #25 64.0 (rank differs from the JevBench Score)| #31 42.8 (rank differs from the JevBench Score)
kev 4B| #27 59.7| #10 67.2 (rank differs from the JevBench Score)| #18 66.2 (rank differs from the JevBench Score)| #12 70.5 (rank differs from the JevBench Score)| #6 65.0 (rank differs from the JevBench Score)
DeepSeek V4.1 Flash| #28 57.5| #34 48.4 (rank differs from the JevBench Score)| #28 63.2| #33 56.6 (rank differs from the JevBench Score)| #40 31.7 (rank differs from the JevBench Score)
kev 8B| #29 56.4| #20 61.2 (rank differs from the JevBench Score)| #24 64.3 (rank differs from the JevBench Score)| #19 66.3 (rank differs from the JevBench Score)| #19 53.6 (rank differs from the JevBench Score)
Open-Jev 9B| #30 55.0| #32 52.4 (rank differs from the JevBench Score)| #31 59.3 (rank differs from the JevBench Score)| #30 59.5| #35 40.9 (rank differs from the JevBench Score)
system-one| #31 54.8| #17 62.6 (rank differs from the JevBench Score)| #21 65.6 (rank differs from the JevBench Score)| #11 70.6 (rank differs from the JevBench Score)| #20 53.1 (rank differs from the JevBench Score)
jeff| #32 54.4| #31 53.6 (rank differs from the JevBench Score)| #33 48.2 (rank differs from the JevBench Score)| #34 54.5 (rank differs from the JevBench Score)| #13 58.7 (rank differs from the JevBench Score)
Laya| #33 54.4| #29 55.0 (rank differs from the JevBench Score)| #34 47.7 (rank differs from the JevBench Score)| #32 56.8 (rank differs from the JevBench Score)| #12 61.4 (rank differs from the JevBench Score)
Open-Jev 2B| #34 51.3| #33 50.1 (rank differs from the JevBench Score)| #32 54.2 (rank differs from the JevBench Score)| #31 58.4 (rank differs from the JevBench Score)| #37 39.8 (rank differs from the JevBench Score)
OpenDecision| #35 40.6| #35 41.7| #35 35.2| #35 46.0| #28 44.9 (rank differs from the JevBench Score)
openJev Verdict 1.4| #36 38.9| #37 37.4 (rank differs from the JevBench Score)| #38 30.7 (rank differs from the JevBench Score)| #37 40.8 (rank differs from the JevBench Score)| #33 41.6 (rank differs from the JevBench Score)
openJev Verdict| #37 38.1| #36 40.1 (rank differs from the JevBench Score)| #36 33.3 (rank differs from the JevBench Score)| #36 43.3 (rank differs from the JevBench Score)| #29 44.7 (rank differs from the JevBench Score)
kev 0.5B| #38 33.2| #39 35.4 (rank differs from the JevBench Score)| #39 29.4 (rank differs from the JevBench Score)| #38 38.9| #38 38.7
GLiNER2 large| #39 29.6| #38 36.5 (rank differs from the JevBench Score)| #37 31.8 (rank differs from the JevBench Score)| #39 37.8| #36 40.5 (rank differs from the JevBench Score)
smalljev semantic-v9| #40 27.4| #41 26.9 (rank differs from the JevBench Score)| #41 22.6 (rank differs from the JevBench Score)| #41 31.4 (rank differs from the JevBench Score)| #41 27.6 (rank differs from the JevBench Score)
GLiNER2| #41 24.0| #40 30.3 (rank differs from the JevBench Score)| #40 24.6 (rank differs from the JevBench Score)| #40 32.6 (rank differs from the JevBench Score)| #39 34.6 (rank differs from the JevBench Score)
open-jev-deberta-v3-large| #42 23.1| #42 21.9| #42 17.8| #42 23.7| #42 24.9
GLiNER2.5 multi| #43 16.6| #43 16.4| #43 12.6| #43 18.0| #43 19.5
GLiNER2.5 small| #44 13.8| #44 14.4| #44 10.6| #44 16.5| #44 16.9
Mixedbread mxbai-rerank-base-v2| #45 0.8| #45 0.6| #45 0.3| #45 0.9| #45 0.8
BAAI bge-reranker-v2-m3| #46 0.7| #46 0.5| #46 0.3| #46 0.8| #46 0.7
Alibaba GTE Reranker ModernBERT-base| #47 0.3| #47 0.3| #47 0.1| #47 0.4| #47 0.4
Certo v1| #48 0.0| #48 0.0| #48 0.0| #48 0.0| #48 0.0