🔶 面向语音 Agent 的 Jev

这是 VAmoS Pro Bench——我们的语音 Agent 基准测试（发布文章见此）的扩展测试。我们选取了其中 14 个 Agent 之一的 Pipecat，让 Jev 决定来电者何时说完话，并以相同的 100 项任务重新运行测试，每项任务运行 3 次。此前，该 Agent 会在来电者 52% 的发言轮次中抢话；接入 Jev 后，这一比例降至 11%。

图表：VAmoS Pro Bench 中 Agent 在来电者发言期间开始说话的轮次占比

数据范围为 0%、20%、40%、60%。

在 VAmoS Pro Bench 中，Agent 在来电者发言期间开始说话的轮次占比：每个 Agent 均使用相同的 100 项任务，每项运行 3 次；每种噪声条件下有 75 通电话。所有条件下的降幅均具有统计显著性（均为 p < 10−7）。

如果你构建的是文本 Agent，你从来不必判断用户是否已经说完话：用户会按下 Enter。语音 Agent 没有 Enter 键。它接收到的是连续的音频流，并且在任何时刻都必须决定自己是否应该说话。决定得太早，它就会打断用户；决定得太晚，它就会让用户陷入沉默等待。

这个决策被称为轮次检测（turn detection），而大多数 Agent 开发者此前从未需要认真考虑它。这也是 Jev 的适用场景之一——这是一款来自 TypeSafe、近期受到广泛讨论的快速分类模型。

为什么轮次检测很难

大多数语音 Agent 通过计时器作出判断。一种称为语音活动检测（voice activity detection）的小型声学模型会标记是否有人正在说话。当在设定的一段时间内没有人说话时，该轮次便结束，Agent 开始回答。

计时器只能听到静音。来电者可能会在说话中间停顿：例如分段念出账号（“四四七……二一九”），或者说“等等，我找一下账单”。计时器会判定轮次结束，Agent 在半句话时开始回答，来电者则继续说话，于是双方开始相互抢话。在电话通话中，这会进一步演变为重复、号码听错，以及来电者不得不从头再说一遍。

背景噪声会让问题更加严重。房间里的电视声音同样是语音，因此语音识别器会把它记录下来，仿佛是来电者说出的内容。

可直接接入的语义轮次检测

语义轮次检测不仅关注来电者是否仍在发声，也会理解其说话内容。“我的账号是四四七”是一个尚未完成的表达；“就这些了，谢谢”则是一个完整的表达。多种语音框架已经提供了相应版本的功能。

Jev 让这一能力可以直接接入。它是一个快速、通用的分类器：向它提供一些文本和一个问题，它会在大约 150 ms 内返回一个标签。对于轮次检测，输入文本是来电者截至目前说出的内容，加上 Agent 的上一句话；标签则为 complete（已完成）、short（在短语中间被截断）和 long（因寻找账单等更长时间的事情而暂停）。

它适用于你的 Agent 吗？

Jev 是否有帮助，取决于它周围的一切：语音识别器、来电者所处环境的噪声、他们停顿的方式，以及 Agent 会将一个轮次保持开放多长时间。找出答案的方式，与选择模型或架构的方式相同：进行基准测试。

我们选取了 VAmoS Pro Bench 中的 Pipecat Agent，在相同的 100 项任务上分别运行两次，每项运行 3 次，并使用相同的模拟来电者和相同的背景噪声组合：一次使用普通计时器，一次由 Jev 作出判断。唯一的其他差异是 Pipecat 版本：使用 Jev 的版本为 1.12，未使用 Jev 的版本为 1.8。该基准测试会根据 Agent 在其背后的账单和贷款系统中实际作出的变更，对每通电话进行评分。

发生了什么变化，以及没有发生什么变化

• 打断大幅减少： 来电者发言轮次中的抢话比例从 52% 降至 11%，所有噪声条件下都是如此。
• 任务完成情况相同： 未使用 Jev 时，300 通电话中有 50 通通过；使用 Jev 后有 53 通通过。通话变得更平静，但并没有更准确。
• 回复更慢： 中位回复时间从 2.1 秒增至 4.6 秒，主要原因是轮次被保持开放的时间更长。
• 电视噪声仍未解决： 语音识别器仍会将电视语音记录为来电者的话，而 Jev 往往会等待这些语音结束。

这项测试并未证明什么

我们测试的是一个 Agent 的一种配置，并使用了我们自行设定的等待时长。要对轮次检测器进行公平的基准测试，应让所有检测器在相同的通话条件下运行，因此这些结果并不能说明 Jev 的总体表现。换用不同的 Agent，结果可能会有所不同。

在你自己的通话中运行测试

我们从 600 通模拟电话中获得了这些答案。Veris 可以围绕你的来电者、策略和噪声构建同类模拟环境，让你在客户实际听到之前，了解一个新组件会如何影响你的通话。

在来电者测试你之前先测试它

新的轮次检测器、模型或提示词，都可能改变每一通电话的进展方式。Veris 会让它与模拟来电者及你的系统副本进行交互，并对 Agent 实际做出的操作进行评分。

预约演示

This is an extension of VAmoS Pro Bench, our voice-agent benchmark (here is the launch post). We took one of its fourteen agents, the Pipecat one, let Jev decide when the caller had finished talking, and reran the same 100 tasks, three runs each. The agent cut its callers off on 52% of their turns before, and 11% after.

0% 20% 40% 60% 31% 8% Clean 28% 9% Street 33% 8% Café 64% 14% TV 52% 11% All calls Pipecat, 0.8 s silence ends the turn Pipecat + Jev turn gate

Share of the caller's turns that the agent started speaking over, on VAmoS Pro Bench: the same 100 tasks × 3 runs per agent, 75 calls per noise condition. The drop is significant in every condition (all p < 10−7).

If you build agents that work in text, you have never had to decide when the user has finished talking. They press Enter. A voice agent gets no Enter key. It hears a continuous stream of audio, and at any moment it has to decide whether it should talk. Decide too early and it talks over people. Decide too late and it leaves them in silence.

That decision is called turn detection, and most agent developers have never had to think about it. It is also a good job for Jev, the fast classifier model from TypeSafe that people have been talking about.

Why turn detection is hard

Most voice agents decide with a timer. A small acoustic model, called voice activity detection, marks whether anyone is speaking. When nobody has spoken for a set stretch, the turn is over and the agent answers.

A timer only hears silence. Callers pause in the middle of things: reading an account number in chunks ("four four seven… two one nine"), or saying "hang on, let me find the bill". The timer ends the turn, the agent answers half a sentence, the caller carries on, and now the two are talking over each other. On a phone call that snowballs into repeats, misheard numbers, and a caller who has to start again.

Background noise makes it worse. A television in the room is speech too, so the speech recognizer writes it down as if the caller said it.

Semantic turn detection, as a drop-in

Semantic turn detection looks at what the caller said as well as whether they are still making sound. "My account number is four four seven" is an unfinished thought; "that's everything, thanks" is a finished one. Several voice frameworks already ship a version of it.

Jev makes it a drop-in. It is a fast, general classifier: give it some text and a question, and in about 150 ms it returns a label. For turn detection, the text is the caller's words so far plus the agent's last line, and the labels are complete , short (cut off mid-phrase), and long (paused for something longer, like finding a bill).

Does it work for your agent?

Whether Jev helps depends on everything around it: the speech recognizer, the noise your callers have, how they pause, and how long the agent will hold a turn open. You find out the same way you choose a model or an architecture: with a benchmark.

We took the Pipecat agent from VAmoS Pro Bench and ran it twice through the same 100 tasks, three runs each, with the same simulated callers and the same mix of background noise: once with the plain timer, and once with Jev deciding. The only other difference was the Pipecat release: 1.12 with Jev, 1.8 without. The benchmark grades each call on what the agent changed in the billing and loan systems behind it.

What changed, and what did not

• Far fewer interruptions: from 52% of caller turns to 11%, in every noise condition.
• The same task completion: 50 of 300 calls passed without Jev and 53 with it. The calls got calmer, but no more correct.
• Slower replies: the median reply went from 2.1 s to 4.6 s, mostly from turns held open.
• TV is still unsolved: the speech recognizer still writes down TV speech as the caller's, and Jev often waits for it to finish.

What this does not show

We tested one agent in one configuration, with hold times we chose. A fair benchmark of turn detectors would run all of them through the same calls, so these results say nothing about Jev in general. A different agent could land somewhere else.

Run it on your own calls

We got these answers from 600 simulated calls. Veris builds the same kind of simulation around your callers, policies, and noise, so you can see what a new component does to your calls before your customers hear it.

Test it before your callers do

A new turn detector, model, or prompt can change how every call goes. Veris runs it against simulated callers and twins of your systems, and grades what the agent did.

Book a demo