Speech processing
Voice Activity Detection Benchmark: What to Measure
Set voice-detection onset and offset controls using clipped words, false triggers, and response delay.
A voice assistant needs to hear the first word without waking up for every noise. Pick voice activity detection (VAD) settings by measuring clipped starts, false triggers per hour, and reply delay together. A different microphone or room can change the choice. This is a method, not a BuildAI VAD test.
Choose limits from the interaction
In this example, the team allows at most 2 clipped first words in 100 spoken turns, at most 1 false trigger per hour of ordinary room audio, and a detected speech start within 300 ms for 95% of turns. The first limit reflects how often a user would need to repeat themselves; the second reflects how often an unwanted wake-up is tolerable. These are product choices, not standard VAD targets.
Label audio and test settings
Record held-out sessions on the target microphones with quiet speech, music, keyboard noise, distant voices, interruptions, and silence. Mark where speech begins and ends; agree whether breaths and laughter count. Tune several onset and offset thresholds on development audio, then lock a candidate for the final set. The pyannote VAD pipeline exposes onset and offset controls. Keep the downstream recognizer fixed and count the words it loses after segmentation.
Worked example: an onset sweep
Suppose 100 spoken turns and 10 hours of background audio are used for a first comparison. The table uses hypothetical results. Lowering onset catches more quiet starts but can wake on noise. Offset still needs its own sweep because ending a segment early can cut off the final word.
| Onset setting | Clipped first words | False triggers in 10 h | Starts within 300 ms | Decision |
|---|---|---|---|---|
| Low | 0/100 | 28 (2.8/h) | 99/100 | Too many wakes |
| Middle | 1/100 | 8 (0.8/h) | 97/100 | Passes pilot limits |
| High | 6/100 | 2 (0.2/h) | 90/100 | Clips and delays speech |
Try the middle setting on new rooms and speakers. One clipped word in 100 and eight triggers in 10 hours are uncertain estimates. Add more hours of ordinary audio and quiet speakers before promising a low trigger rate. If a microphone change raises misses, repeat the setting search. If no setting passes, add a wake-word or push-to-talk path instead of hiding the failure in an average detection score.
Record microphone, room, onset and offset settings, labeled speech intervals, clipped words, false triggers per hour, and start delay. Keep audio-level and recognizer-level results side by side.
Download the results sheet. Fill one row for each test item; keep the item-level errors, not just the average.