The goal we set was simple to say and hard to get right: it listens and waits, and when you stop speaking, it answers. On top of that, you should be able to cut it off at any moment, and you should be able to pause and think out loud without it jumping in.
At Balay ni Bruno & Co. we build these systems for our own work first, and a real person runs them every day. That is how we found out what was actually broken, instead of guessing.
The Real Reasons It Felt Slow
When we measured it, the delay was not where we expected. Two problems did most of the damage.
1. It never noticed we had stopped talking
The microphone was a basic webcam mic. When we checked, it marked about 5% of moments of pure silence as speech. The old rule waited for one full second with zero speech before answering, so a single false blip reset the clock. Turns ran on for seconds after we had finished.
2. The answer waited in line
Everything ran through one single loop. The assistant would say a quick "On it", and the real answer queued up behind that. The microphone was also switched off while it spoke, so it could not hear us trying to stop it.
Before
- Needed one second of perfect silence to end a turn
- One false blip from the mic restarted the wait
- Microphone off while the assistant spoke
- The answer waited behind a filler reply
- First word about 8 seconds after we stopped
After
- Sound counts as speech only if it is speech-like and loud enough
- A turn ends when the last second is almost all quiet
- Microphone always listening, even mid-answer
- Each sentence is spoken while the next is being written
- First word 3.5 to 5.5 seconds after we stop
What We Changed
The fix was not one setting. It was a different way for the assistant to listen, think and speak at the same time.
A sound only counts as talking if it is speech-like and above a loudness floor. The turn ends when the last second or so is almost all quiet, with room for a couple of stray blips. Starting a turn needs several loud speech moments in a row, which also stopped it from hearing phantom "Thank you" turns in silence.
The assistant now speaks each sentence while the next one is still being written, instead of waiting for the full answer first.
The microphone never switches off. If you start talking while it is answering, it holds between sentences. Real words mean it drops its answer and takes in your whole thought. A cough means it carries on.
Everything loads when the assistant opens, not on your first question. The voice alone took about 4 to 5 seconds on its first use when cold, and about 0.6 seconds once warm.
Four Ways to Cut It Off
A good assistant has to be easy to stop. Ours stops an answer in about one second, without forgetting the conversation, in any of these ways:
- Tap the assistant on screen.
- Press one key on the keyboard.
- Say "cancel".
- Simply talk over it.
And for thinking out loud, saying "hold on" or "let me think" tells it to keep collecting everything you say until you say "go".
Use a headset if you can. To stop it by talking over it, the assistant has to tell your voice apart from its own. On open speakers, its own voice leaks back into the microphone. In our tests that echo usually sat around a tenth of its own level, but spiked to three quarters. A headset keeps that echo out and makes talking over it far more reliable.
One more small guard made a big difference: for half a second after the assistant goes quiet, it ignores new sound. Without that, its own chime or the tail of its own voice would start a fake turn.
Where the Seconds Go Now
Of that time, about one second is the assistant making sure you really finished, a tenth of a second turns your words into text, 1.6 to 2.6 seconds is the AI starting its reply, and about 0.6 seconds is the first sentence being spoken.
Common Questions
Why does my AI voice assistant take so long to answer after I stop talking?
Often it has not noticed you stopped. A basic microphone can mistake a small share of pure silence for speech, and if the assistant waits for a full second of perfect quiet, one false blip restarts the wait. In our case the fix was to count a sound as speech only when it is both speech-like and loud enough, and to end the turn when the last second is almost all quiet.
How do I stop a voice assistant from talking over me?
Keep the microphone listening while the assistant speaks. When you start talking, it pauses between sentences. If you said real words, it drops its answer and listens to your whole thought. If it was only a cough, it carries on. It also ignores new sound for half a second after it goes quiet, so its own voice does not start a fake turn.
Does a headset make a voice assistant work better?
Yes, especially for cutting it off by talking over it. On open speakers, the assistant's own voice leaks back into the microphone, and in our tests that echo sometimes reached three quarters of the assistant's own level. A headset keeps that echo out, so the assistant can tell your voice from its own far more reliably.
Key Takeaways
- Late replies were mostly the assistant not noticing we had stopped, because a basic mic mistook some silence for speech.
- Counting only loud, speech-like sound fixed the end of each turn.
- Speaking each sentence while the next is written, and keeping the mic on, made it feel like a real conversation.
- You can stop it by tapping, one key, saying "cancel", or talking over it, and a headset makes the last one far more reliable.
- The first word now comes 3.5 to 5.5 seconds after you stop, down from about 8.