What is a time budget?
In a text chat, a three-second wait is fine. On a phone call, a silence longer than about a second feels like the line has dropped. So a voice product has a time budget, and every step must fit inside its share.
Engineers write the budget down step by step: waiting for silence, speech to text, the model's first words, the first sound. If the shares add up to more than the total, something has to get faster or be cut.
The main trick is streaming. No step waits for the one before to finish. The speech model starts talking as soon as the first sentence exists, while the rest is still being written.
Why does a time budget matter?
A reply passes through several stages, and each one adds delay. Without a target, every stage is a little slow and the total is unbearable. Deciding the acceptable total first, then giving each stage a share, turns speed from a hope into something that is designed and measured.
How does a time budget work?
- Decide the longest acceptable wait for the person, such as one second to the first sound.
- List every stage between their request and the first thing they see or hear.
- Give each stage a share of the total.
- Measure each stage on every request.
- When a stage goes over its share, that stage is where the work is.
When should you use a time budget?
Use it when
- People wait for the reply in real time, especially by voice
- A reply passes through several models or tools
- Speed complaints are vague and you need to know where the time goes
Think twice when
- Work runs in the background and nobody is waiting
Where does a time budget sit in a real system?
The budget is a design decision recorded with the system. The measurements come from the harness, which times each stage and stores the result with every request.
What goes wrong with a time budget?
- Measuring only the average. The slowest one in twenty is what people remember.
- Optimising a stage that was already inside its share.
- Counting total time when the person only cares about the first word.
What does a time budget look like in practice?
A voice assistant has one second from the caller's last word to its first sound. The shares: 0.3 seconds to be sure the caller has finished, 0.1 for speech to text, 0.4 for the first sentence, 0.2 for the first sound. Each is measured on every turn.