Ask an AI toy why the dinosaurs died out and it pauses for a second or so before it answers. Most parents read that pause as a weak model. It is not. What you are hearing is the fixed overhead of the voice chain: the audio has to be heard, judged as finished, turned into text, pushed through a model, turned back into speech, and played out. Nobody skips a step, and the steps add up to the second or two you notice.
Where the second or two actually goes
Start with the broad numbers. When a generative AI toy runs in the cloud, the gap from last word to first reply usually lands between 1,000 and 2,500 milliseconds; the same job handled on-device takes 10 to 50 milliseconds (KEYi Robot, 2026). The gap is that wide because the edge only handles the light work - motion control, preset commands, simple wake decisions. The moment a real conversation starts, the audio still has to leave the device.
The first stage is deciding the user has stopped talking. Most designs use silence detection: OpenAI's realtime voice API waits 500 milliseconds of silence by default before it ends a turn, with a further 300 milliseconds of prefix padding (2026). The system is deliberately giving the speaker half a second to continue. That block is often the single largest piece of the chain, and what it buys is politeness - remove it and the toy starts talking while your child is still breathing.
Then come speech-to-text, the model's first words, the first chunk of synthesised audio, and one network round trip. A conventional transcribe-understand-speak pipeline can take 2 to 5 seconds end to end, while teams that treat audio as a stream pull the whole thing down to roughly 500 milliseconds to 1.5 seconds, holding time-to-first-byte near 500 milliseconds and leaving about 300 milliseconds for capture, voice activity detection, network and playback (2026).
Splitting the work between device and cloud
The design the industry is converging on is a split, not a bet on one side. GENiEX, whose Kickstarter campaign funded in about an hour in September 2026, runs inference in two layers: light chatter stays on a small on-device model, while harder questions escalate to a cloud model, and the user can switch between Instant, Medium and High modes to trade speed for depth. The published specs list a 28-gram body and about eight hours of battery per charge. The practical benefit is that the front-end voice stack does not need retraining every time a new foundation model ships.
Why take latency seriously? Because the ear is less forgiving than most vendors assume. Human turn-taking runs at 200 to 400 milliseconds, and once an AI passes 800 milliseconds the brain reads it as a dropped call (2026). There is a quieter cost too. When Scale AI researchers ran Gemini 3 Pro as a cascade instead of a native audio model, its pass rate on conversations that hinged on tone, background sound or interruption fell 30.3% in relative terms (2026). What gets lost is not only speed but the ability to hear emotion - and children notice that first.
If you are buying or specifying, three questions are worth asking outright: is the latency figure measured from last word to first sound or to the API's first byte, because the two can differ by a factor of two; what still works with the Wi-Fi off; and how interruption is handled - with weak echo cancellation the toy hears itself as the child and talks over its own reply. Voice-module vendors in China publish a baseline of 1.5 to 2 seconds as normal and under 0.5 seconds as good (vendor disclosure, 2026). If a seller cannot say which stage their number covers, treat it as marketing.
© SWPO. All rights reserved.