Lokutor · ESP32-S3, 240 MHz dual-core, 8 MB PSRAM, no neural accelerator
Code Model (int8) Model (int4) Demo video
Oído runs NVIDIA's 13M-parameter Conformer-CTC Small speech recognizer entirely on an ESP32-S3, with an int8 engine written for the chip's vector instructions. It transcribes any English sentence, not a list of commands. On LibriSpeech it scores 3.7% / 8.2% word error rate (test-clean / test-other), or 3.3% / 7.2% with the on-chip language model used on this page.
How to read this page. Every clip below was picked at random with a fixed seed, not chosen for looking good, and every error is shown: wrong or extra words in red, missed words struck through. Transcripts come from the released firmware (int8 model + on-chip language model) running in Espressif's ESP32-S3 emulator. The laptop build of the same C code gives an identical transcript on 22 of these 27 clips; on the others the chip's math library rounds the last bit differently from the laptop's and a word or two changes, always on the hard (noisy) clips. On-chip compute is the exact number of instructions the firmware executed, converted to time at 1.3–1.6 cycles per instruction on 240 MHz plus an allowance for memory stalls. These are estimates until the board measurements are published. Text appears after a 0.8 s pause plus this compute time.
Two LibriSpeech test-clean sentences mixed with real noise recordings (DEMAND), other talkers, and simulated room reverberation, as in the robustness benchmark of the repository.
| Ambience | Audio | Reference | Oído on the ESP32-S3 | WER | On-chip compute (est.) |
|---|---|---|---|---|---|
| Quiet room the original recording | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 5.3–6.4 s RTF 0.76–0.92 1606 M instructions |
| Car, 5 dB SNR real car-interior noise (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 5.3–6.4 s RTF 0.76–0.92 1603 M instructions |
| Kitchen, 5 dB SNR real kitchen noise (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 5.3–6.4 s RTF 0.76–0.92 1601 M instructions |
| Living room, 5 dB SNR real living-room noise (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 5.3–6.5 s RTF 0.76–0.92 1607 M instructions |
| Cafeteria, 5 dB SNR real cafeteria noise with background talk (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 5.3–6.5 s RTF 0.76–0.92 1607 M instructions |
| Babble, 10 dB SNR four other talkers | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation would | 6% | 5.4–6.5 s RTF 0.77–0.93 1612 M instructions |
| Babble, 5 dB SNR four other talkers, louder (hardest case) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | don't it is such a noble ambition between that it is a pity it has usually such a shallow proination of the italian | 39% | 5.4–6.5 s RTF 0.77–0.93 1626 M instructions |
| Reverberant room, RT60 0.4 s 5 x 4 x 3 m room, talker 2 m from the microphone | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 5.3–6.4 s RTF 0.76–0.92 1600 M instructions |
| Very reverberant room, RT60 0.8 s same room, harder walls | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such an open ambition that it is a pity it has usually such a shallow foundation | 11% | 5.3–6.4 s RTF 0.76–0.92 1599 M instructions |
| Far field + kitchen noise 0.6 s reverberation and kitchen noise at 10 dB | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow combination | 6% | 5.4–6.5 s RTF 0.77–0.92 1610 M instructions |
| Ambience | Audio | Reference | Oído on the ESP32-S3 | WER | On-chip compute (est.) |
|---|---|---|---|---|---|
| Quiet room the original recording | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 8.5–10.3 s RTF 0.80–0.97 2569 M instructions |
| Car, 5 dB SNR real car-interior noise (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 8.5–10.3 s RTF 0.80–0.97 2575 M instructions |
| Kitchen, 5 dB SNR real kitchen noise (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 8.5–10.3 s RTF 0.80–0.97 2571 M instructions |
| Living room, 5 dB SNR real living-room noise (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 8.5–10.3 s RTF 0.80–0.97 2572 M instructions |
| Cafeteria, 5 dB SNR real cafeteria noise with background talk (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness and some against event | 12% | 8.5–10.2 s RTF 0.80–0.96 2558 M instructions |
| Babble, 10 dB SNR four other talkers | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling a greater or less remoteness and some member of | 21% | 8.5–10.3 s RTF 0.80–0.97 2570 M instructions |
| Babble, 5 dB SNR four other talkers, louder (hardest case) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur by giving a spar or of greater or less remote and some remembered event | 38% | 8.5–10.3 s RTF 0.80–0.97 2566 M instructions |
| Reverberant room, RT60 0.4 s 5 x 4 x 3 m room, talker 2 m from the microphone | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 8.5–10.3 s RTF 0.80–0.97 2575 M instructions |
| Very reverberant room, RT60 0.8 s same room, harder walls | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 8.5–10.3 s RTF 0.80–0.97 2574 M instructions |
| Far field + kitchen noise 0.6 s reverberation and kitchen noise at 10 dB | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that occur in giving us the feeling a greater or less remoteness and some remembered event | 12% | 8.5–10.3 s RTF 0.80–0.97 2576 M instructions |
Unmodified recordings from public corpora: home microphones (Common Voice), parliament (VoxPopuli) and meetings (AMI).
| Source | Audio | Reference | Oído on the ESP32-S3 | WER | On-chip compute (est.) |
|---|---|---|---|---|---|
| Common Voice volunteers' own microphones, at home | 6.2 s | no such authority was given for the court of appeal whose registries remained closed | no such authority was given for the course of peel whose registries remained closed | 14% | 4.7–5.7 s RTF 0.76–0.92 1425 M instructions |
| Common Voice volunteers' own microphones, at home | 7.7 s | the performance was recorded and later released on a live album titled late orchestration | the performance was recorded and later released on a live album titled late orchestration | 0% | 5.9–7.1 s RTF 0.77–0.93 1777 M instructions |
| Common Voice volunteers' own microphones, at home | 5.8 s | he has been locked up so long he has been mostly forgotten about | he has been locked up so long he has been mostly forgotten about | 0% | 4.3–5.2 s RTF 0.75–0.91 1302 M instructions |
| VoxPopuli European Parliament microphones | 3.8 s | the qatari government and the football authorities must take notice | the qari government and the football authorities must take notice | 10% | 2.8–3.4 s RTF 0.75–0.90 844 M instructions |
| VoxPopuli European Parliament microphones | 7.9 s | the court still sees scope for further improving the quality of spending by simply applying the current system to its full potential | the court still sees scope for further improving the quality of spending by simply applying the current system to its full potential | 0% | 6.2–7.5 s RTF 0.79–0.95 1883 M instructions |
| AMI meeting spontaneous meeting speech, headset microphone | 3.6 s | so that indicates that there's an menu under that menu | so that indicates a there's an menu under that menu | 30% | 2.7–3.2 s RTF 0.74–0.90 797 M instructions |
| AMI meeting spontaneous meeting speech, headset microphone | 3.8 s | yeah i think of it in terms of you know not even maybe to well | yeah i think of it in terms of you know not even maybe to what | 7% | 2.9–3.4 s RTF 0.74–0.90 856 M instructions |
Word error rate (%) over 300 sentences per condition. Oído on the chip, with and without the language model; Whisper tiny.en and Moonshine tiny in full precision on a laptop.
| Ambience | Oído + LM (ESP32-S3) | Oído, no LM (ESP32-S3) | Whisper tiny.en (laptop) | Moonshine tiny (laptop) |
|---|---|---|---|---|
| Quiet room | 3.3 | 3.5 | 5.6 | 4.3 |
| Car, 5 dB SNR | 3.2 | 3.6 | 5.9 | 4.4 |
| Kitchen, 5 dB SNR | 4.2 | 4.7 | 7.5 | 6.1 |
| Living room, 5 dB SNR | 4.8 | 5.7 | 7.9 | 7.0 |
| Cafeteria, 5 dB SNR | 8.6 | 9.5 | 12.3 | 12.2 |
| Babble, 10 dB SNR | 9.7 | 10.9 | 14.5 | 14.1 |
| Babble, 5 dB SNR | 24.7 | 27.6 | 35.9 | 37.3 |
| Reverberant room, RT60 0.4 s | 4.9 | 5.8 | 10.2 | 9.8 |
| Very reverberant room, RT60 0.8 s | 14.1 | 16.0 | 22.5 | 28.6 |
| Far field + kitchen noise | 11.4 | 13.5 | 19.8 | 24.3 |
Model: NVIDIA stt_en_conformer_ctc_small (CC-BY-4.0). Speech: LibriSpeech (CC-BY-4.0), Common Voice 17 (CC0), VoxPopuli (CC0), AMI Meeting Corpus (CC-BY-4.0). Noise: DEMAND (CC-BY-4.0). Reverberation: pyroomacoustics. Page generated by eval/make_demo_page.py.