Lokutor · ESP32-S3, 240 MHz, 8 MB PSRAM, no neural accelerator; measured on one core
Code Model (int8) Model (int4) Demo video
Oído runs NVIDIA's 13M-parameter Conformer-CTC Small speech recognizer entirely on an ESP32-S3, with an int8 engine written for the chip's vector instructions. It transcribes any English sentence, not a list of commands. On LibriSpeech it scores 3.7% / 8.2% word error rate (test-clean / test-other), or 3.3% / 7.2% with the on-chip language model used on this page.
How to read this page. Every clip below was picked at random with a fixed seed, not chosen for looking good, and every error is shown: wrong or extra words in red, missed words struck through. Transcripts come from the released firmware (int8 model + on-chip language model) running on a physical ESP32-S3 board. The laptop build of the same C code gives an identical transcript on 22 of these 27 clips; on the others the laptop's math library rounds the last bit differently from the chip's and a word or two changes, always on the hard (noisy) clips. Compute on the board is the time the chip took, measured with its 64-bit timer, on one of its two cores (RTF is compute time divided by the length of the audio). This v1 demo ran at RTF about 2. The v2 models run faster than real time (RTF 0.54–0.72 on one core; see the README). The same firmware in the emulator produced the same transcripts. Text appears after a 0.8 s pause plus this compute time.
Two LibriSpeech test-clean sentences mixed with real noise recordings (DEMAND), other talkers, and simulated room reverberation, as in the robustness benchmark of the repository.
| Ambience | Audio | Reference | Oído on the ESP32-S3 | WER | Compute on the board |
|---|---|---|---|---|---|
| Quiet room the original recording | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 13.4 s RTF 1.91 1606 M instructions |
| Car, 5 dB SNR real car-interior noise (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 13.3 s RTF 1.91 1603 M instructions |
| Kitchen, 5 dB SNR real kitchen noise (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 13.4 s RTF 1.91 1601 M instructions |
| Living room, 5 dB SNR real living-room noise (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 13.4 s RTF 1.91 1607 M instructions |
| Cafeteria, 5 dB SNR real cafeteria noise with background talk (DEMAND) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 13.4 s RTF 1.92 1607 M instructions |
| Babble, 10 dB SNR four other talkers | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation wood | 6% | 13.5 s RTF 1.93 1612 M instructions |
| Babble, 5 dB SNR four other talkers, louder (hardest case) | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | don't it is such a noble ambition between that it is a pity it has usually such a shallow proination of the italian | 39% | 13.7 s RTF 1.96 1626 M instructions |
| Reverberant room, RT60 0.4 s 5 x 4 x 3 m room, talker 2 m from the microphone | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow foundation | 0% | 13.4 s RTF 1.91 1600 M instructions |
| Very reverberant room, RT60 0.8 s same room, harder walls | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such an open ambition that it is a pity it has usually such a shallow foundation | 11% | 13.4 s RTF 1.91 1599 M instructions |
| Far field + kitchen noise 0.6 s reverberation and kitchen noise at 10 dB | 7.0 s | it is such a noble ambition that it is a pity it has usually such a shallow foundation | it is such a noble ambition that it is a pity it has usually such a shallow combination | 6% | 13.5 s RTF 1.93 1610 M instructions |
| Ambience | Audio | Reference | Oído on the ESP32-S3 | WER | Compute on the board |
|---|---|---|---|---|---|
| Quiet room the original recording | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 21.3 s RTF 2.01 2569 M instructions |
| Car, 5 dB SNR real car-interior noise (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 21.4 s RTF 2.02 2575 M instructions |
| Kitchen, 5 dB SNR real kitchen noise (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 21.3 s RTF 2.01 2571 M instructions |
| Living room, 5 dB SNR real living-room noise (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 21.4 s RTF 2.01 2572 M instructions |
| Cafeteria, 5 dB SNR real cafeteria noise with background talk (DEMAND) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness than some against event | 12% | 21.2 s RTF 2.00 2558 M instructions |
| Babble, 10 dB SNR four other talkers | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling a greater or less remoteness and some member of | 21% | 21.2 s RTF 2.00 2570 M instructions |
| Babble, 5 dB SNR four other talkers, louder (hardest case) | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur by giving a spar or of greater or less remote and some remembered event | 38% | 21.2 s RTF 2.00 2566 M instructions |
| Reverberant room, RT60 0.4 s 5 x 4 x 3 m room, talker 2 m from the microphone | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 21.4 s RTF 2.02 2575 M instructions |
| Very reverberant room, RT60 0.8 s same room, harder walls | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | 0% | 21.4 s RTF 2.02 2574 M instructions |
| Far field + kitchen noise 0.6 s reverberation and kitchen noise at 10 dB | 10.6 s | in actual fact there are doubtless various factors that concur in giving us the feeling of greater or less remoteness in some remembered event | in actual fact there are doubtless various factors that occur in giving us the feeling a greater or less remoteness and some remembered event | 12% | 21.4 s RTF 2.02 2576 M instructions |
Unmodified recordings from public corpora: home microphones (Common Voice), parliament (VoxPopuli) and meetings (AMI).
| Source | Audio | Reference | Oído on the ESP32-S3 | WER | Compute on the board |
|---|---|---|---|---|---|
| Common Voice volunteers' own microphones, at home | 6.2 s | no such authority was given for the court of appeal whose registries remained closed | no such authority was given for the course of peel whose registries remained closed | 14% | 12.0 s RTF 1.93 1425 M instructions |
| Common Voice volunteers' own microphones, at home | 7.7 s | the performance was recorded and later released on a live album titled late orchestration | the performance was recorded and later released on a live album titled late orchestration | 0% | 14.7 s RTF 1.92 1777 M instructions |
| Common Voice volunteers' own microphones, at home | 5.8 s | he has been locked up so long he has been mostly forgotten about | he has been locked up so long he has been mostly forgotten about | 0% | 11.1 s RTF 1.92 1302 M instructions |
| VoxPopuli European Parliament microphones | 3.8 s | the qatari government and the football authorities must take notice | the qatari government and the football authorities must take notice | 0% | 7.4 s RTF 1.96 844 M instructions |
| VoxPopuli European Parliament microphones | 7.9 s | the court still sees scope for further improving the quality of spending by simply applying the current system to its full potential | the court still sees scope for further improving the quality of spending by simply applying the current system to its full potential | 0% | 15.9 s RTF 2.01 1883 M instructions |
| AMI meeting spontaneous meeting speech, headset microphone | 3.6 s | so that indicates that there's an menu under that menu | so that indicates a t an menu under that menu | 30% | 6.9 s RTF 1.94 797 M instructions |
| AMI meeting spontaneous meeting speech, headset microphone | 3.8 s | yeah i think of it in terms of you know not even maybe to well | yeah i think of it in terms of you know not even maybe to what | 7% | 7.3 s RTF 1.91 856 M instructions |
Word error rate (%) over 300 sentences per condition. Oído on the chip, with and without the language model; Whisper tiny.en and Moonshine tiny in full precision on a laptop.
| Ambience | Oído + LM (ESP32-S3) | Oído, no LM (ESP32-S3) | Whisper tiny.en (laptop) | Moonshine tiny (laptop) |
|---|---|---|---|---|
| Quiet room | 3.3 | 3.5 | 5.6 | 4.3 |
| Car, 5 dB SNR | 3.2 | 3.6 | 5.9 | 4.4 |
| Kitchen, 5 dB SNR | 4.2 | 4.7 | 7.5 | 6.1 |
| Living room, 5 dB SNR | 4.8 | 5.7 | 7.9 | 7.0 |
| Cafeteria, 5 dB SNR | 8.6 | 9.5 | 12.3 | 12.2 |
| Babble, 10 dB SNR | 9.7 | 10.9 | 14.5 | 14.1 |
| Babble, 5 dB SNR | 24.7 | 27.6 | 35.9 | 37.3 |
| Reverberant room, RT60 0.4 s | 4.9 | 5.8 | 10.2 | 9.8 |
| Very reverberant room, RT60 0.8 s | 14.1 | 16.0 | 22.5 | 28.6 |
| Far field + kitchen noise | 11.4 | 13.5 | 19.8 | 24.3 |
Model: Oído, adapted from NVIDIA stt_en_conformer_ctc_small (CC-BY-4.0); the Oído model files are CC BY-NC-SA 4.0 with Lokutor's terms (see models/LICENSE-WEIGHTS). Speech: LibriSpeech (CC-BY-4.0), Common Voice 17 (CC0), VoxPopuli (CC0), AMI Meeting Corpus (CC-BY-4.0). Noise: DEMAND (CC-BY-4.0). Reverberation: pyroomacoustics. Page generated by eval/make_demo_page.py.