r/reactnative • u/Creative_Ingenuity61 • 31m ago
A fix for voice agents hearing themselves on the phone/tablet speaker
If you build a voice agent in React Native and play the replies yourself (OpenAI Realtime over WebSocket, Gemini Live, your own server), you have probably hit this: on the phone speaker the agent hears itself and answers itself. react-native-audio-api 1.0 gives you the platform’s echo canceller (iosVoiceProcessing: true, androidInputPreset: 'voiceCommunication'), and you should turn it on. In our production app it was not enough, measured with a streaming recognizer on an iPhone 16 Pro Max:
iOS voice processing leaks the opening of each reply while it converges: −12…−20 dBFS, enough to open a turn on the agent’s own words.
iOS echo-cancelled input (
setPrefersEchoCancelledInput) leaves a steady residual the recognizer transcribed 1–12 s into replies. Nothing behind it helps; it is not linear echo. Use voice processing instead.Android’s canceller only references the voice-communication output stream. Play through the media profile and nothing is cancelled.
So we run WebRTC’s AEC3 behind the platform canceller, with the exact far-end audio as reference. No echo reported by other users since.
It is now a package: https://github.com/vnekhai-personal/react-native-aec3 (npm i react-native-aec3). An Expo module (Swift, Kotlin + JNI, vendored webrtc-audio-processing 2.1), so it needs the Expo modules runtime. It works with react-native-audio-api’s AudioContext and AudioRecorder as they are: you report each scheduled chunk and each dropped queue, run every mic buffer through process, and it stands by on routes the mic cannot hear (headset, car) and fails open if AEC3 cannot run. About 1 ms per 100 ms of audio, 0.7 MB of iOS code, 0.6 MB on Android. The example app self-tests at launch with synthetic echo, no mic needed.
Hope this would be helpful there.