Send processed microphone audio over RTP in voice-host (#43090)

## Why

`voice-host` previously drained captured audio locally without sending it to the peer. Connect capture to the outgoing media track while preserving mute boundaries and limiting stale audio.

## What changed

- Resample capture and render references to 48 kHz with Rubato, apply Sonora echo cancellation, noise suppression, and gain control, and encode capture as 20 ms mono Opus packets.
- Attach an Opus track to the WebRTC peer and advance RTP timestamps across mute gaps without introducing sequence-number gaps.
- Prioritize queued controls and shutdown between sends. Clear pending audio and processing history on microphone mute transitions, reject delayed pre-unmute buffers, and reset partial history after capture gaps.
- Size bounded capture and render queues for device rates, validate callback timing, enforce a 500 ms processing deadline, and time out stalled sends after 100 ms.

Receive/decode and TUI integration remain unimplemented. The bundled Opus encoder build requires CMake and a C compiler.

## Testing

Add synthetic-audio tests for Opus decoding at 44.1 and 48 kHz input rates, echo reduction, capture gaps, stale audio, queue capacity, and control priority. A loopback WebRTC test verifies RTP delivery, decoded signal, mute isolation, and timestamp and sequence continuity across mute gaps.

GitOrigin-RevId: 38b8438577cfadb8124f8aeb426dffbab4845964
This commit is contained in:
Benjamin Carlsson authored and copyberry committed 2026-09-05 20:02:55 +00:00
1 parent 0683368584
commit 7dc7c7a756
20 files changed
+1526 -67

No files matched your search

+4
View File
@@ -27,6 +27,7 @@ use tokio::sync::Notify;
use tokio::sync::mpsc;
use tokio::time::timeout;
use webrtc::data_channel::DataChannel;
use webrtc::peer_connection::MediaEngine;
use webrtc::peer_connection::PeerConnection;
use webrtc::peer_connection::PeerConnectionBuilder;
use webrtc::peer_connection::PeerConnectionEventHandler;
@@ -307,7 +308,10 @@ async fn installed_client_negotiates_and_closes_over_udp_and_tcp() -> Result<()>
let gathered = Arc::new(Notify::new());
let mut settings = SettingEngine::default();
settings.set_lite(/*lite*/ true);
let mut media = MediaEngine::default();
media.register_default_codecs()?;
let builder = PeerConnectionBuilder::new()
.with_media_engine(media)
.with_setting_engine(settings)
.with_handler(Arc::new(RemoteEvents {
channels: sender,