Send processed microphone audio over RTP in voice-host (#43090)

## Why

`voice-host` previously drained captured audio locally without sending it to the peer. Connect capture to the outgoing media track while preserving mute boundaries and limiting stale audio.

## What changed

- Resample capture and render references to 48 kHz with Rubato, apply Sonora echo cancellation, noise suppression, and gain control, and encode capture as 20 ms mono Opus packets.
- Attach an Opus track to the WebRTC peer and advance RTP timestamps across mute gaps without introducing sequence-number gaps.
- Prioritize queued controls and shutdown between sends. Clear pending audio and processing history on microphone mute transitions, reject delayed pre-unmute buffers, and reset partial history after capture gaps.
- Size bounded capture and render queues for device rates, validate callback timing, enforce a 500 ms processing deadline, and time out stalled sends after 100 ms.

Receive/decode and TUI integration remain unimplemented. The bundled Opus encoder build requires CMake and a C compiler.

## Testing

Add synthetic-audio tests for Opus decoding at 44.1 and 48 kHz input rates, echo reduction, capture gaps, stale audio, queue capacity, and control priority. A loopback WebRTC test verifies RTP delivery, decoded signal, mute isolation, and timestamp and sequence continuity across mute gaps.

GitOrigin-RevId: 38b8438577cfadb8124f8aeb426dffbab4845964
This commit is contained in:
Benjamin Carlsson authored and copyberry committed 2026-09-05 20:02:55 +00:00
1 parent 0683368584
commit 7dc7c7a756
20 files changed
+1526 -67

No files matched your search

+8 -2
View File
@@ -44,8 +44,14 @@ stale at one second. The queue's sample capacity must span more than that servic
interval.
Processing lag can still overflow a queue. Before devices open, the worker blocks
on commands instead of polling.
`devicesOpened` confirms device opening only: capture/reference are drained locally
until the following media stage connects encoding, decoding and the peer.
`devicesOpened` confirms device opening only. Capture now uses Rubato resampling,
Sonora echo/noise/gain processing and 20 ms Opus encoding before sending RTP.
Mute resets retained capture history and rejects delayed pre-unmute buffers.
The receive/decode pipeline and TUI connection remain subsequent stages.
The capture encoder uses `opus 0.4.0` and its bundled `opusic-sys` build, which
requires CMake and a C compiler. This is separate from the runtime's decoder Opus
copy; final symbol binding and packaging validation must cover both copies.
Cargo Linux builds require ALSA development inputs discoverable by pkg-config
(for example `libasound2-dev` on Debian/Ubuntu). Bazel uses the declared `alsa_lib`