Add audio output support to dynamic tools and code mode (#34080)

## What changed

- Add `inputAudio` content items to dynamic tool responses, app-server events, thread history, and generated protocol schemas.
- Add an `audio()` code-mode helper that accepts inline data URLs, audio URL objects, and MCP audio blocks.
- Convert MCP audio blocks into model input when audio is supported, and replace unsupported audio with an explanatory text item.
- Reject non-data audio URLs and track audio item counts in dynamic tool analytics.

## Testing

- Cover audio serialization, protocol round trips, thread-history conversion, MCP modality filtering, code-mode helper inputs, and invalid URL handling.

GitOrigin-RevId: 1ed52a8f9c62d4840fb71c5ec736b4a3566243d6
This commit is contained in:
nhamidi-oai authored and copyberry committed 2026-07-18 23:22:13 +00:00
1 parent 312caf176a
commit 643de86a19
46 files changed
+987 -72

No files matched your search

@@ -56,5 +56,8 @@ pub(super) fn output_item(item: FunctionCallOutputContentItem) -> CellOutputItem
ImageDetail::Original => CellImageDetail::Original,
}),
},
FunctionCallOutputContentItem::InputAudio { audio_url } => {
CellOutputItem::Audio { audio_url }
}
}
}