Add audio variants to user input protocols (#33923)

## What changed

- Add remote and local audio variants to core and app-server user inputs, including generated JSON and TypeScript schemas.
- Preserve audio fields when converting app-server inputs and response content into thread items.
- Keep unsupported audio out of model requests, TUI rendering, text extraction, and truncated tool output, using explicit text placeholders where audio is submitted as a model input.

## Testing

- Cover app-server media conversion, unsupported-input placeholders, TUI omission, and tool-output truncation.

GitOrigin-RevId: 16152bf3af674575ce569764a8b472e032e7569d
This commit is contained in:
nhamidi-oai authored and copyberry committed 2026-07-17 23:14:37 +00:00
1 parent e98765c212
commit 97dbd01a3d
49 files changed
+1330 -33

No files matched your search

+9
View File
@@ -202,6 +202,9 @@ fn response_input_to_code_mode_result(response: ResponseInputItem) -> JsonValue
detail: detail.or(Some(DEFAULT_IMAGE_DETAIL)),
}
}
codex_protocol::models::ContentItem::InputAudio { audio_url } => {
FunctionCallOutputContentItem::InputAudio { audio_url }
}
})
.collect::<Vec<_>>(),
),
@@ -233,8 +236,14 @@ fn content_items_to_code_mode_result(items: &[FunctionCallOutputContentItem]) ->
{
Some(image_url.clone())
}
FunctionCallOutputContentItem::InputAudio { audio_url }
if !audio_url.trim().is_empty() =>
{
Some(audio_url.clone())
}
FunctionCallOutputContentItem::InputText { .. }
| FunctionCallOutputContentItem::InputImage { .. }
| FunctionCallOutputContentItem::InputAudio { .. }
| FunctionCallOutputContentItem::EncryptedContent { .. } => None,
})
.collect::<Vec<_>>()