nhamidi-oai 23899f7cb6 Forward audio inputs to the Responses API (#33932) ...

## Why

Audio variants were available in the user-input protocol but were replaced with unsupported-input placeholders instead of being sent to the model.

## What changed

- Serialize audio data URLs as `input_audio` content and convert local `wav`, `mp3`, `m4a`, `webm`, and `ogg` files to labeled data URLs.
- Validate and canonicalize base64 audio before requests, enforce the 50 MiB input limit, and replace invalid, unsupported, or oversized audio with explanatory text.
- Preserve audio attachments when mapping response items back to user-message events and document the app-server input variants.

## Testing

- Add unit coverage for local-file conversion, data URL validation, size and format errors, event mapping, and attachment extraction.
- Add client tests that verify data URL and local audio payloads sent to the Responses API.

GitOrigin-RevId: f72cd6b6e43ab12757eeb47621b1d670594ac7d9

2026-07-18 00:26:24 +00:00

..

prompts/base_instructions

chore(core) rm AskForApproval::OnFailure (#28418)

2026-06-23 12:13:54 -07:00

account.rs

Support custom transports for Amazon Bedrock (#33695)

2026-07-16 21:41:00 +00:00

agent_path.rs

chore: morpheus to path (#18353)

2026-04-20 10:32:20 +01:00

approvals.rs

Support openai/form extended form elicitations (#27500)

2026-06-18 11:54:49 -07:00

auth.rs

[codex] Add externally provided Codex auth (#31274)

2026-07-07 17:43:19 -07:00

capabilities_tests.rs

Make selected plugin roots URI-native (#28918)

2026-06-23 22:51:19 +01:00

capabilities.rs

Make selected plugin roots URI-native (#28918)

2026-06-23 22:51:19 +01:00

compacted_item.rs

core: add context window lineage IDs (#29256)

2026-06-20 13:15:49 -07:00

config_types.rs

[codex] Add configurable multi-agent mode hint text (#30493)

2026-07-02 18:44:34 -07:00

dynamic_tools.rs

Expose explicit dynamic tool namespaces in thread start (#27371)

2026-06-15 15:35:57 +00:00

error_tests.rs

Honor workspace spend controls in rate-limit handling (#33187)

2026-07-14 22:43:26 +00:00

error.rs

[codex] rename rollout budget error to session budget error (#29744)

2026-06-23 16:49:13 -07:00

exec_output_tests.rs

extract models manager and related ownership from core (#16508)

2026-04-02 23:00:02 -07:00

exec_output.rs

extract models manager and related ownership from core (#16508)

2026-04-02 23:00:02 -07:00

items.rs

Forward audio inputs to the Responses API (#33932)

2026-07-18 00:26:24 +00:00

legacy_events.rs

Remove template IDs from MCP tool call metadata (#33467)

2026-07-16 01:23:52 +00:00

lib.rs

Require prefixes for outbound response item IDs (#32312)

2026-07-11 02:29:14 +00:00

mcp_approval_meta.rs

Route opted-in MCP elicitations through Guardian (#19431)

2026-05-06 19:42:45 +00:00

mcp.rs

Expose MCP server info as part of server status (#24698)

2026-05-28 09:38:34 -07:00

memory_citation.rs

feat: add memory citation to agent message (#14821)

2026-03-18 10:03:38 +00:00

models.rs

Forward audio inputs to the Responses API (#33932)

2026-07-18 00:26:24 +00:00

network_policy.rs

extract models manager and related ownership from core (#16508)

2026-04-02 23:00:02 -07:00

num_format.rs

Fix execpolicy parsing for multiline quoted args (#9565)

2026-01-22 22:16:53 -08:00

openai_models.rs

Support model catalog templates for Guardian policy prompts (#33177)

2026-07-14 22:01:11 +00:00

parse_command.rs

[app-server] remove serde(skip_serializing_if = "Option::is_none") annotations (#5939)

2025-10-30 18:18:53 +00:00

permissions.rs

Remove generated-default filesystem path variants (#33632)

2026-07-16 16:13:53 +00:00

plan_tool.rs

Plan mode: stream proposed plans, emit plan items, and render in TUI (#9786)

2026-01-30 18:59:30 +00:00

protocol.rs

Track inherited paginated rollout prefixes (#33930)

2026-07-18 00:06:49 +00:00

request_permissions.rs

Let permission hooks resolve strict auto-review requests (#32232)

2026-07-10 18:12:27 +00:00

request_user_input.rs

Add request_user_input auto-resolution window contract (#27256)

2026-06-11 22:30:41 -07:00

response_item_id_tests.rs

Require prefixes for outbound response item IDs (#32312)

2026-07-11 02:29:14 +00:00

response_item_id.rs

Require prefixes for outbound response item IDs (#32312)

2026-07-11 02:29:14 +00:00

review_format.rs

feat(core): emit canonical review mode items (#31473)

2026-07-08 09:59:50 -07:00

session_id.rs

feat: add session_id (#20437)

2026-05-06 10:48:37 +02:00

shell_environment.rs

expand the set of core shell env vars for Windows. (#20089)

2026-04-29 19:23:46 +00:00

thread_id.rs

app-server: document thread and turn IDs are UUID7 (#27714)

2026-06-23 11:46:36 -07:00

tool_name.rs

Remove ToolName display helper (#21465)

2026-05-08 12:17:48 -07:00

user_input.rs

Forward audio inputs to the Responses API (#33932)

2026-07-18 00:26:24 +00:00