nhamidi-oai 97dbd01a3d Add audio variants to user input protocols (#33923) ...

## What changed

- Add remote and local audio variants to core and app-server user inputs, including generated JSON and TypeScript schemas.
- Preserve audio fields when converting app-server inputs and response content into thread items.
- Keep unsupported audio out of model requests, TUI rendering, text extraction, and truncated tool output, using explicit text placeholders where audio is submitted as a model input.

## Testing

- Cover app-server media conversion, unsupported-input placeholders, TUI omission, and tool-output truncation.

GitOrigin-RevId: 16152bf3af674575ce569764a8b472e032e7569d

2026-07-17 23:14:37 +00:00

..

account.rs

Fix the managed Bedrock logout test assertion (#33848)

2026-07-17 15:33:32 +00:00

analytics.rs

[codex] Attribute app-server analytics by thread originator (#29935)

2026-06-25 18:15:48 -07:00

app_installed.rs

Add an API for reading installed app runtime state (#33843)

2026-07-17 15:23:35 +00:00

app_list.rs

Avoid duplicate cached app list update notifications (#33640)

2026-07-16 16:43:38 +00:00

app_read.rs

Add an app-server API for reading app metadata (#33651)

2026-07-16 17:46:36 +00:00

attestation.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

auto_env.rs

test: add delayed exec-server transport (#31427)

2026-07-09 00:17:38 +00:00

client_metadata.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

collaboration_mode_list.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

command_exec.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

compaction.rs

Add cache-write tokens to the raw response schema (#33500)

2026-07-16 04:45:05 +00:00

config_rpc.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

connection_handling_websocket_unix.rs

[codex] Add user input client ids (#24653)

2026-05-28 14:54:39 -07:00

connection_handling_websocket.rs

Emit exec-policy warnings for freshly loaded thread config (#31253)

2026-07-06 10:17:15 -07:00

current_time.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

dynamic_tools.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

environment_add.rs

Report selected environment connection transitions (#33251)

2026-07-15 05:21:13 +00:00

environment_info.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

environment_status.rs

Expose environment status through app-server (#32920)

2026-07-14 01:49:16 +00:00

exec_server_test_support.rs

[app-server] expose environment info RPC (#30291)

2026-06-27 19:34:10 +00:00

executor_mcp.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

executor_skills.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

experimental_api.rs

Route realtime V3 handoffs by response channel (#33903)

2026-07-17 21:12:40 +00:00

experimental_feature_list.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

external_agent_config.rs

Report detailed session import error types (#33863)

2026-07-17 16:14:36 +00:00

fs.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

hooks_list.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

host_skills.rs

Refresh host skill catalogs through world state (#33425)

2026-07-15 20:59:21 +00:00

imagegen_extension.rs

Forward thread originators from standalone extensions (#33677)

2026-07-16 19:31:26 +00:00

initialize.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

marketplace_add.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

marketplace_remove.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

marketplace_upgrade.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

mcp_resource.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

mcp_server_elicitation.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

mcp_server_status.rs

Expand MCP tool catalog cache regression coverage (#33308)

2026-07-15 10:18:51 +00:00

mcp_tool.rs

Scope runtime workspace roots to execution environments (#32952)

2026-07-14 04:41:08 +00:00

memory_reset.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

mod.rs

Add SessionEnd hooks for thread teardown (#33895)

2026-07-17 19:58:56 +00:00

model_list.rs

Always send reasoning parameters in Responses requests (#32206)

2026-07-10 16:11:58 +00:00

model_provider_capabilities_read.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

output_schema.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

permission_profile_list.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

plan_item.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

plugin_install.rs

Import enabled plugins from known marketplaces (#31672)

2026-07-09 12:16:15 -04:00

plugin_list.rs

Expose plugin installation interstitial requirements (#33896)

2026-07-17 20:02:51 +00:00

plugin_read.rs

Expose plugin installation interstitial requirements (#33896)

2026-07-17 20:02:51 +00:00

plugin_share.rs

Allow publishing plugins through share updates (#33908)

2026-07-17 21:29:51 +00:00

plugin_uninstall.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

process_exec.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

rate_limit_reset_credits.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

rate_limits.rs

Honor workspace spend controls in rate-limit handling (#33187)

2026-07-14 22:43:26 +00:00

realtime_conversation.rs

Route realtime V3 handoffs by response channel (#33903)

2026-07-17 21:12:40 +00:00

recommended_plugins.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

remote_control.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

remote_thread_store.rs

Preserve paginated history for spawned subagents (#33432)

2026-07-15 21:25:24 +00:00

request_permissions.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

request_user_input.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

request_validation.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

review.rs

Enable paginated thread history in app-server (#33364)

2026-07-15 15:29:13 +00:00

safety_check_downgrade.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

selected_capability_stack.rs

Add batched executor capability discovery (#33852)

2026-07-17 15:45:48 +00:00

selected_environment.rs

Scope runtime workspace roots to execution environments (#32952)

2026-07-14 04:41:08 +00:00

session_end.rs

Add SessionEnd hooks for thread teardown (#33895)

2026-07-17 19:58:56 +00:00

skills_list.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

sleep.rs

Move sleep items to the extension-owned lifecycle path (#33159)

2026-07-14 21:01:54 +00:00

thread_archive.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

thread_delete.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

thread_fork.rs

Preserve thread context when retrying or editing turns (#33211)

2026-07-15 02:35:49 +00:00

thread_inject_items.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

thread_list.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

thread_loaded_list.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

thread_memory_mode_set.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

thread_metadata_update.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00

thread_name_websocket.rs

Restore app-server websocket listener with auth guard (#22404)

2026-05-12 18:40:53 -07:00

thread_read.rs

Add audio variants to user input protocols (#33923)

2026-07-17 23:14:37 +00:00

thread_resume.rs

Remove template IDs from MCP tool call metadata (#33467)

2026-07-16 01:23:52 +00:00

thread_rollback.rs

Enable paginated thread history in app-server (#33364)

2026-07-15 15:29:13 +00:00

thread_settings_update.rs

Scope runtime workspace roots to execution environments (#32952)

2026-07-14 04:41:08 +00:00

thread_shell_command.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

thread_start.rs

Enable paginated thread history in app-server (#33364)

2026-07-15 15:29:13 +00:00

thread_status.rs

Timestamp app-server notifications at emission (#32905)

2026-07-14 00:38:04 +00:00

thread_unarchive.rs

Preserve paginated history for spawned subagents (#33432)

2026-07-15 21:25:24 +00:00

thread_unsubscribe.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

turn_interrupt.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

turn_start_zsh_fork.rs

Give the zsh fork decline test more execution time (#33842)

2026-07-17 15:18:47 +00:00

turn_start.rs

Add cache-write tokens to the raw response schema (#33500)

2026-07-16 04:45:05 +00:00

turn_steer.rs

test: migrate app-server v2 starts to auto env (#31614)

2026-07-08 14:13:59 -07:00

web_search.rs

Forward thread originators from standalone extensions (#33677)

2026-07-16 19:31:26 +00:00

windows_sandbox_setup.rs

test: migrate TestAppServer callers to builder (#31451)

2026-07-08 00:15:49 +00:00