mirror of
https://github.com/openai/codex.git
synced 2026-10-28 16:06:15 +00:00
2342b2c2a66b5ac2a07278afb4f5f38700be452e
508
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 Owen Lin
|
2342b2c2a6 |
feat(rollout): persist TurnItems for paginated thread rollouts (#30188)
## Description This PR makes new threads with `history_mode = "paginated"` persist `ItemCompleted(item: <turn_item>)` in their rollout JSONL file. Legacy threads keep persisting the existing legacy events. Because the format is selected per thread, a rollout is either legacy or paginated; we do not need to support mixed rollouts containing both representations. This PR depends on [#31473](https://github.com/openai/codex/pull/31473). ## Why Paginated thread history needs stable turn/item IDs and completed item snapshots so the later SQLite projector can materialize appended rollout JSONL without rebuilding the whole thread. Keeping the legacy persistence policy unchanged avoids changing historical rollouts or the readers that still consume them. ## What changed - Made rollout filtering history-mode aware. Paginated threads keep completed canonical `ItemCompleted` events and drop their redundant legacy projections; legacy threads keep the existing event set. - Made forks inherit the source thread history mode, so copied legacy history is never filtered as paginated. - Made paginated threads assign IDs to locally-created response items even when `Feature::ItemIds` is off, and reject streamed output items that arrive without server IDs. - Updated legacy turn replay, rollout list/search, and SQLite metadata extraction to understand completed canonical user-message items. |
||
|
Michael Bolin
|
5892c7b69d | model-provider: route model discovery through HTTP client factory (#31361) | ||
|
stevenlee-oai
|
555aa79d5a |
[connectors] Refresh codex_apps /ps/mcp auth (#31486)
[Codex Thread 019f2408-dc59-79f2-b245-4c11debd1a61](https://codex-thread-link.openai.chatgpt-team.site/thread/019f2408-dc59-79f2-b245-4c11debd1a61) ## Why Long-lived Codex sessions can outlive the ChatGPT bearer token that was present when the MCP runtime started. The Responses path already recovers from token expiration by refreshing or reloading the shared `AuthManager`. The reserved `codex_apps` hosted-plugin client did not observe that update: `McpConnectionManager` built its `/ps/mcp` HTTP auth once from a `CodexAuth` snapshot, and `auth_provider_from_auth` copied that snapshot bearer into a static `BearerAuthProvider`. After the copied bearer expired, `/ps/mcp` kept sending it even though Responses had a newer token in the same `AuthManager`. The failure occurred before downstream connector execution, so unrelated apps such as Gmail, Slack, and Google Calendar could all fail with the same transport-level `401 token_expired`. This replaces [openai/codex#29474](https://github.com/openai/codex/pull/29474), which was closed for inactivity without being merged. A new long-lived-session report reproduced the same simultaneous `/ps/mcp` expiry pattern across unrelated apps. ## What changed - Add an `AuthManager`-backed request-header provider in `codex-model-provider`. It keeps an `Arc<AuthManager>` and reads `auth_cached()` for each outbound request, so the next `/ps/mcp` call sees a token refreshed by the existing Responses/auth-recovery flow. - Scope that provider to the startup account, ChatGPT user, and workspace identity. Same-identity token reloads are followed; an account switch emits no ambient auth until account-scoped MCP state is rebuilt. - Have `McpConnectionManager` construct the dynamic provider only for the reserved `codex_apps` registration used by the hosted-plugin `/ps/mcp` path. | MCP path | Auth behavior after this change | | --- | --- | | Reserved `codex_apps` hosted-plugin `/ps/mcp` | Read current same-identity auth from the shared `AuthManager` per request | | `codex_apps` with `CODEX_CONNECTORS_TOKEN` | Keep the environment bearer-token override | | User-configured/direct MCP registrations | Keep their existing configured auth path | ## Non-goals - No plugin-service changes. - No downstream Slack, Gmail, Calendar, or other connector OAuth/link-refresh changes. - No auth UI changes. - No behavior change for user-configured/direct MCP registrations. - No new `/ps/mcp`-initiated token refresh; this makes `/ps/mcp` observe refreshes already performed through the shared `AuthManager`. ## Tests - `just test -p codex-model-provider` - Covers same-identity token reloads and refuses a changed startup identity. - `just test -p codex-mcp` - `just test -p codex-core mcp_auth_refresh` - Creates the reserved hosted-plugin `codex_apps` `/ps/mcp` client before the shared `AuthManager` changes, updates that same manager through its public external-auth path, performs a real `tools/call`, and asserts the request uses the current bearer. |
||
|
viyatb-oai
|
0746e8a345 |
[codex] Preserve reviewer when resuming threads (#30278)
## Why A thread resumed without an explicit reviewer could pick up the reviewer from the current config instead of preserving the reviewer already in use by the thread. After an app restart, this meant a thread running with auto review could silently switch back to user review, and the next turn could continue under the wrong reviewer. ## What changed Persist the effective reviewer with each turn and restore the latest persisted value when the thread resumes. If the resume request explicitly provides a reviewer, that value still takes precedence. ## Test plan - Added a regression test that starts a thread with auto review, records a turn, restarts with user review in config, resumes without an override, and verifies that auto review is preserved. - `just test -p codex-protocol` - `just test -p codex-state` - `just test -p codex-rollout` - `just test -p codex-app-server thread_resume_preserves_persisted_approvals_reviewer` - Clippy for the affected crates |
||
|
|
b780738014 |
Update models.json (#21818)
Automated update of models.json. --------- Co-authored-by: aibrahim-oai <219906144+aibrahim-oai@users.noreply.github.com> Co-authored-by: Ahmed Ibrahim <aibrahim@openai.com> Co-authored-by: Sayan Sisodiya <sayan@openai.com> |
||
|
Channing Conger
|
c55cb4b363 |
code-mode: make all approvals trigger elicitation pause (#31650)
### summary We want to pause code-mode from yielding back to the model when a subcommand triggers an approval prompt. This means that all of these previously inline blocking requests should also take out a ElicitationService registration. This also does some plumbing refactoring to request patch approval to make it match the other `request_*_approval` methods in that it blocks on the approval in the function instead of returning the oneshot channel, this affords our ability to encapsulate the ElicitationService registration via RAII. Adds tests to confirm the blocking behavior for code_mode both in suite tests and that the session holds them. |
||
|
Owen Lin
|
ba5dd1fd3a |
feat(core): emit canonical hook prompt items (#31630)
## Description This PR moves hook prompts onto the canonical `TurnItem` lifecycle in core. Stop hooks now record their `ResponseItem` through the existing lifecycle path, which emits `ItemStarted` and `ItemCompleted`. App-server consumes those events directly instead of deriving a hook prompt from `RawResponseItem`. ## Why Hook prompts were the only `ThreadItem` app-server synthesized from `RawResponseItem`. This brings them in line with other core-owned turn items while preserving legacy rollout replay. ## What changed - Route stop-hook prompts through `record_response_item_and_emit_turn_item`. - Materialize canonical hook prompts in `ThreadHistoryBuilder`. - Remove `RawResponseItem` to `ThreadItem` synthesis while preserving legacy rollout replay. - Add focused coverage for lifecycle emission and canonical and legacy history materialization. |
||
|
sayan-oai
|
bdaad6820c |
Reuse MCP tool snapshot within a sampling request (#31292)
Follow-up to #30226. ## Why #30226 makes Apps World State inspect the MCP tool list, while tool-router construction reads the same list again later in the sampling request. `list_all_tools()` walks the MCP clients and may reconnect or wait for tools, so doing that work twice adds latency and lets context and tool construction observe different MCP states for one request. ## What - Add a lazy MCP tool snapshot to `StepContext`. - Reuse that snapshot for Apps World State and tool-router construction. - Let each new `StepContext` refresh naturally for the next sampling request, without manager-level caching or invalidation. ## Testing - `just test -p codex-core apps_instructions` - `just test -p codex-core apps_guidance_appears_after_background_recovery_within_a_turn` |
||
|
Owen Lin
|
23aac925e7 |
feat(core): emit canonical review mode items (#31473)
## Description This PR moves review-mode markers onto canonical `TurnItem` lifecycle: - `TurnItem::EnteredReviewMode` - `TurnItem::ExitedReviewMode` Core now emits `ItemStarted` / `ItemCompleted` for both. The completed items map back into the existing `EnteredReviewMode` / `ExitedReviewMode` events, so raw core event consumers and legacy rollout persistence keep seeing the old events. This is the compatibility layer needed before paginated rollouts persist review markers as `ItemCompleted(TurnItem)`. ## Why Review markers were one of the remaining app-server thread items created directly from legacy events. Giving them canonical items lets paginated history persist stable turn/item IDs without changing legacy rollouts. ## What changed - Added canonical review-mode `TurnItem`s and switched review flow to emit their lifecycle. - Added completed-item → legacy review event mappings with stable turn/item IDs. - Switched app-server live notifications to the generic canonical item path and kept legacy replay compatible with old payloads. - Updated `ThreadHistoryBuilder` to replay canonical review items even though review turns still do not emit `TurnStarted`. |
||
|
Celia Chen
|
172ab264bd |
fix: retry rejected previous-model compaction with selected model (#30319)
## Why Pre-sampling compaction intentionally uses the previous turn's model when the compaction compatibility hash changes or when switching to a model with a smaller context window. This keeps compaction aligned with the settings that produced the history, but it can block the next turn when a resumed ChatGPT thread still references a model slug that has since been retired. The Codex backend rejects that compaction request before the user's currently selected model gets a chance to sample. This PR lets those threads recover without changing previous-model compaction behavior for API-key authentication or custom providers. It is stacked on #31316, which is a behavior-preserving extraction of the individual remote compaction attempts; this PR contains the fallback behavior. ## What changed - For automatic previous-model compaction, capture the selected model's request context when using ChatGPT authentication with the OpenAI provider and the selected model differs from the previous model. - If the previous-model attempt returns an `InvalidRequest`, retry compaction once with the selected model for both `/responses/compact` and Responses Compaction V2. - Complete history processing, lifecycle events, and token accounting with the context of the model that successfully compacted the thread. - If the fallback also fails, return the original previous-model error so the retry does not change the user-visible failure. - Record fallback attempts with reason, implementation, and outcome telemetry. - Leave API-key authentication, custom providers, same-model turns, and non-`InvalidRequest` failures on their existing paths. ## Testing - `just test -p codex-core -E 'test(pre_sampling_compact) | test(model_unavailable_error)'` (10 tests) - Added integration coverage for a resumed thread whose model was renamed, a model downshift using Responses Compaction V2, and API-key authentication with a custom provider. |
||
|
Dylan Hurd
|
358575465c |
Use model catalog approval messages (#31312)
## Why Approval guidance is currently assembled entirely by the client. Model Messages V2 needs model catalogs to provide model-specific `on_request` guidance for both user-reviewed and auto-reviewed approval flows while retaining the existing generated prompt as a compatibility fallback. ## What changed - add nullable `on_request` and `on_request_auto_review` catalog messages - select the message matching the active approvals reviewer for `on_request` policies - replace the complete legacy approval section when the selected catalog value exists, including support for an empty string that suppresses the section - retain legacy rendering when the object or selected key is absent, and for non-`on_request` policies - preserve approval messages when base-instruction or personality overrides clear instruction templates - refresh permissions instructions when the active model changes - pass catalog messages through initial and incremental permissions construction ## Relationship to reviewer persistence PR #31309 independently persists the approvals reviewer in turn context and refreshes permissions when that reviewer changes. This PR is based directly on `main` and does not duplicate that rollout migration; once both land, reviewer switches will also select and append the new catalog variant. ## Testing - `just test -p codex-protocol` - `just test -p codex-prompts` - `just test -p codex-models-manager` - `just test -p codex-core permissions_messages` |
||
|
Michael Bolin
|
6afcf26d5d |
core: route Responses API through system proxy (#31335)
## Why `features.respect_system_proxy` already routes authentication traffic through the OS proxy APIs, but it does not affect the primary inference path. That leaves users behind OS-managed proxies unable to send normal Responses API requests even after login succeeds. This PR is the first product-path migration onto the route-aware transport introduced in #31323 and refined in #31331. It also establishes the construction pattern for later migrations: the effective feature state is resolved once into a required HTTP client factory rather than represented by an optional per-call setting. The scope remains limited to the two HTTP Responses endpoints; WebSockets, model discovery, memories, realtime, and file uploads remain follow-up migrations. ## What changed - Replace the optional proxy marker with an explicit `OutboundProxyPolicy::{ReqwestDefault, RespectSystemProxy}` and a required `HttpClientFactory`. The policy has no default, and the lower-level route-aware reqwest builder is now private. - Have `Config` construct the factory from the effective feature state and require every `ModelClient` constructor to receive it. There is no optional setter or implicit `None` fallback. - Build HTTP clients for `/responses` and `/responses/compact` with `ClientRouteClass::Api`, using the complete destination URL so PAC rules can make URL-specific decisions. - Layer route-aware selection onto Codex's existing default headers, Cloudflare cookie store, custom CA handling, and sandbox no-proxy behavior. - Add an integration test that loads `features.respect_system_proxy` through `config.toml`, creates a real Codex session, and verifies that both a normal Responses turn and remote compaction reach an isolated local proxy. ## Review guide 1. `http-client/src/outbound_proxy.rs` defines the mandatory policy/factory boundary and keeps route resolution private. 2. `core/src/config/mod.rs`, `core/src/session/session.rs`, and `core/src/client.rs` show the compile-time invariant: effective config creates the factory, and `ModelClient` cannot be constructed without one. 3. `login/src/auth/default_client.rs` preserves existing default-client behavior while accepting the required factory for migrated routes. 4. `core/src/client.rs` switches only streaming Responses and remote compaction HTTP transports to the API route class. 5. `core/tests/suite/responses_api_system_proxy.rs` is the behavioral regression boundary. Its Linux subprocess deliberately sets the CGI marker that disables reqwest's implicit environment-proxy handling, so the test fails if session wiring or either Responses call site falls back to the default client. ## Test plan - `cargo check --tests -p codex-http-client -p codex-login -p codex-core` - `just test -p codex-login` - `just test -p codex-core respect_system_proxy_feature_resolves_enabled` - Existing `compact_uses_bearer_after_agent_identity_session_fallback` coverage passes with the new transport construction. - New Linux integration coverage: `responses_and_compact_use_enabled_system_proxy` - `just bazel-lock-check` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/31335). * #31342 * __->__ #31335 |
||
|
Owen Lin
|
b9b934e99b |
refactor(protocol): map canonical tool items to legacy events (#31296)
## Description This PR adds legacy `EventMsg` mappings for the `TurnItem` types introduced in [#30282](https://github.com/openai/codex/pull/30282): - `CommandExecution` - `DynamicToolCall` - `CollabAgentToolCall` - `SubAgentActivity` When their producers move to canonical `ItemStarted` / `ItemCompleted`, raw core event consumers can still receive the existing begin/end-style events. The canonical item lifecycle remains the live source of truth. We also record the mapped legacy events in rollout trace so the producer migration preserves the existing tool-runtime trace entries. ## Why This is the compatibility layer for the follow-up producer migrations. Splitting it out first keeps each producer PR small and keeps the legacy mapping in one place. ## What changed - Added `TurnItem` → legacy `EventMsg` mappings in `protocol/src/legacy_events.rs`. - Added the command execution status conversion used by the exec mapping. - Added focused coverage for command execution and dynamic tool mappings. |
||
|
ashwinnathan-openai
|
775ef7dcc7 |
[codex] Support sequential cutoff reasoning summaries (#31306)
## Summary When enabled for the OpenAI provider, Codex sends `stream_options.reasoning_summary_delivery = "sequential_cutoff"` on HTTP and WebSocket requests, including prewarm, and renders completed summary sections from `reasoning_summary_text.done`. Flag-off and non-OpenAI behavior is unchanged. ## Expected rollout ```text reasoning 0 added summary 0 done summary 1 done summary 2 starts summary 2 cancelled / incomplete reasoning 0 done <-- cancel summary 2 work and mark it incomplete message 1 added message 1 text streams message 1 completed ``` Depends on [openai/openai#1096660](https://github.com/openai/openai/pull/1096660). |
||
|
Eric Traut
|
45be435135 |
Warn when configured service tiers are unsupported (#31284)
## Why Codex currently omits a configured `service_tier` when the selected model's catalog entry does not advertise support for it. That fallback is silent, so users can unknowingly send requests at the default tier instead. This makes cases such as #26604 difficult to diagnose. ## What changed - Emit the shared core `Warning` event during session startup when a configured service tier will be omitted because the initial model does not advertise support for it. - Do not warn on later model or service-tier changes, which keeps warning emission stateless and avoids client-specific handling. - Keep the existing request filtering behavior unchanged. ## Validation - `just test -p codex-core unsupported_service_tier` - `just test -p codex-core unsupported_configured_service_tier_warns_at_session_start` ### Manual validation - Launched the real TUI with the bundled catalog, where `gpt-5.5` advertises the `priority` tier but not `flex`, and configured `service_tier = "flex"`. - Confirmed the unsupported-tier warning appeared exactly once during startup. - Submitted two turns through a local Responses API SSE stub; both received mock replies and neither emitted another warning. - Inspected both captured `/v1/responses` request bodies and confirmed that neither contained a `service_tier` field. - Repeated the two-turn TUI flow against the live Responses API; both turns completed and the warning was not repeated. |
||
|
sayan-oai
|
3c2bfe7c5d |
Make Apps guidance react to MCP availability (#30226)
## Why Generic Apps guidance is emitted only while building static initial context. If the Apps MCP is unavailable then and recovers later in the same turn, its tools can become usable without the model receiving the guidance for using them. ## What - move generic Apps guidance into a persisted `apps_instructions` World State section - derive availability from the request's MCP runtime while preserving the existing feature, auth, orchestrator-MCP, and config gates - recognize legacy and retained Apps fragments so resume and compaction do not duplicate guidance - register Apps guidance as rollback-trimmable context - remove the old static injection path and its now-unused connector helper - keep tool construction on its existing independent `list_all_tools()` read rather than adding a request-wide cache; a reconnect between the two reads can differ for one request and reconciles on the next request Apps and plugin guidance now render after the remaining static host-skills block, with Apps before Plugins. ## Testing - `just fmt` - `just test -p codex-core apps_instructions` - `just test -p codex-core apps_guidance_appears_after_background_recovery_within_a_turn` - `just test -p codex-core drop_last_n_user_turns_trims_context_updates_above_rolled_back_turn` |
||
|
guinness-oai
|
c976741124 |
[codex] Flush trailing realtime transcript tail (#29918)
## Why A realtime session can end after more transcript has accumulated than was included in its last handoff. That tail already lives in core's active transcript state, but the stop path aborted the realtime input/fanout tasks before routing it, so the final bit of the conversation could disappear before `thread/realtime/closed`. This behavior is still being evaluated, so clients must opt in per realtime session. Omitted or false leaves shutdown behavior unchanged. ## What changed - Add optional `flushTranscriptTailOnSessionEnd` to `thread/realtime/start`, defaulting to false in app-server. - Expose an idempotent `take_transcript_tail()` from the existing active transcript state using `last_handoff_entry_count`. - When enabled, let shutdown cancel the input owner cleanly and publish at most one final existing `<realtime_delegation>` with the remaining text in `<transcript_delta>`. - Have the existing fanout drain already-parsed events before routing that final delegation, so a queued handoff wins first and is not duplicated in the tail. - Flush realtime shutdown before ordinary session task abort during core cleanup. ## Validation - `just test -p codex-app-server-protocol` - `just test -p codex-app-server realtime_conversation_stop_emits_closed_notification` - `just test -p codex-core conversation_transport_close_` - `just test -p codex-core conversation_close_routes_only_remaining_transcript_tail_once` - scoped `just fix` for `codex-app-server-protocol`, `codex-app-server`, `codex-protocol`, and `codex-core` - `just fmt` |
||
|
Dylan Hurd
|
aa94ea1397 |
chore(approvals) consolidate guardian calls for shell tools (#31267)
## Summary As part of our effort to start simplifying approvals code, this PR extracts Guardian approvals logic from shell tool calls, and replaces it with the ApprovalAction abstraction instead. This way, tools don't need to know about Guardian at all. ## Testing - [x] Adds integration test |
||
|
Adam Perry @ OpenAI
|
a86d525e4d |
core: trace executor skill discovery (#30318)
## Why Make it easier to measure the performance of different parts of skill loading. ## What - Add spans for step-context capture, world-state construction, executor catalog snapshot/root loading, and environment skill loading. - Record the discovered environment skill count. - Trace outbound exec-server requests with client kind and RPC method fields. - Update trace propagation tests to assert that requests keep the parent trace id while creating their own child span. |
||
|
Channing Conger
|
84fe70c30e |
elicitations: Move to shared ElicitationService (#30627)
## Why Code-mode tool results could return to the model while an MCP elicitation was still waiting for user input. This differed from parallel tool calling and could let the model continue before the user resolved the request. We need one session-level view of outstanding elicitations so tool runtimes can consistently hold results until every pending elicitation is resolved. ## What changed - Added a counted, session-owned ElicitationService with RAII registrations. - Registered both core-originated and server-originated MCP elicitations with the service. - Migrated out-of-band elicitation tracking and unified exec timeout pausing to the shared service. - Made code-mode functions.exec and functions.wait capture their runtime result normally, then hold it before returning while an elicitation is outstanding. - Kept terminate: true immediate; only its result is held. - Preserved model-visible wall time across the elicitation hold. - Kept the behavior session-scoped, with concurrent elicitations holding the pause until all registrations are released. |
||
|
Alexi Christakis
|
7b4e70d567 |
Revert "[core] Support interleaved response items" (#31261)
Reverts openai/codex#30876 |
||
|
Alexi Christakis
|
8917244f7d |
[core] Support interleaved response items (#30876)
## Summary - preserve reasoning item IDs on summary part and text delta events - track streamed response items by ID so reasoning summaries can continue after later items begin - keep TUI output complete and deduplicated when reasoning and final-answer events interleave ## Stack - **1/2: this PR — interleaved item support** - 2/2: #30752 — wire reasoning summary delivery configuration through the CLI and app-server ## Validation - just test -p codex-api preserves_reasoning_summary_item_ids - just test -p codex-core interleaved_reasoning_summary_events_keep_reasoning_item_metadata - just test -p codex-tui live_reasoning_summary_is_not_rendered_twice_when_item_completes - just fix -p codex-api -p codex-core -p codex-tui - just fmt |
||
|
sayan-oai
|
9c5be7e1d5 |
Make plugin guidance react to environment readiness (#30223)
## Why Generic plugin guidance is currently emitted only with initial context from host plugin state. An executor-selected plugin can become available later in the same turn, making its skills and tools usable without ever telling the model how plugin capabilities should be used. ## What - project every ready selected plugin package, including skill-only plugins - carry plugin availability with the exact MCP runtime projection while preserving MCP manager reuse when servers and connectors are unchanged - move generic plugin guidance from the static initial-context path into persisted World State - recognize legacy and retained plugin fragments so resume and compaction do not duplicate guidance ## Testing - `just test -p codex-mcp-extension` - `just test -p codex-core plugins_instructions` - `just test -p codex-core plugin_availability_change_reuses_the_mcp_manager` - `just test -p codex-app-server --test all selected_capabilit` |
||
|
Shijie Rao
|
da4c8ca57d |
[codex] Add configurable multi-agent mode hint text (#30493)
## Why
Multi-agent V2 normally derives its mode instructions from reasoning
effort: Ultra enables proactive delegation, while other efforts require
an explicit request. Some deployments need to provide one configured
delegation policy that replaces those built-ins and remains stable when
reasoning effort changes.
## What changed
- Add `features.multi_agent_v2.multi_agent_mode_hint_text` alongside the
existing root and subagent hint settings.
- Treat any configured value, including an empty string, as
`MultiAgentMode::Custom(hint_text)`, so the configured text replaces the
built-in explicit-only and proactive policies.
- Persist the full custom variant and hint text in the turn-context
snapshot, so the durable comparison baseline detects both
reasoning-effort changes and configured policy-text changes.
- Preserve the existing explicit-only/proactive behavior when the
setting is absent.
- Replace the ambiguous `MultiAgentMode::None` variant with
`MultiAgentMode::Custom(String)` in new rollouts and API schemas. A
compatibility wire type maps legacy serialized `none` values to
`Custom("")` when resuming existing rollouts.
- Regenerate the config and app-server schemas.
## Configuration examples
The distinction is whether `multi_agent_mode_hint_text` is present. An
empty string is still a configured value and intentionally suppresses
the built-in mode instructions.
### Unset: preserve existing effort-derived behavior
```toml
[features.multi_agent_v2]
enabled = true
# multi_agent_mode_hint_text is omitted
```
- Ultra reasoning uses the built-in proactive delegation instructions.
- Other reasoning efforts use the built-in explicit-request-only
instructions.
### Empty: suppress all mode hint text
```toml
[features.multi_agent_v2]
enabled = true
multi_agent_mode_hint_text = ""
```
This selects effective mode `custom` at every reasoning effort and
injects an empty mode body, suppressing both built-in policies.
### Set: always use the configured text
```toml
[features.multi_agent_v2]
enabled = true
multi_agent_mode_hint_text = "Delegate to subagents when it will materially improve the result."
```
This selects effective mode `custom` at every reasoning effort and
injects the configured text verbatim instead of either built-in policy.
## Verification
- `just test -p codex-core multi_agent_mode`
- Covers a configured hint across High and Ultra reasoning efforts and
verifies the full custom hint is recorded for both turns.
- Covers an empty-string override suppressing both built-in instruction
bodies.
- `just test -p codex-protocol -p codex-app-server-protocol`
- Covers legacy `none` turn-context deserialization as `Custom("")` and
verifies the regenerated schemas.
|
||
|
Michael Bolin
|
129ea2aaf5 |
Log multi-agent communication lifecycle (#30872)
## Why [#30867](https://github.com/openai/codex/pull/30867) makes `submit_inter_agent_communication` the common outbound sink for multi-agent v2 communications. This follow-up uses that single point to log every communication lifecycle without requiring new hooks as spawn, messaging, follow-up, or result paths evolve. For each communication, the logs need to identify its type, sender and receiver threads, and content, while correlating the successful send with receipt by the destination mailbox. The logging path must not query externally supplied time providers because those calls can be expensive for app-server clients. ## What changed - Added structured `INFO` events on the OpenTelemetry-exported `codex_otel.agent_communication` target for `spawn`, `message`, `followup`, and `result` communications. - Logged successful sends from `submit_inter_agent_communication` with the communication kind, sender and receiver thread IDs, content, and submission ID. - Logged receives after the communication has been enqueued in the receiver mailbox, using the same submission ID. - Avoided time-provider calls and other asynchronous work in the logging path. - Narrowed ordinary spawn and send-input APIs to `Vec<UserInput>` so `Op::InterAgentCommunication` cannot bypass the context-bearing centralized path. The refactor does not change submission IDs, capacity checks, last-task bookkeeping, mailbox ordering, protocol types, rollout data, or model-visible context. ## Event shape Illustrative JSON representation of the two independently emitted records: ```json [ { "event.name": "codex.agent_communication", "communication_id": "019f20e1-40d1-7890-a123-456789abcdef", "kind": "spawn", "state": "send", "sender_thread_id": "019f20df-fbe1-7890-a123-456789abcdef", "receiver_thread_id": "019f20e1-3f79-7890-a123-456789abcdef", "content": "inspect the repository" }, { "event.name": "codex.agent_communication", "communication_id": "019f20e1-40d1-7890-a123-456789abcdef", "state": "receive" } ] ``` Consumers join the receive record to the send record by `communication_id` for the immutable communication metadata. ## Testing - Extended the existing end-to-end multi-agent v2 spawn test to verify content, both thread IDs, and a correlated send/receive submission ID. - Re-ran focused control and handler coverage for direct messages, follow-up tasks, and completion results. |
||
|
ani-oai
|
6b5f5743b3 |
[codex] Use model metadata for skills usage instructions (#29740)
## Summary - add a false-by-default `include_skills_usage_instructions` model metadata field - enable the field for the bundled `gpt-5.5` model metadata - consume the metadata in both core and extension skill rendering - remove hardcoded legacy-model matching and its marker plumbing |
||
|
nhamidi-oai
|
328e95110c |
Preserve namespaces on custom tool calls (#30302)
## Summary - Preserve the optional namespace on custom tool calls during response deserialization and app-server replay. - Use the namespaced tool identifier for streaming argument handling and tool dispatch. - Regenerate app-server protocol schemas. - Add regression tests covering namespace serialization and routing. ## Testing - Ran affected protocol and app-server test suites. - Ran the full core test suite; two load-sensitive timing tests passed when rerun individually. - Ran Clippy and formatting checks. - Verified with a local end-to-end app-server replay that the namespace is preserved through the complete request/response flow. |
||
|
Adam Perry @ OpenAI
|
3ae0543fdd |
core: overlap diff root discovery with world state (#30286)
## Why Remote diff-root discovery is independent of world-state construction, but it ran afterward and added filesystem metadata latency before the first model request. Overlap the independent work so thread-cold turns do not pay those waits serially. ## What - Run `record_context_updates_and_set_reference_context_item` and `turn_diff_display_roots` with `tokio::join!`. - Reuse the same resolved display roots when constructing `TurnDiffTracker`; no cache or behavior lifecycle changes are introduced. ## Validation A synthetic executor-skill benchmark with artificial network delay: thread-cold model-request p50 improved from about 1.79 s to 1.58 s. |
||
|
Owen Lin
|
a107b84967 |
feat(protocol): define missing rollout turn items (#30282)
## Description
This PR adds canonical core `TurnItem` shapes for command execution,
dynamic tool calls, collab agent tool calls, and sub-agent activity, to
be stored in the rollout file soon.
It also teaches app-server protocol / `ThreadHistoryBuilder` how to
render those items, and adds the small legacy fanout helpers needed for
existing event-based consumers. No core producer or rollout persistence
behavior changes here, that will be done in a followup.
## Making ThreadHistoryBuilder stateless
This is the first PR in a stack to make `ThreadHistoryBuilder` stateless
enough that we can materialize app-server `ThreadItem`s from only a
given slice of `RolloutItem` history, without ever needing to replay the
whole thread from the beginning.
The persisted legacy `RolloutItem::EventMsg` records are mostly shaped
like live UI events, not like materialized `ThreadItem`s. They work if
we replay the full rollout in order, but they often do not contain
enough stable identity or complete item state to project an arbitrary
suffix on its own.
A few examples:
- `UserMessageEvent` and `AgentMessageEvent` have content, but
historically do not carry the persisted app-server item ID that should
become the SQLite primary key.
- `AgentReasoningEvent` and `AgentReasoningRawContentEvent` are
fragments. `ThreadHistoryBuilder` currently merges them into the last
reasoning item, which means a slice starting in the middle of reasoning
cannot know whether to append to an earlier item or create a new one.
- `WebSearchEndEvent`, `McpToolCallEndEvent`, collab end events, and
similar legacy events can often render a final-looking item, but they
usually rely on prior replay state to know which turn owns the item.
- Begin/end legacy events are partial views of one logical item. The
builder correlates them by `call_id` and mutates prior state to
synthesize the final `ThreadItem`.
That is the problem this direction fixes. A persisted canonical
lifecycle record looks much closer to the read model we actually want
later:
```rust
ItemCompletedEvent {
turn_id,
item: TurnItem { id, ...full snapshot... },
completed_at_ms,
}
```
Once rollout has explicit `turn_id`, stable `item.id`, and a canonical
completed item snapshot, the future SQLite projector can reduce only the
new rollout suffix and upsert the affected `thread_items` rows. It no
longer needs to synthesize `item-N`, infer item ownership from the
active turn, or replay earlier events just to reconstruct the current
item snapshot.
## What changed
- Added core `TurnItem` variants and item structs for command execution,
dynamic tool calls, collab agent tool calls, and sub-agent activity.
- Added conversions from those canonical items back into the legacy
event shapes where current consumers still need them.
- Added app-server v2 `ThreadItem` conversion for the new core item
variants.
- Taught `ThreadHistoryBuilder` and rollout persistence metrics to
recognize the new item variants.
## Follow-up
The next PR https://github.com/openai/codex/pull/30283 switches the live
core producers for these item families onto canonical `ItemStarted` /
`ItemCompleted` events.
|
||
|
Abdulrahman Alfozan
|
c55ce3b51b |
Close thread persistence when submission channel closes (#30173)
### Summary Release live thread persistence when a session ends because its submission channel closes. This prevents a later same-process resume from failing with `thread ... already has a live local writer`. ### Details The issue is in the `codex-core` session teardown path used by Codex hosts, rather than in Managed Agents API or exec-server itself. Explicit shutdown already closes the `LiveThread`, which releases the process-scoped writer held by `LocalThreadStore`. The submission-channel-close fallback ran runtime and extension teardown but skipped that persistence shutdown, leaving the thread ID registered as having a live writer. This change: - closes the `LiveThread` on the channel-close fallback path; - preserves the existing teardown order used by explicit shutdowns; - extends the lifecycle regression test to assert that the thread store receives `shutdown_thread`. Context: [original report](https://openai.slack.com/archives/C0B4NBHQGTV/p1782136364948039), [recent occurrence 1](https://openai.slack.com/archives/C0B4NBHQGTV/p1782434817895839?thread_ts=1782136364.948039&cid=C0B4NBHQGTV), [recent occurrence 2](https://openai.slack.com/archives/C0B4NBHQGTV/p1782335107474429?thread_ts=1782136364.948039&cid=C0B4NBHQGTV) ### Testing - `just test -p codex-core submission_loop_channel_close_runs_full_thread_teardown` - `just test -p codex-core --lib` (1,989 passed; 3 skipped) - `just fix -p codex-core` - `just fmt` - Native code review: no findings I also attempted `just test -p codex-core`. The new regression passed; 79 unrelated integration tests failed in the local harness, primarily because helper binaries such as `test_stdio_server` were unavailable, plus local proxy/shell timing failures. |
||
|
Owen Lin
|
f91334380e |
fix main (#30276)
Introduced by a merge race around thread.history_mode. |
||
|
Owen Lin
|
5267e805fb |
feat(app-server): add history_mode to thread (#29927)
## Description This PR adds a new `historyMode = "legacy" | "paginated"` to `Thread`. This will be stored in `SessionMeta` in the JSONL rollout file and as a new column in the SQLite thread_metadata table, and exposed on `thread/start` and on the `Thread` object in app-server. ## What changed - Added canonical `ThreadHistoryMode` with `legacy` and `paginated`, defaulting old and new SessionMeta to `legacy`. - Carried `history_mode` through core session config, ThreadStore stored metadata, local/in-memory stores, rollout metadata extraction, and the existing SQLite `threads` table. - Added experimental `historyMode` to app-server v2 `Thread` and `thread/start`. - Made paginated stored threads metadata-discoverable but unsupported for legacy full-history reads, `load_history`, live resume, and create paths. - Regenerated app-server schema fixtures and added protocol/state/thread-store/app-server coverage for persistence and fail-closed behavior. ## Compatibility floor Because users may be running various versions of Codex binaries on the same machine (TUI, Codex App, etc.), we will need to establish a compatibility floor for upcoming paginated threads, which will change how thread storage reads and writes work. The overall plan here: ``` Release N: - Add historyMode to SessionMeta / Thread / SQLite metadata. - Teach binaries to understand paginated threads. - If a binary sees `historyMode="paginated"` but does not support the paginated contract, it refuses to resume/mutate the thread. - Default remains `"legacy"`. Release N+1: - First-party clients start opting into paginated threads where appropriate. - Internal dogfood / staged rollout. - Measure old-client usage and paginated-thread unsupported errors. Release N+2: - Only after Release N+ is overwhelmingly deployed, make paginated the default. - Accept that a small tail of N-1-or-older binaries may not understand paginated threads. ``` The important behavior change is fail-closed handling for a binary that encounters a persisted `paginated` thread before it knows how to fully support paginated history. In app-server, if a thread is `paginated`, we will: - allow metadata-only discovery paths like `thread/list` and `thread/read(includeTurns=false)`, so clients can still see the thread and inspect its `historyMode` - reject legacy full-history/live-thread paths like `thread/read(includeTurns=true)` and `thread/resume` with an unsupported JSON-RPC error - avoid silently treating an unknown or future `historyMode` as `legacy` Under the hood, the ThreadStore layer also rejects legacy operations that would need to load or replay the full thread history for a paginated thread. That gives us the behavior we want for Release N: future paginated threads are visible, but this binary fails closed instead of trying to operate on them as if they were legacy threads. |
||
|
jif
|
6d2168f06a |
Reuse MCP runtimes when selected availability changes nothing (#30148)
## Why
MCP runtime reuse was keyed by every ready selected-capability
environment, even when an environment contributed no MCP servers or
connectors.
For example:
1. a global stdio MCP is running;
2. a selected remote environment contains only a skill;
3. that environment becomes ready;
4. the MCP and connector projection stays exactly the same;
5. Codex nevertheless rebuilds the MCP manager and restarts the global
stdio process.
That restart can interrupt active calls and discard process-local state
even though nothing about MCP changed.
## What changes
When selected-environment availability changes, Codex now resolves the
candidate MCP and connector projection before deciding whether to
replace the runtime:
- if the winning MCP servers or their ownership change, rebuild as
before;
- if the selected connector snapshot changes, rebuild as before;
- if an enabled MCP is explicitly bound to an environment whose
availability changed, rebuild as before;
- otherwise, keep the exact live manager and processes, and update only
the availability input remembered by the snapshot.
```text
ready selected environments: [] -> [skills-env]
resolved MCP servers: {global_probe} -> {global_probe}
resolved connectors: {} -> {}
result: reuse manager; keep the same process
```
The comparison uses the resolved winning servers and their sources, so
plugin/config ownership remains part of the runtime identity.
## Existing stack coverage
The integration PR directly below this one already covers both rebuild
boundaries: a selected MCP becomes callable and a selected connector
tool becomes model-visible when their environment becomes available. It
also verifies that an unchanged selected MCP runtime keeps its process.
This PR does not add another remote-attachment integration scenario for
the no-change optimization. `environment/add` returns before readiness,
and app-server does not currently expose a deterministic readiness
signal for an environment that contributes only skills. Keeping a
fixed-delay test would add flake risk; adding a new readiness API would
be outside this fix.
## Scope and assumptions
- This does not change skill discovery, World State rendering, or plugin
metadata caching.
- This does not add file watching or hot reload behavior.
- This does not change disconnect/reconnect handling.
- Selected environment IDs and their capability contents retain the
stack's existing stability assumption.
- Delayed `required = true` executor MCP behavior remains out of scope.
|
||
|
Channing Conger
|
7d8906b478 |
[codex] wire process-owned code mode host into core (#30142)
## Summary - add the `code_mode_host` feature flag and select `ProcessOwnedCodeModeSessionProvider` in `CodeModeService` when enabled - initialize code-mode sessions lazily so a missing host reports a tool error without failing thread startup - resolve `codex-code-mode-host` beside the running Codex binary by default while preserving `CODEX_CODE_MODE_HOST_PATH` as an override - add unit and end-to-end coverage for host resolution and graceful missing-host behavior ## Why This wires the process-owned session client from #30112 into the core service behind an opt-in rollout gate. Packaged Codex installations can place the helper in the same `bin` directory as the main executable without relying on `PATH`, while development and custom installations can continue to override the helper path. ## Stack - Depends on #30112 - Base branch: `cconger/process-owned-session-runtime-4-client` ## Validation Build `codex` and `codex-code-mode-host` `CODEX_CODE_MODE_HOST_PATH="$PWD/target/debug/codex-code-mode-host" ./target/debug/codex --enable code_mode_host` |
||
|
Tom
|
f5f812389e |
[codex] fix terminal rollout event durability (#30144)
Currently session code does not flush the thread store after appending the `TurnComplete` / `TurnAborted` events. This isn't a problem in practice for local storage because append_items itself effectively blocks, but any thread stores that buffer in append_items and only commit on flush effectively never get these events persisted. The fix adds explicit rollout flushes at the terminal emitters after normal completion and interruption. Added test cases that assert the number of flushes when completing or aborting turns. These are admittedly a little brittle and I'm open to better ideas on how to add automated testing. |
||
|
jif
|
fb8598df3f |
Keep MCP elicitation routable across runtime refreshes (#30127)
## Why
An MCP tool call can still be waiting for an elicitation response when
an environment update replaces the thread's MCP runtime.
Before this change:
```text
runtime A starts a tool call and asks the user
environment becomes ready, so runtime B is published
client answers the prompt through runtime B
runtime B cannot find runtime A's pending responder
```
The response is lost and the original tool call stays blocked.
## What changed
All MCP runtimes for one thread now share a small elicitation router:
```text
runtime A ---\
shared router: response token -> exact pending responder
runtime B ---/
```
When Codex surfaces an MCP elicitation, it assigns a unique opaque
response token. The router records which pending request owns that
token. A replacement runtime reuses the same router, so the latest
runtime can deliver a response to a request started by the previous
runtime.
The Codex-owned token also prevents two runtime connections that reuse
the same MCP server request ID from receiving each other's responses.
This does not retain or search old MCP managers. Only the pending
responder map is shared.
## Covered scenario
The integration test exercises the complete failure mode:
1. A thread starts while its selected environment is still unavailable.
2. A configured MCP server starts a tool call and asks the client for
input.
3. The environment becomes ready, causing Codex to publish a replacement
MCP runtime.
4. The client answers the original prompt after the replacement.
5. The original tool call receives that answer and completes.
A focused routing test also creates two runtimes with the same server
request ID and verifies that each response reaches the exact request
that emitted its token.
## Scope
This PR changes only elicitation response routing across MCP runtime
replacement. It does not change when runtimes are rebuilt, which
environments contribute MCP configuration, or how environment
availability is detected.
|
||
|
jif
|
3095ea9c3d |
Project selected plugin runtime by environment availability (#30093)
## Why
Selected plugin metadata is stable, but MCP processes are live runtime
state. They need different lifetimes:
- the MCP extension caches manifest, MCP, and connector declarations for
each stable selected root;
- each model step projects that cached metadata through the roots that
resolved as ready for that exact step;
- the MCP manager is rebuilt only when that availability projection
changes.
This matches executor skills: both features consume the same resolved
step roots instead of inferring readiness from the turn's selected
environments.
## Behavior
```text
E1 not ready for this step
-> no E1 MCP servers or connectors
-> cached plugin metadata stays in ext/mcp
E1 becomes ready
-> reuse cached metadata
-> publish one MCP runtime containing E1 capabilities
same ready roots on the next step
-> reuse the exact runtime; no rediscovery and no MCP restart
resume
-> create new extension thread state and a new MCP runtime
```
All model-facing consumers use the same step snapshot:
```text
resolved selected roots
|
v
extension MCP/connector projection
|
v
{ MCP config, connector snapshot, MCP manager }
|
+-> advertise model tools
+-> build app/connector tools
+-> execute MCP calls
```
## Cache contract
The existing MCP extension owns a cache keyed by the full
`SelectedCapabilityRoot`:
```rust
let state = thread_store.get_or_init(SelectedExecutorPluginMcpState::default);
```
The cache lives with extension thread state. Environment availability
filters projection but does not invalidate metadata. Resume creates new
thread state. There is no file watcher or executor generation because
contents behind a stable environment/root are assumed stable.
## What changes
- Keeps executor plugin discovery and cached metadata in `ext/mcp`.
- Caches MCP and connector declarations together per selected root.
- Uses the step's already-resolved capability roots, including lazy
environments that are not turn environments.
- Reuses the current MCP runtime when the ready-root projection is
unchanged.
- Uses the same step MCP manager and connector snapshot for
model-visible tools and execution.
- Resolves direct thread-scoped MCP requests from the current
selected-root projection.
## Deliberately out of scope
- `app/list` remains based on the latest global host-plugin state; this
PR does not make its response or notifications thread-specific.
- `required = true` startup semantics do not apply to delayed executor
MCP activation.
- No filesystem/content invalidation.
- No transport-disconnect watcher.
- No executor generations or environment replacement semantics.
- No client sharing across complete manager replacements.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. Pin one MCP runtime to each model step.
4. **This PR:** project selected MCP and connector state from
extension-owned metadata.
5. Integration coverage for selected capability availability and resume.
## Verification
-
`selected_plugin_servers_use_managed_requirements_for_the_selected_root_id`
- The stacked integration PR covers unavailable to ready activation,
unchanged-runtime reuse, skills, MCP tools, connector attribution, and
cold resume.
|
||
|
jif
|
ee9e0f6387 |
Pin MCP runtimes to model steps (#30101)
## Why
An MCP refresh can replace the session's current manager while a model
step is still running. The step must execute calls through the same
manager whose tools it advertised.
## Boundary
```text
current session MCP runtime
|
| capture once for this model step
v
StepContext.mcp
- exact MCP config
- exact connection manager
- exact runtime environment context
```
```rust
pub struct McpRuntimeSnapshot {
config: Arc<McpConfig>,
manager: Arc<McpConnectionManager>,
runtime_context: McpRuntimeContext,
}
```
## Example
```text
step A captures runtime A and advertises A's tools
refresh publishes runtime B
step A tool call -> runtime A
next step -> runtime B
```
Capturing the snapshot is only an `Arc` clone. It does not restart MCPs
or make an RPC.
## What changes
- Captures one MCP runtime in `StepContext`.
- Uses it for tool planning, tool calls, resources, approvals, connector
attribution, and elicitation.
- Publishes replacement runtimes atomically.
- Lets an old runtime live only while an in-flight step or request still
holds its `Arc`.
Most of this diff is mechanical routing from the session-global manager
to `step_context.mcp`; it does not introduce selected-plugin discovery
yet.
## What does not change
- No plugin or extension migration.
- No new MCP cache policy.
- No environment file watching.
- No client sharing between separate managers.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. **This PR:** pin one MCP runtime to each model step.
4. Project selected MCP/app/connector metadata by environment
availability.
5. One end-to-end integration scenario.
|
||
|
jif
|
5eebeb8169 |
Project executor skills through World State (#30088)
## Why A selected executor environment can be unavailable in one model step and ready in the next. The model should see its skills only while that environment is ready, without rescanning stable files on every sample. The product assumption is simple: - an environment ID names one stable logical environment; - the selected root contents do not change during the thread. ## Behavior ```text E1 unavailable -> do not show E1 skills E1 ready -> discover once, cache, show through World State E1 unavailable -> hide skills, keep cache E1 ready again -> reuse cache, show skills again resume -> create a new thread cache and discover again ``` The cache key is the full `SelectedCapabilityRoot`. Availability does not invalidate it; dropping the extension's thread state does. The step supplies the ready selected roots directly. They do not have to be turn environments: ```text turn environment: laptop selected root: worker:/plugins/lint-fix worker ready -> lint-fix skills are visible ``` ## What changes - Keeps executor skill catalogs in the existing skills extension. - Passes the roots resolved as ready for the step into World State contributors. - Loads each ready selected root at most once per thread. - Contributes the executor catalog as the `skills` World State section. - Uses the exact step catalog for explicit skill selection and body reads. - Leaves host and orchestrator skill behavior where it already lives. Taking a step snapshot itself does not add an RPC. Executor filesystem calls happen only on the first discovery of a stable root for that thread. ## What does not change - No filesystem watcher or content-based invalidation. - No retry/generation framework. - No skill runtime migration into core. - No general rewrite of the skills extension. ## Stack 1. Extension-owned World State sections. 2. **This PR:** project cached executor skills through World State. 3. Pin one MCP runtime to each model step. 4. Project selected MCP/app/connector metadata by environment availability. 5. One end-to-end integration scenario. |
||
|
jif
|
c9e6d9783d |
Let extensions contribute World State sections (#30100)
## Why #29856 already owns the durable thread intent and exact environment binding. This PR adds only the small missing extension boundary: an extension can contribute one named World State section, while core still owns persistence, diffing, and model-visible fragment types. This lets skills stay in the skills extension instead of moving their runtime into core. ## Shape ```text extension-owned state | | contribute section id + JSON snapshot + renderer v core World State | | compare with the previous snapshot v no message, or one incremental model-visible update ``` The extension API is deliberately small: ```rust fn contribute_world_state(...) -> Vec<WorldStateSectionContribution> ``` Core adapts the rendered result to `ContextualUserFragment`, records the snapshot, and keeps the existing compaction/resume behavior. ## What changes - Adds extension-owned World State section contributions. - Calls those contributors from the existing per-step World State builder. - Restores durable selected capability roots into extension thread state on resume. - Keeps the actual model-context fragment and rollout machinery in core. ## What does not change - No skill or MCP implementation moves out of its extension. - No new file watcher, generation, or RPC. - No generic migration of existing World State sections. - No change to the stable environment-ID assumption from #29856. ## Example ```text step 1 snapshot: skills = [] step 2 snapshot: skills = [executor-demo:deploy] core asks the skills extension to render only that change. ``` ## Stack 1. **This PR:** let extensions contribute World State sections. 2. Project executor skills through the skills extension. 3. Pin one MCP runtime to each model step. 4. Project selected MCP/app/connector metadata by environment availability. 5. One end-to-end integration scenario. |
||
|
Owen Lin
|
703793c22e |
feat(core, mcp): cache codex_apps tools in memory (#29003)
## Description This makes Codex Apps tool reads use a shared in-memory snapshot instead of rereading the disk cache every time `list_all_tools()` runs. Disk still seeds the cache on startup and gets updated after successful fetches, but it is no longer the live read path. The core change is that `McpManager` now owns a process-scoped `CodexAppsToolsCache`. Codex threads in the same app-server process now share this Codex Apps in-memory tools snapshot. The snapshot is keyed by the Codex home plus the Codex Apps identity: the active Codex auth user/workspace and the effective Codex Apps MCP source config. There's already code to hard-refresh the cache, so we respect it in this PR. ## Local benchmark I ran a local steady-state microbenchmark of the exact repeated Codex Apps cached-tools read this PR removes, using the same real local cache payload in both trees: `3,678,138` bytes and `381` tools. The cache file was already warm in the OS page cache, so this measures same-process reread/deserialization work rather than cold-disk latency or full turn latency. Each run is 25 iterations (mimicking a turn that makes 25 inference calls). | Version | Run 1 | Run 2 | Avg | |---|---:|---:|---:| | `origin/main` disk read + JSON deserialize + `filter_tools` | `50.755 ms` | `52.894 ms` | `51.825 ms` | | This branch in-memory `current_tools` + `filter_tools` | `0.740 ms` | `0.778 ms` | `0.759 ms` | That removes about `51 ms` from each repeated Codex Apps cached-tools read on this machine, roughly `68x` faster for that subpath. It is useful evidence for the hot path this PR changes, but not a claim that every production turn gets `51 ms` faster; end-to-end impact also depends on the rest of `list_all_tools()` and tool-payload construction. This is on my M2 Max macbook, so with a slower disk this would be much worse (and indeed we did see this really blew up turn runtime with a slow disk). |
||
|
rka-oai
|
adccb464d0 |
[codex] impl delivery_mode: current time reminders on response boundaries (#30033)
## Summary - track user-like input and tool-output boundaries in current-time reminder state - gate reminder injection when delivery_mode is after_user_or_tool_output - preserve interval debounce and forced reminders after context-window changes ## Why Training can request reminders only after user or tool-output items while keeping the existing canonical pre-inference history-injection path. ## Validation - just test -p codex-core current_time_reminders_can_follow_only_user_or_tool_outputs - just test -p codex-core current_time_reminders_follow_time_interval_and_persist_in_history - just test -p codex-core current_time_reminder_is_refreshed_after_compaction - just fix -p codex-core |
||
|
rka-oai
|
e8d4a1a411 |
[codex] add current time reminder delivery mode config (#30031)
```python delivery_mode = "any_inference" # default delivery_mode = "after_user_or_tool_output" # new mode ``` ## Validation - just test -p codex-core load_config_resolves_current_time_reminder - just test -p codex-core lock_contains_prompts_and_materializes_features |
||
|
rka-oai
|
cc78903379 |
[codex] current time reminder interval to be set to 0 (#30029)
A zero interval lets callers request a reminder at every otherwise-eligible inference boundary. ## Validation - just test -p codex-core load_config_resolves_current_time_reminder |
||
|
Celia Chen
|
6d9dbacf1a |
feat: add provider-aware model fallback to thread start (#29942)
## Why Helper threads such as task title generation can request a model ID that is valid for the default OpenAI provider but unavailable from the active provider. With Amazon Bedrock, `gpt-5.4-mini` is rejected while the provider static catalog exposes Bedrock model IDs such as `openai.gpt-5.5` and `openai.gpt-5.4`. This causes repeated background 404s and can surface a misleading turn error even when the main turn succeeds. Clients need an explicit way to ask app-server to resolve an unavailable helper model to the active provider default. That fallback must remain limited to providers with an authoritative static catalog so custom or dynamically discovered model IDs are not rewritten based on an incomplete catalog. Fixes #28741. ## What changed - Add the experimental `allowProviderModelFallback` option to `thread/start`, defaulting to `false` to preserve existing behavior. - Thread the option through thread creation and model selection. - When enabled for a static model manager, preserve requested models present in the catalog and replace unavailable models with the provider default. - Continue preserving explicit model IDs for dynamic model managers without fetching a catalog solely to validate them. - Document the new `thread/start` behavior in the app-server API overview. ## Test Temporary test-client harness: ``` ThreadStartParams { model: Some("gpt-5.4-mini".to_string()), allow_provider_model_fallback: true, ..Default::default() } ``` Command: ``` CODEX_HOME=/tmp/codex-bedrock-thread-start-home \ CODEX_E2E_BEDROCK_THREAD_START_ONLY=1 \ ./target/debug/codex-app-server-test-client \ --codex-bin ./target/debug/codex \ -c 'model_provider="amazon-bedrock"' \ send-message-v2 --experimental-api ignored ``` Relevant output: ``` > "method": "thread/start", > "params": { > "model": "gpt-5.4-mini", > "modelProvider": null, > "allowProviderModelFallback": true, > ... > } < "result": { < "model": "openai.gpt-5.5", < "modelProvider": "amazon-bedrock", < ... < } ``` |
||
|
jif
|
8f02973d25 |
Persist selected capability roots and resolve availability per model step (#29856)
## Why
`selectedCapabilityRoots` is durable thread intent: “use this capability
root from environment `worker`.”
The important product assumption is:
> One environment ID always names the same logical executor and stable
contents.
`worker` does not silently change from executor A to an unrelated
executor B. The process-local connection handle for `worker` can still
be replaced while Codex is running, though, for example when
`environment/add` registers a fresh handle for the same logical
environment.
The thread should persist only the stable selection. Each model step
should pair that selection with the exact ready handle captured for that
step.
## The boundary
```text
persisted thread intent
plugin@1 -> environment "worker"
|
| capture the current step
v
model-step view
unavailable, or
plugin@1 + worker's exact captured ready handle
```
The environment ID is the stable identity and cache key. The
`Arc<Environment>` is only a process-local handle retained so consumers
of one model step use the same captured environment. It is never
persisted and it does not imply different environment contents.
## What changes
### Persist the stable selection
Selected roots are written into `SessionMeta` and restored with the
thread. Forked subagents inherit the same selections, including
bounded-history forks.
Only stable data is persisted: root ID, environment ID, and root path.
### Capture readiness together with the exact handle
The environment snapshot records:
```rust
environment_id -> Some(Arc<Environment>) // ready in this step
environment_id -> None // still starting in this step
```
This prevents readiness and execution from coming from different
registry snapshots.
For example:
```text
step snapshot: worker -> handle A, ready
environment/add: worker -> fresh handle B for the same logical environment
current step: plugin@1 still uses captured handle A
```
Without carrying handle A in the snapshot, the resolver could combine “A
was ready” with handle B and treat B as ready before it had finished
starting.
This does not change cache invalidation. Stable capability metadata
remains identified by environment ID and capability root. Replacing a
process-local handle under the same stable environment ID does not
invalidate or rediscover that metadata.
### Resolve availability per model step
- A ready captured environment produces resolved roots using its
captured handle.
- A starting, missing, or failed environment is omitted from that step.
- A selected lazy environment that is outside the turn's captured
environment set is asked to start, and a later step can observe it as
ready.
- No capability files are scanned here.
Transient transport disconnects remain the remote client's reconnect
concern. This PR models initial attachment/readiness; it does not add
live socket-connectivity state.
## Example
```text
thread selection: plugin@1 -> environment "worker"
step 1: worker is starting -> plugin@1 unavailable
step 2: worker is ready -> plugin@1 resolves through worker's captured handle
step 3: fresh local handle -> current step remains pinned; a later step captures its own view
```
Temporary unavailability does not discard the durable selection. Later
PRs can retain stable metadata caches while projecting only currently
available capabilities into model-visible World State.
## Compatibility
The app-server request shape does not change. Older rollouts without
`selected_capability_roots` deserialize to an empty list.
## Stack
1. **This PR:** persist stable selected roots and resolve them through
an exact model-step handle.
2. #29960: cache stable skill metadata and project available skills into
World State.
3. #29946: cache stable plugin declarations and manage the separate live
MCP runtime.
|
||
|
jif
|
b215961a56 |
Support OAuth for HTTP MCP servers from selected executor plugins (#28529)
## Why #28522 routes selected-plugin HTTP MCP traffic through the owning executor, but OAuth bootstrap and refresh still used host-local clients. Executor-only servers therefore cannot complete discovery or login through the same network boundary as the MCP connection. ## What changed - adapt `codex_exec_server::HttpClient` to RMCP 1.8's `OAuthHttpClient` contract - let RMCP own discovery, dynamic registration, PKCE, token exchange, and refresh - route auth status, persisted-token startup, and app-server login through the server runtime while preserving the existing local discovery path - add optional `threadId` to `mcpServer/oauth/login` and echo it in the completion notification - implement RMCP's redirect policy and 1 MiB OAuth response limit over executor HTTP - cover selected-thread OAuth discovery and login through an executor-only route Depends on #28522. |
||
|
sayan-oai
|
f2f80ef442 |
core: make AGENTS.md react to environment changes (#29810)
## Why With deferred executors, a turn can begin before a remote environment attaches. AGENTS.md discovery previously ran only during session setup, so instructions from a later environment never reached the model or the session instruction sources. WorldState persistence has now landed, so this uses the durable model-visible baseline directly instead of carrying a temporary resume/fork compatibility path. ## What - Add an `AgentsMdManager` in `SessionServices` to own host instructions, loaded state, and refresh caching. - When `DeferredExecutor` is enabled, refresh AGENTS.md when attached environment selections change and freeze the result in the corresponding `StepContext`. - Represent AGENTS.md as a persisted WorldState section for every session, with bounded initial, replacement, and removal updates. - Remove duplicate AGENTS.md state and rendering from `SessionConfiguration` and `TurnContext`. - Build initial context, per-request updates, and compaction context from the same step-scoped value. - On resume and fork, compare current instructions with the restored WorldState baseline and inject a replacement exactly once when they differ. Builds on #29833, #29835, and #29837. ## Tests - Covers a remote environment becoming ready mid-turn, with AGENTS.md appearing on the next request exactly once and updating canonical instruction sources. - Covers full, unchanged, replaced, and removed AGENTS.md WorldState rendering. - Covers changed instructions across cold resume and fork without duplicate reinjection. - Covers remote-v2 compaction retaining creation-time instructions in the live session and cold resume appending one replacement when the source changed. - Ran focused `codex-core` AGENTS.md, WorldState, and context-update test suites. |
||
|
Adrian
|
51864b0b4b |
feat: use run agent task auth for inference (#19051)
## Stack This is PR 3 of the simplified HAI single-run-task stack: - [#19047](https://github.com/openai/codex/pull/19047) Agent Identity assertion and task-registration primitives, including the shared run-task helper used by existing Agent Identity JWT auth. - [#19049](https://github.com/openai/codex/pull/19049) Disabled-by-default ChatGPT auth opt-in that provisions/reuses persisted Agent Identity runtime auth and its single run task. - [#19051](https://github.com/openai/codex/pull/19051) Run-scoped provider auth that uses one backend-owned task id for first-party inference and compaction requests. [#19054](https://github.com/openai/codex/pull/19054) collapsed out of the active stack because the simplified design no longer needs a separate background/control-plane task helper. ## Summary This PR moves Agent Identity usage into provider auth resolution. That keeps `AgentAssertion` auth tied to first-party OpenAI provider requests instead of applying a late session-wide override that could affect local, custom, Bedrock, API-key, or external-bearer providers. What changed: - adds a small `ProviderAuthScope` struct carrying the run auth policy and session source needed by provider-scoped auth resolution - lets `Session` opt the existing `ModelClient` into `ChatGptAuth` policy when `use_agent_identity` is enabled, without adding a second model-client constructor - resolves Agent Identity only for first-party OpenAI provider auth paths - uses the persisted run task id from the `AgentIdentityAuth` record to build `AgentAssertion` auth for Responses requests - routes shared request setup through scoped provider auth so unary compact requests use the same run-task assertion path as inference turns - keeps local/custom/Bedrock/env-key/external-bearer provider auth unchanged - lets missing run-task state surface through the existing model-request error path instead of silently falling back to bearer auth This PR intentionally does not create thread-scoped, target-scoped, or background-scoped task identities. The run task is the only task Codex registers in this POC shape. ## Testing - `just test -p codex-model-provider` - `just test -p codex-core client::tests::provider_auth_scope_uses` - `just test -p codex-core remote_compact_uses_agent_identity_assertion` |
||
|
sayan-oai
|
a74771340d |
[3/3] core: replay persisted world state (#29837)
## Why Persisting `WorldState` snapshots and patches is only useful if resume and fork restore that exact comparison baseline. Rebuilding it from `TurnContextItem` loses section state and can either repeat or suppress model-visible updates. This is the third PR in the WorldState persistence stack, built on #29835. ## What - Replay full WorldState snapshots and RFC 7386 patches through the existing rollout reconstruction segments. - Discard state from rolled-back turns and treat compaction as a baseline reset. - Hydrate `ContextManager` from the reconstructed snapshot on resume and fork. - Remove the synthetic `TurnContextItem` to WorldState conversion path. - Leave legacy or malformed rollouts without a baseline so the next update safely emits a full snapshot. ## Testing - `just test -p codex-core world_state` - `just test -p codex-core rollout_reconstruction_tests` - `just fix -p codex-core` - `just test -p codex-core` *(the changed tests passed; the full run also hit unrelated existing/test-environment failures, primarily a missing `test_stdio_server` binary)* |