mirror of
https://github.com/openai/codex.git
synced 2026-10-28 00:03:58 +00:00
2e156cbe31212ccfa9173cb2679c1a201a940b18
154
Commits
| Author | SHA1 | Message | Date | |
|---|---|---|---|---|
 jif
|
2e156cbe31 |
Avoid blocking thread startup on MCP OAuth discovery (#32825)
## Why OAuth metadata discovery for an optional HTTP MCP server can stall, delaying `thread/start` even though that server is not required. ## What changed - Remove eager MCP authentication-status discovery from session and connection manager startup. - When an MCP connection reports an authentication error, inspect configured and stored credentials locally to preserve login and reauthentication failure reasons without making another discovery request. - Continue using server configuration directly for startup error messages and timeout hints. ## Testing Add an app-server regression test with an optional HTTP MCP endpoint that accepts a connection but never responds, and verify that `thread/start` completes without waiting for its configured startup timeout. GitOrigin-RevId: 56fe0188cc1ceaf746b0b65e8b7dd1fd2e13420d |
||
|
Matthew Zeng
|
2f7d89b141 |
Extract connector runtime snapshot management (#32698)
## What changed - Move the Codex Apps tool cache into a reusable `codex-connectors` runtime manager keyed by account and workspace. - Represent cached tools as atomically published snapshots with refresh timestamps while preserving the newest accepted fetch generation. - Harden disk persistence with bounded reads, atomic file replacement, and serialized writes so an older fetch cannot overwrite newer state. ## Testing Add coverage for identity isolation, snapshot timestamps, oversized cache files, atomic replacement, and concurrent persistence ordering. GitOrigin-RevId: 5ea2234469daae3abf54b030244c3251de62ca5a |
||
|
jif
|
6ad0e943cc |
Reduce startup latency for ancestor discovery (#32106)
## Why Serial metadata probes add remote round trips while Codex discovers project instructions, repository skills, and project-root markers during startup. ## What changed - Probe ancestor directories concurrently while preserving discovery order. - Raise the shared find-up probe window to cover typical project hierarchies. - Refresh project instructions in parallel with plugin and skill warmup. ## Testing Add coverage that blocks one metadata request and verifies all expected project-root and `AGENTS.md` ancestor probes have started before it is released. GitOrigin-RevId: 11449837c7a212ec8ed385e7290476e926d93531 |
||
|
raquel-openai
|
b58952b0fa |
fix: forward originator to Codex Apps MCP (#31481)
## Summary - Forward Codexs canonical `originator` header on ChatGPT-hosted Apps and plugin-runtime MCP requests. - Preserve the configured `X-OpenAI-Product-Sku` header. - Cover originator-only and originator-plus-SKU configurations. ## Why Sites project creation is logged downstream of Apps MCP. Production validation found `CODEX_UNKNOWN_DEFAULT` project-created threads that matched `codex_surface=desktop_app` and `originator=Codex Desktop` in `fact_codex_cli` ([query](https://kepler.gateway.data-1.internal.api.openai.org/permalink/H_mVoVPqLQ0)). The hosted Apps MCP configuration forwarded the product SKU but not Codexs canonical originator, so codex-backend could not derive `CODEX_DESKTOP_APP` for those tool calls. ## Validation - `just fmt` - `CARGO_HOME=/private/tmp/codex-cargo-home CARGO_TARGET_DIR=/private/tmp/codex-target just test -p codex-mcp` (106 passed) - `CARGO_HOME=/private/tmp/codex-cargo-home CARGO_TARGET_DIR=/private/tmp/codex-target just fix -p codex-mcp` |
||
|
Owen Lin
|
2342b2c2a6 |
feat(rollout): persist TurnItems for paginated thread rollouts (#30188)
## Description This PR makes new threads with `history_mode = "paginated"` persist `ItemCompleted(item: <turn_item>)` in their rollout JSONL file. Legacy threads keep persisting the existing legacy events. Because the format is selected per thread, a rollout is either legacy or paginated; we do not need to support mixed rollouts containing both representations. This PR depends on [#31473](https://github.com/openai/codex/pull/31473). ## Why Paginated thread history needs stable turn/item IDs and completed item snapshots so the later SQLite projector can materialize appended rollout JSONL without rebuilding the whole thread. Keeping the legacy persistence policy unchanged avoids changing historical rollouts or the readers that still consume them. ## What changed - Made rollout filtering history-mode aware. Paginated threads keep completed canonical `ItemCompleted` events and drop their redundant legacy projections; legacy threads keep the existing event set. - Made forks inherit the source thread history mode, so copied legacy history is never filtered as paginated. - Made paginated threads assign IDs to locally-created response items even when `Feature::ItemIds` is off, and reject streamed output items that arrive without server IDs. - Updated legacy turn replay, rollout list/search, and SQLite metadata extraction to understand completed canonical user-message items. |
||
|
stevenlee-oai
|
555aa79d5a |
[connectors] Refresh codex_apps /ps/mcp auth (#31486)
[Codex Thread 019f2408-dc59-79f2-b245-4c11debd1a61](https://codex-thread-link.openai.chatgpt-team.site/thread/019f2408-dc59-79f2-b245-4c11debd1a61) ## Why Long-lived Codex sessions can outlive the ChatGPT bearer token that was present when the MCP runtime started. The Responses path already recovers from token expiration by refreshing or reloading the shared `AuthManager`. The reserved `codex_apps` hosted-plugin client did not observe that update: `McpConnectionManager` built its `/ps/mcp` HTTP auth once from a `CodexAuth` snapshot, and `auth_provider_from_auth` copied that snapshot bearer into a static `BearerAuthProvider`. After the copied bearer expired, `/ps/mcp` kept sending it even though Responses had a newer token in the same `AuthManager`. The failure occurred before downstream connector execution, so unrelated apps such as Gmail, Slack, and Google Calendar could all fail with the same transport-level `401 token_expired`. This replaces [openai/codex#29474](https://github.com/openai/codex/pull/29474), which was closed for inactivity without being merged. A new long-lived-session report reproduced the same simultaneous `/ps/mcp` expiry pattern across unrelated apps. ## What changed - Add an `AuthManager`-backed request-header provider in `codex-model-provider`. It keeps an `Arc<AuthManager>` and reads `auth_cached()` for each outbound request, so the next `/ps/mcp` call sees a token refreshed by the existing Responses/auth-recovery flow. - Scope that provider to the startup account, ChatGPT user, and workspace identity. Same-identity token reloads are followed; an account switch emits no ambient auth until account-scoped MCP state is rebuilt. - Have `McpConnectionManager` construct the dynamic provider only for the reserved `codex_apps` registration used by the hosted-plugin `/ps/mcp` path. | MCP path | Auth behavior after this change | | --- | --- | | Reserved `codex_apps` hosted-plugin `/ps/mcp` | Read current same-identity auth from the shared `AuthManager` per request | | `codex_apps` with `CODEX_CONNECTORS_TOKEN` | Keep the environment bearer-token override | | User-configured/direct MCP registrations | Keep their existing configured auth path | ## Non-goals - No plugin-service changes. - No downstream Slack, Gmail, Calendar, or other connector OAuth/link-refresh changes. - No auth UI changes. - No behavior change for user-configured/direct MCP registrations. - No new `/ps/mcp`-initiated token refresh; this makes `/ps/mcp` observe refreshes already performed through the shared `AuthManager`. ## Tests - `just test -p codex-model-provider` - Covers same-identity token reloads and refuses a changed startup identity. - `just test -p codex-mcp` - `just test -p codex-core mcp_auth_refresh` - Creates the reserved hosted-plugin `codex_apps` `/ps/mcp` client before the shared `AuthManager` changes, updates that same manager through its public external-auth path, performs a real `tools/call`, and asserts the request uses the current bearer. |
||
|
Michael Bolin
|
6afcf26d5d |
core: route Responses API through system proxy (#31335)
## Why `features.respect_system_proxy` already routes authentication traffic through the OS proxy APIs, but it does not affect the primary inference path. That leaves users behind OS-managed proxies unable to send normal Responses API requests even after login succeeds. This PR is the first product-path migration onto the route-aware transport introduced in #31323 and refined in #31331. It also establishes the construction pattern for later migrations: the effective feature state is resolved once into a required HTTP client factory rather than represented by an optional per-call setting. The scope remains limited to the two HTTP Responses endpoints; WebSockets, model discovery, memories, realtime, and file uploads remain follow-up migrations. ## What changed - Replace the optional proxy marker with an explicit `OutboundProxyPolicy::{ReqwestDefault, RespectSystemProxy}` and a required `HttpClientFactory`. The policy has no default, and the lower-level route-aware reqwest builder is now private. - Have `Config` construct the factory from the effective feature state and require every `ModelClient` constructor to receive it. There is no optional setter or implicit `None` fallback. - Build HTTP clients for `/responses` and `/responses/compact` with `ClientRouteClass::Api`, using the complete destination URL so PAC rules can make URL-specific decisions. - Layer route-aware selection onto Codex's existing default headers, Cloudflare cookie store, custom CA handling, and sandbox no-proxy behavior. - Add an integration test that loads `features.respect_system_proxy` through `config.toml`, creates a real Codex session, and verifies that both a normal Responses turn and remote compaction reach an isolated local proxy. ## Review guide 1. `http-client/src/outbound_proxy.rs` defines the mandatory policy/factory boundary and keeps route resolution private. 2. `core/src/config/mod.rs`, `core/src/session/session.rs`, and `core/src/client.rs` show the compile-time invariant: effective config creates the factory, and `ModelClient` cannot be constructed without one. 3. `login/src/auth/default_client.rs` preserves existing default-client behavior while accepting the required factory for migrated routes. 4. `core/src/client.rs` switches only streaming Responses and remote compaction HTTP transports to the API route class. 5. `core/tests/suite/responses_api_system_proxy.rs` is the behavioral regression boundary. Its Linux subprocess deliberately sets the CGI marker that disables reqwest's implicit environment-proxy handling, so the test fails if session wiring or either Responses call site falls back to the default client. ## Test plan - `cargo check --tests -p codex-http-client -p codex-login -p codex-core` - `just test -p codex-login` - `just test -p codex-core respect_system_proxy_feature_resolves_enabled` - Existing `compact_uses_bearer_after_agent_identity_session_fallback` coverage passes with the new transport construction. - New Linux integration coverage: `responses_and_compact_use_enabled_system_proxy` - `just bazel-lock-check` --- [//]: # (BEGIN SAPLING FOOTER) Stack created with [Sapling](https://sapling-scm.com). Best reviewed with [ReviewStack](https://reviewstack.dev/openai/codex/pull/31335). * #31342 * __->__ #31335 |
||
|
ashwinnathan-openai
|
775ef7dcc7 |
[codex] Support sequential cutoff reasoning summaries (#31306)
## Summary When enabled for the OpenAI provider, Codex sends `stream_options.reasoning_summary_delivery = "sequential_cutoff"` on HTTP and WebSocket requests, including prewarm, and renders completed summary sections from `reasoning_summary_text.done`. Flag-off and non-OpenAI behavior is unchanged. ## Expected rollout ```text reasoning 0 added summary 0 done summary 1 done summary 2 starts summary 2 cancelled / incomplete reasoning 0 done <-- cancel summary 2 work and mark it incomplete message 1 added message 1 text streams message 1 completed ``` Depends on [openai/openai#1096660](https://github.com/openai/openai/pull/1096660). |
||
|
Channing Conger
|
84fe70c30e |
elicitations: Move to shared ElicitationService (#30627)
## Why Code-mode tool results could return to the model while an MCP elicitation was still waiting for user input. This differed from parallel tool calling and could let the model continue before the user resolved the request. We need one session-level view of outstanding elicitations so tool runtimes can consistently hold results until every pending elicitation is resolved. ## What changed - Added a counted, session-owned ElicitationService with RAII registrations. - Registered both core-originated and server-originated MCP elicitations with the service. - Migrated out-of-band elicitation tracking and unified exec timeout pausing to the shared service. - Made code-mode functions.exec and functions.wait capture their runtime result normally, then hold it before returning while an elicitation is outstanding. - Kept terminate: true immediate; only its result is held. - Preserved model-visible wall time across the elicitation hold. - Kept the behavior session-scoped, with concurrent elicitations holding the pause until all registrations are released. |
||
|
sayan-oai
|
9c5be7e1d5 |
Make plugin guidance react to environment readiness (#30223)
## Why Generic plugin guidance is currently emitted only with initial context from host plugin state. An executor-selected plugin can become available later in the same turn, making its skills and tools usable without ever telling the model how plugin capabilities should be used. ## What - project every ready selected plugin package, including skill-only plugins - carry plugin availability with the exact MCP runtime projection while preserving MCP manager reuse when servers and connectors are unchanged - move generic plugin guidance from the static initial-context path into persisted World State - recognize legacy and retained plugin fragments so resume and compaction do not duplicate guidance ## Testing - `just test -p codex-mcp-extension` - `just test -p codex-core plugins_instructions` - `just test -p codex-core plugin_availability_change_reuses_the_mcp_manager` - `just test -p codex-app-server --test all selected_capabilit` |
||
|
Owen Lin
|
5267e805fb |
feat(app-server): add history_mode to thread (#29927)
## Description This PR adds a new `historyMode = "legacy" | "paginated"` to `Thread`. This will be stored in `SessionMeta` in the JSONL rollout file and as a new column in the SQLite thread_metadata table, and exposed on `thread/start` and on the `Thread` object in app-server. ## What changed - Added canonical `ThreadHistoryMode` with `legacy` and `paginated`, defaulting old and new SessionMeta to `legacy`. - Carried `history_mode` through core session config, ThreadStore stored metadata, local/in-memory stores, rollout metadata extraction, and the existing SQLite `threads` table. - Added experimental `historyMode` to app-server v2 `Thread` and `thread/start`. - Made paginated stored threads metadata-discoverable but unsupported for legacy full-history reads, `load_history`, live resume, and create paths. - Regenerated app-server schema fixtures and added protocol/state/thread-store/app-server coverage for persistence and fail-closed behavior. ## Compatibility floor Because users may be running various versions of Codex binaries on the same machine (TUI, Codex App, etc.), we will need to establish a compatibility floor for upcoming paginated threads, which will change how thread storage reads and writes work. The overall plan here: ``` Release N: - Add historyMode to SessionMeta / Thread / SQLite metadata. - Teach binaries to understand paginated threads. - If a binary sees `historyMode="paginated"` but does not support the paginated contract, it refuses to resume/mutate the thread. - Default remains `"legacy"`. Release N+1: - First-party clients start opting into paginated threads where appropriate. - Internal dogfood / staged rollout. - Measure old-client usage and paginated-thread unsupported errors. Release N+2: - Only after Release N+ is overwhelmingly deployed, make paginated the default. - Accept that a small tail of N-1-or-older binaries may not understand paginated threads. ``` The important behavior change is fail-closed handling for a binary that encounters a persisted `paginated` thread before it knows how to fully support paginated history. In app-server, if a thread is `paginated`, we will: - allow metadata-only discovery paths like `thread/list` and `thread/read(includeTurns=false)`, so clients can still see the thread and inspect its `historyMode` - reject legacy full-history/live-thread paths like `thread/read(includeTurns=true)` and `thread/resume` with an unsupported JSON-RPC error - avoid silently treating an unknown or future `historyMode` as `legacy` Under the hood, the ThreadStore layer also rejects legacy operations that would need to load or replay the full thread history for a paginated thread. That gives us the behavior we want for Release N: future paginated threads are visible, but this binary fails closed instead of trying to operate on them as if they were legacy threads. |
||
|
Channing Conger
|
7d8906b478 |
[codex] wire process-owned code mode host into core (#30142)
## Summary - add the `code_mode_host` feature flag and select `ProcessOwnedCodeModeSessionProvider` in `CodeModeService` when enabled - initialize code-mode sessions lazily so a missing host reports a tool error without failing thread startup - resolve `codex-code-mode-host` beside the running Codex binary by default while preserving `CODEX_CODE_MODE_HOST_PATH` as an override - add unit and end-to-end coverage for host resolution and graceful missing-host behavior ## Why This wires the process-owned session client from #30112 into the core service behind an opt-in rollout gate. Packaged Codex installations can place the helper in the same `bin` directory as the main executable without relying on `PATH`, while development and custom installations can continue to override the helper path. ## Stack - Depends on #30112 - Base branch: `cconger/process-owned-session-runtime-4-client` ## Validation Build `codex` and `codex-code-mode-host` `CODEX_CODE_MODE_HOST_PATH="$PWD/target/debug/codex-code-mode-host" ./target/debug/codex --enable code_mode_host` |
||
|
jif
|
fb8598df3f |
Keep MCP elicitation routable across runtime refreshes (#30127)
## Why
An MCP tool call can still be waiting for an elicitation response when
an environment update replaces the thread's MCP runtime.
Before this change:
```text
runtime A starts a tool call and asks the user
environment becomes ready, so runtime B is published
client answers the prompt through runtime B
runtime B cannot find runtime A's pending responder
```
The response is lost and the original tool call stays blocked.
## What changed
All MCP runtimes for one thread now share a small elicitation router:
```text
runtime A ---\
shared router: response token -> exact pending responder
runtime B ---/
```
When Codex surfaces an MCP elicitation, it assigns a unique opaque
response token. The router records which pending request owns that
token. A replacement runtime reuses the same router, so the latest
runtime can deliver a response to a request started by the previous
runtime.
The Codex-owned token also prevents two runtime connections that reuse
the same MCP server request ID from receiving each other's responses.
This does not retain or search old MCP managers. Only the pending
responder map is shared.
## Covered scenario
The integration test exercises the complete failure mode:
1. A thread starts while its selected environment is still unavailable.
2. A configured MCP server starts a tool call and asks the client for
input.
3. The environment becomes ready, causing Codex to publish a replacement
MCP runtime.
4. The client answers the original prompt after the replacement.
5. The original tool call receives that answer and completes.
A focused routing test also creates two runtimes with the same server
request ID and verifies that each response reaches the exact request
that emitted its token.
## Scope
This PR changes only elicitation response routing across MCP runtime
replacement. It does not change when runtimes are rebuilt, which
environments contribute MCP configuration, or how environment
availability is detected.
|
||
|
jif
|
3095ea9c3d |
Project selected plugin runtime by environment availability (#30093)
## Why
Selected plugin metadata is stable, but MCP processes are live runtime
state. They need different lifetimes:
- the MCP extension caches manifest, MCP, and connector declarations for
each stable selected root;
- each model step projects that cached metadata through the roots that
resolved as ready for that exact step;
- the MCP manager is rebuilt only when that availability projection
changes.
This matches executor skills: both features consume the same resolved
step roots instead of inferring readiness from the turn's selected
environments.
## Behavior
```text
E1 not ready for this step
-> no E1 MCP servers or connectors
-> cached plugin metadata stays in ext/mcp
E1 becomes ready
-> reuse cached metadata
-> publish one MCP runtime containing E1 capabilities
same ready roots on the next step
-> reuse the exact runtime; no rediscovery and no MCP restart
resume
-> create new extension thread state and a new MCP runtime
```
All model-facing consumers use the same step snapshot:
```text
resolved selected roots
|
v
extension MCP/connector projection
|
v
{ MCP config, connector snapshot, MCP manager }
|
+-> advertise model tools
+-> build app/connector tools
+-> execute MCP calls
```
## Cache contract
The existing MCP extension owns a cache keyed by the full
`SelectedCapabilityRoot`:
```rust
let state = thread_store.get_or_init(SelectedExecutorPluginMcpState::default);
```
The cache lives with extension thread state. Environment availability
filters projection but does not invalidate metadata. Resume creates new
thread state. There is no file watcher or executor generation because
contents behind a stable environment/root are assumed stable.
## What changes
- Keeps executor plugin discovery and cached metadata in `ext/mcp`.
- Caches MCP and connector declarations together per selected root.
- Uses the step's already-resolved capability roots, including lazy
environments that are not turn environments.
- Reuses the current MCP runtime when the ready-root projection is
unchanged.
- Uses the same step MCP manager and connector snapshot for
model-visible tools and execution.
- Resolves direct thread-scoped MCP requests from the current
selected-root projection.
## Deliberately out of scope
- `app/list` remains based on the latest global host-plugin state; this
PR does not make its response or notifications thread-specific.
- `required = true` startup semantics do not apply to delayed executor
MCP activation.
- No filesystem/content invalidation.
- No transport-disconnect watcher.
- No executor generations or environment replacement semantics.
- No client sharing across complete manager replacements.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. Pin one MCP runtime to each model step.
4. **This PR:** project selected MCP and connector state from
extension-owned metadata.
5. Integration coverage for selected capability availability and resume.
## Verification
-
`selected_plugin_servers_use_managed_requirements_for_the_selected_root_id`
- The stacked integration PR covers unavailable to ready activation,
unchanged-runtime reuse, skills, MCP tools, connector attribution, and
cold resume.
|
||
|
jif
|
ee9e0f6387 |
Pin MCP runtimes to model steps (#30101)
## Why
An MCP refresh can replace the session's current manager while a model
step is still running. The step must execute calls through the same
manager whose tools it advertised.
## Boundary
```text
current session MCP runtime
|
| capture once for this model step
v
StepContext.mcp
- exact MCP config
- exact connection manager
- exact runtime environment context
```
```rust
pub struct McpRuntimeSnapshot {
config: Arc<McpConfig>,
manager: Arc<McpConnectionManager>,
runtime_context: McpRuntimeContext,
}
```
## Example
```text
step A captures runtime A and advertises A's tools
refresh publishes runtime B
step A tool call -> runtime A
next step -> runtime B
```
Capturing the snapshot is only an `Arc` clone. It does not restart MCPs
or make an RPC.
## What changes
- Captures one MCP runtime in `StepContext`.
- Uses it for tool planning, tool calls, resources, approvals, connector
attribution, and elicitation.
- Publishes replacement runtimes atomically.
- Lets an old runtime live only while an in-flight step or request still
holds its `Arc`.
Most of this diff is mechanical routing from the session-global manager
to `step_context.mcp`; it does not introduce selected-plugin discovery
yet.
## What does not change
- No plugin or extension migration.
- No new MCP cache policy.
- No environment file watching.
- No client sharing between separate managers.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. **This PR:** pin one MCP runtime to each model step.
4. Project selected MCP/app/connector metadata by environment
availability.
5. One end-to-end integration scenario.
|
||
|
jif
|
c9e6d9783d |
Let extensions contribute World State sections (#30100)
## Why #29856 already owns the durable thread intent and exact environment binding. This PR adds only the small missing extension boundary: an extension can contribute one named World State section, while core still owns persistence, diffing, and model-visible fragment types. This lets skills stay in the skills extension instead of moving their runtime into core. ## Shape ```text extension-owned state | | contribute section id + JSON snapshot + renderer v core World State | | compare with the previous snapshot v no message, or one incremental model-visible update ``` The extension API is deliberately small: ```rust fn contribute_world_state(...) -> Vec<WorldStateSectionContribution> ``` Core adapts the rendered result to `ContextualUserFragment`, records the snapshot, and keeps the existing compaction/resume behavior. ## What changes - Adds extension-owned World State section contributions. - Calls those contributors from the existing per-step World State builder. - Restores durable selected capability roots into extension thread state on resume. - Keeps the actual model-context fragment and rollout machinery in core. ## What does not change - No skill or MCP implementation moves out of its extension. - No new file watcher, generation, or RPC. - No generic migration of existing World State sections. - No change to the stable environment-ID assumption from #29856. ## Example ```text step 1 snapshot: skills = [] step 2 snapshot: skills = [executor-demo:deploy] core asks the skills extension to render only that change. ``` ## Stack 1. **This PR:** let extensions contribute World State sections. 2. Project executor skills through the skills extension. 3. Pin one MCP runtime to each model step. 4. Project selected MCP/app/connector metadata by environment availability. 5. One end-to-end integration scenario. |
||
|
Owen Lin
|
703793c22e |
feat(core, mcp): cache codex_apps tools in memory (#29003)
## Description This makes Codex Apps tool reads use a shared in-memory snapshot instead of rereading the disk cache every time `list_all_tools()` runs. Disk still seeds the cache on startup and gets updated after successful fetches, but it is no longer the live read path. The core change is that `McpManager` now owns a process-scoped `CodexAppsToolsCache`. Codex threads in the same app-server process now share this Codex Apps in-memory tools snapshot. The snapshot is keyed by the Codex home plus the Codex Apps identity: the active Codex auth user/workspace and the effective Codex Apps MCP source config. There's already code to hard-refresh the cache, so we respect it in this PR. ## Local benchmark I ran a local steady-state microbenchmark of the exact repeated Codex Apps cached-tools read this PR removes, using the same real local cache payload in both trees: `3,678,138` bytes and `381` tools. The cache file was already warm in the OS page cache, so this measures same-process reread/deserialization work rather than cold-disk latency or full turn latency. Each run is 25 iterations (mimicking a turn that makes 25 inference calls). | Version | Run 1 | Run 2 | Avg | |---|---:|---:|---:| | `origin/main` disk read + JSON deserialize + `filter_tools` | `50.755 ms` | `52.894 ms` | `51.825 ms` | | This branch in-memory `current_tools` + `filter_tools` | `0.740 ms` | `0.778 ms` | `0.759 ms` | That removes about `51 ms` from each repeated Codex Apps cached-tools read on this machine, roughly `68x` faster for that subpath. It is useful evidence for the hot path this PR changes, but not a claim that every production turn gets `51 ms` faster; end-to-end impact also depends on the rest of `list_all_tools()` and tool-payload construction. This is on my M2 Max macbook, so with a slower disk this would be much worse (and indeed we did see this really blew up turn runtime with a slow disk). |
||
|
jif
|
8f02973d25 |
Persist selected capability roots and resolve availability per model step (#29856)
## Why
`selectedCapabilityRoots` is durable thread intent: “use this capability
root from environment `worker`.”
The important product assumption is:
> One environment ID always names the same logical executor and stable
contents.
`worker` does not silently change from executor A to an unrelated
executor B. The process-local connection handle for `worker` can still
be replaced while Codex is running, though, for example when
`environment/add` registers a fresh handle for the same logical
environment.
The thread should persist only the stable selection. Each model step
should pair that selection with the exact ready handle captured for that
step.
## The boundary
```text
persisted thread intent
plugin@1 -> environment "worker"
|
| capture the current step
v
model-step view
unavailable, or
plugin@1 + worker's exact captured ready handle
```
The environment ID is the stable identity and cache key. The
`Arc<Environment>` is only a process-local handle retained so consumers
of one model step use the same captured environment. It is never
persisted and it does not imply different environment contents.
## What changes
### Persist the stable selection
Selected roots are written into `SessionMeta` and restored with the
thread. Forked subagents inherit the same selections, including
bounded-history forks.
Only stable data is persisted: root ID, environment ID, and root path.
### Capture readiness together with the exact handle
The environment snapshot records:
```rust
environment_id -> Some(Arc<Environment>) // ready in this step
environment_id -> None // still starting in this step
```
This prevents readiness and execution from coming from different
registry snapshots.
For example:
```text
step snapshot: worker -> handle A, ready
environment/add: worker -> fresh handle B for the same logical environment
current step: plugin@1 still uses captured handle A
```
Without carrying handle A in the snapshot, the resolver could combine “A
was ready” with handle B and treat B as ready before it had finished
starting.
This does not change cache invalidation. Stable capability metadata
remains identified by environment ID and capability root. Replacing a
process-local handle under the same stable environment ID does not
invalidate or rediscover that metadata.
### Resolve availability per model step
- A ready captured environment produces resolved roots using its
captured handle.
- A starting, missing, or failed environment is omitted from that step.
- A selected lazy environment that is outside the turn's captured
environment set is asked to start, and a later step can observe it as
ready.
- No capability files are scanned here.
Transient transport disconnects remain the remote client's reconnect
concern. This PR models initial attachment/readiness; it does not add
live socket-connectivity state.
## Example
```text
thread selection: plugin@1 -> environment "worker"
step 1: worker is starting -> plugin@1 unavailable
step 2: worker is ready -> plugin@1 resolves through worker's captured handle
step 3: fresh local handle -> current step remains pinned; a later step captures its own view
```
Temporary unavailability does not discard the durable selection. Later
PRs can retain stable metadata caches while projecting only currently
available capabilities into model-visible World State.
## Compatibility
The app-server request shape does not change. Older rollouts without
`selected_capability_roots` deserialize to an empty list.
## Stack
1. **This PR:** persist stable selected roots and resolve them through
an exact model-step handle.
2. #29960: cache stable skill metadata and project available skills into
World State.
3. #29946: cache stable plugin declarations and manage the separate live
MCP runtime.
|
||
|
jif
|
b215961a56 |
Support OAuth for HTTP MCP servers from selected executor plugins (#28529)
## Why #28522 routes selected-plugin HTTP MCP traffic through the owning executor, but OAuth bootstrap and refresh still used host-local clients. Executor-only servers therefore cannot complete discovery or login through the same network boundary as the MCP connection. ## What changed - adapt `codex_exec_server::HttpClient` to RMCP 1.8's `OAuthHttpClient` contract - let RMCP own discovery, dynamic registration, PKCE, token exchange, and refresh - route auth status, persisted-token startup, and app-server login through the server runtime while preserving the existing local discovery path - add optional `threadId` to `mcpServer/oauth/login` and echo it in the completion notification - implement RMCP's redirect policy and 1 MiB OAuth response limit over executor HTTP - cover selected-thread OAuth discovery and login through an executor-only route Depends on #28522. |
||
|
sayan-oai
|
f2f80ef442 |
core: make AGENTS.md react to environment changes (#29810)
## Why With deferred executors, a turn can begin before a remote environment attaches. AGENTS.md discovery previously ran only during session setup, so instructions from a later environment never reached the model or the session instruction sources. WorldState persistence has now landed, so this uses the durable model-visible baseline directly instead of carrying a temporary resume/fork compatibility path. ## What - Add an `AgentsMdManager` in `SessionServices` to own host instructions, loaded state, and refresh caching. - When `DeferredExecutor` is enabled, refresh AGENTS.md when attached environment selections change and freeze the result in the corresponding `StepContext`. - Represent AGENTS.md as a persisted WorldState section for every session, with bounded initial, replacement, and removal updates. - Remove duplicate AGENTS.md state and rendering from `SessionConfiguration` and `TurnContext`. - Build initial context, per-request updates, and compaction context from the same step-scoped value. - On resume and fork, compare current instructions with the restored WorldState baseline and inject a replacement exactly once when they differ. Builds on #29833, #29835, and #29837. ## Tests - Covers a remote environment becoming ready mid-turn, with AGENTS.md appearing on the next request exactly once and updating canonical instruction sources. - Covers full, unchanged, replaced, and removed AGENTS.md WorldState rendering. - Covers changed instructions across cold resume and fork without duplicate reinjection. - Covers remote-v2 compaction retaining creation-time instructions in the live session and cold resume appending one replacement when the source changed. - Ran focused `codex-core` AGENTS.md, WorldState, and context-update test suites. |
||
|
Adrian
|
51864b0b4b |
feat: use run agent task auth for inference (#19051)
## Stack This is PR 3 of the simplified HAI single-run-task stack: - [#19047](https://github.com/openai/codex/pull/19047) Agent Identity assertion and task-registration primitives, including the shared run-task helper used by existing Agent Identity JWT auth. - [#19049](https://github.com/openai/codex/pull/19049) Disabled-by-default ChatGPT auth opt-in that provisions/reuses persisted Agent Identity runtime auth and its single run task. - [#19051](https://github.com/openai/codex/pull/19051) Run-scoped provider auth that uses one backend-owned task id for first-party inference and compaction requests. [#19054](https://github.com/openai/codex/pull/19054) collapsed out of the active stack because the simplified design no longer needs a separate background/control-plane task helper. ## Summary This PR moves Agent Identity usage into provider auth resolution. That keeps `AgentAssertion` auth tied to first-party OpenAI provider requests instead of applying a late session-wide override that could affect local, custom, Bedrock, API-key, or external-bearer providers. What changed: - adds a small `ProviderAuthScope` struct carrying the run auth policy and session source needed by provider-scoped auth resolution - lets `Session` opt the existing `ModelClient` into `ChatGptAuth` policy when `use_agent_identity` is enabled, without adding a second model-client constructor - resolves Agent Identity only for first-party OpenAI provider auth paths - uses the persisted run task id from the `AgentIdentityAuth` record to build `AgentAssertion` auth for Responses requests - routes shared request setup through scoped provider auth so unary compact requests use the same run-task assertion path as inference turns - keeps local/custom/Bedrock/env-key/external-bearer provider auth unchanged - lets missing run-task state surface through the existing model-request error path instead of silently falling back to bearer auth This PR intentionally does not create thread-scoped, target-scoped, or background-scoped task identities. The run task is the only task Codex registers in this POC shape. ## Testing - `just test -p codex-model-provider` - `just test -p codex-core client::tests::provider_auth_scope_uses` - `just test -p codex-core remote_compact_uses_agent_identity_assertion` |
||
|
Shijie Rao
|
df1199fddb |
[codex] Add Ultra reasoning effort (#29899)
## Why Ultra should be one user-facing reasoning selection for work that benefits from both maximum reasoning and proactive multi-agent delegation. Without it, clients must coordinate maximum reasoning with the experimental `multiAgentMode` setting, even though the inference backend still expects its existing `max` effort value. This change makes reasoning effort the source of truth: clients select `ultra`, core derives proactive multi-agent behavior when the turn is eligible for multi-agent V2, and inference requests continue to use the backend-compatible `max` value. ## What changed - Add `ultra` as a first-class reasoning effort and preserve model-catalog ordering when exposing it to clients. - Convert `ultra` to `max` at the inference request boundary, including Responses HTTP/WebSocket requests, startup prewarm, compaction, and memory summarization. - Derive effective multi-agent mode per turn from effective reasoning effort: - eligible multi-agent V2 + `ultra` → `proactive` - eligible multi-agent V2 + any other effort → `explicitRequestOnly` - V1 or otherwise ineligible sessions → no multi-agent mode instruction - Keep the derived effective mode in turn context history so successive turns can emit a developer-message update only when the effective mode changes. - Remove selected multi-agent mode from core session configuration, turn construction, thread settings, resume/fork restoration, and subagent spawn plumbing. Subagents inherit reasoning effort and derive their own effective mode. - Retain the experimental app-server `multiAgentMode` fields for wire compatibility while marking them deprecated. Request values are accepted but ignored; compatibility response fields report `explicitRequestOnly`. - Display Ultra in the TUI using the order supplied by `model/list`. ## Validation - `just test -p codex-core ultra_reasoning_uses_max_for_requests` - `just test -p codex-tui model_reasoning_selection_popup` |
||
|
alexsong-oai
|
1acb722e8a |
Support thread-level originator overrides (#29477)
## Why Work(TPP) threads can be launched from the Desktop app, but if they all keep the Desktop app's default originator then downstream attribution cannot distinguish local Work launches from cloud-backed Work launches. `thread/start.serviceName` already carries that launch signal, while `SessionMeta.originator` is the durable thread-level value that survives resume and fork. This change converts the Desktop Work service names into an effective originator at thread creation time, persists that originator with the thread, and keeps using it for later model requests and memory writes. ## What changed - Map `CODEX_WORK_LOCAL` and `CODEX_WORK_CLOUD` service names to per-thread originators, while preserving `CODEX_INTERNAL_ORIGINATOR_OVERRIDE` as the highest-precedence override. - Persist the effective originator in `SessionMeta.originator`, read it back on resume/fork, and inherit the parent originator for subagent spawns when there is no persisted session metadata. - Handle truncated `SpawnAgentForkMode::LastNTurns` forks by falling back to the live parent originator when the forked history no longer includes `SessionMeta`. - Thread the per-thread originator through Responses headers, websocket/compaction request paths, thread-store creation, rollout metadata, and memory stage-one telemetry. ## Verification - `just test -p codex-core agent::control::tests::spawn_thread_subagent_inherits_parent_originator_without_fork agent::control::tests::spawn_thread_subagent_fork_last_n_turns_inherits_parent_originator_without_session_meta thread_manager::tests::originator_override_precedes_service_name_remapping` - `just test -p codex-core agent::control::tests::resume_thread_subagent_restores_stored_metadata_and_effective_multi_agent_mode` - `just test -p codex-memories-write` - `just fix -p codex-core -p codex-memories-write` - `git diff --check` |
||
|
Michael Bolin
|
01f89c8c59 |
core: persist initial context window metadata (#29519)
## Why PR #29494 made context-window IDs visible to the model by wrapping the token-budget window payload in `<context_window>`, but rollout JSONL consumers still could not see the initial window identity by tailing the session file. Compacted rollout items carry window IDs only after compaction has happened, so a session with no compaction had no durable JSONL record for window 0. This change gives tailing consumers a stable initial-window record at session creation time. ## What Changed - Added `session_meta.context_window.window_id` for the initial context-window identity. - `CreateThreadParams` now requires `initial_window_id: String`, so thread-store callers cannot accidentally create new threads without window-0 metadata. - Live thread creation derives the persisted initial window ID from the same `AutoCompactWindowIds` used to initialize `SessionState`, keeping runtime state and JSONL metadata aligned. - Rollout reconstruction uses `session_meta.context_window.window_id` as the initial-window fallback and derives `window_number = 0`, `first_window_id = window_id`, and `previous_window_id = None` internally. - Fork reconstruction intentionally uses the same rollout reconstruction path; consumers that need to distinguish copied initial-window metadata can use the rollout `thread_id`. - Legacy compactions without `window_number` still use compaction-count fallback accounting instead of being reset to window 0 by the initial-window fallback. - Compacted rollout metadata still takes precedence once compaction records exist, preserving the richer chain fields there. ## JSONL Shape Real rollout JSONL is one object per line. This example is expanded for readability, but shows the new initial `session_meta.context_window` record followed by the existing compacted rollout item shape that also carries window IDs: ```jsonl { "timestamp": "2026-06-22T12:00:00.000Z", "type": "session_meta", "payload": { "session_id": "<THREAD_ID>", "id": "<THREAD_ID>", "timestamp": "2026-06-22T12:00:00.000Z", "cwd": "/repo", "originator": "codex", "cli_version": "0.0.0", "source": "cli", "model_provider": "<MODEL_PROVIDER>", "context_window": { "window_id": "<INITIAL_WINDOW_ID>" } } } ... { "timestamp": "2026-06-22T12:34:56.000Z", "type": "compacted", "payload": { "message": "<COMPACTION_SUMMARY>", "replacement_history": [ "..." ], "window_number": 1, "first_window_id": "<INITIAL_WINDOW_ID>", "previous_window_id": "<INITIAL_WINDOW_ID>", "window_id": "<NEXT_WINDOW_ID>" } } ``` The nested `context_window` object is intentional: it gives rollout consumers a stable namespace for context-window metadata while only writing the non-derivable initial `window_id`. For the initial window, `window_number`, `first_window_id`, and `previous_window_id` are derived internally instead of being written to the rollout. ## Verification - `just test -p codex-protocol` - `just test -p codex-rollout recorder_materializes_on_flush_with_pending_items` - `just test -p codex-core reconstruct_history` - `just test -p codex-core record_initial_history_reconstructs_forked_transcript` - `just test -p codex-thread-store` - `just test -p codex-state` - `just test -p codex-app-server thread_read_returns_summary_without_turns` - `just test -p codex-rollout persistence_metrics` |
||
|
Dylan Hurd
|
2cf2a6a844 |
chore(core) rm AskForApproval::OnFailure (#28418)
## Summary Deletes the OnFailure variant of the `AskForApproval` enum. This option has been deprecated since #11631. ## Testing - [x] Tests pass |
||
|
Ahmed Ibrahim
|
a22e3d0b82 |
Remove redundant Codex Apps manager flag (#29518)
## Why Codex Apps server admission is already decided before `McpConnectionManager` is constructed. `effective_mcp_servers` and `effective_mcp_servers_from_configured` remove the server when the apps feature or required authentication is unavailable, so storing the same decision on the manager duplicates state that can drift from the effective server map. ## What changed - Remove `host_owned_codex_apps_enabled` from `McpConnectionManager` and its constructor. - Identify the host-owned Codex Apps server by its reserved server name once it is present in the effective server map. - Remove the now-unused flag calculations and constructor arguments from production and test callsites. |
||
|
jgershen-oai
|
15f448d8b0 |
[codex] Start the guardian child session when parent session is started (#27982)
## Why The first auto-review currently creates its Guardian child session on demand, adding avoidable latency before the review can begin. Creating the ordinary Guardian child during parent-session initialization lets that child use the existing session startup WebSocket prewarm before the first escalation. This does not introduce a Guardian-specific prewarm mechanism. ## What changed - initialize the existing Guardian review-session manager owned by `Session` when a thread starts with auto-review enabled and an approval policy that routes to Guardian - use the standard Guardian child-session construction and the existing session startup WebSocket prewarm - preserve the existing reuse-key invalidation and lazy creation fallback when startup initialization fails or the effective review configuration changes - add an integration test that verifies normal root-session startup emits a Guardian `generate=false` prewarm request ## Benchmark I compared release builds against main. Each prompt first ran a non-escalated `sleep 3`, then requested an escalated marker command. | binary | count | avg Guardian duration | median Guardian duration | avg Guardian TTFT | |---|---:|---:|---:|---:| | origin-main | 10 | 4008.7 ms | 3949.5 ms | 3746.5 ms | | session-fix | 10 | 2865.0 ms | 2594.0 ms | 2492.7 ms | Guardian duration fell by 28.5% and Guardian TTFT fell by 33.5%. These measurements cover Guardian review latency; they do not measure parent thread-start latency. |
||
|
jif
|
c03742ca0a |
Simplify multi-agent mode controls (#29324)
## Why Multi-agent delegation policy was split across `multiAgentMode`, `features.multi_agent_mode`, and `usage_hint_enabled`. These controls could disagree: a requested mode could be downgraded by the feature flag, and disabling usage hints also disabled mode instructions. Some clients also need multi-agent tools without adding delegation-policy text to model context. The previous two-mode API could not express that directly. ## What changed `multiAgentMode` is now the only live delegation-policy control: | Mode | Behavior | | --- | --- | | `none` | Keep multi-agent tools available without adding mode instructions. | | `explicitRequestOnly` | Only delegate after an explicit user request. | | `proactive` | Delegate when parallel work materially improves speed or quality. | - new threads default to `explicitRequestOnly`; omitting the mode on later turns keeps the current value - thread start, resume, fork, and settings responses always report the concrete current mode instead of `null` - mode selection remains sticky across turns and resume - usage-hint text no longer controls whether mode instructions apply - `features.multi_agent_mode` and `usage_hint_enabled` remain accepted as ignored compatibility settings so existing configs continue to load - app-server documentation and generated schemas describe the three-mode API ## Tests - `just test -p codex-core multi_agent_mode` - `just test -p codex-core multi_agent_v2_config_from_feature_table` - `just test -p codex-core spawn_agent_description` - `just test -p codex-features` - `just test -p codex-app-server-protocol` - `just test -p codex-app-server multi_agent_mode` |
||
|
jif
|
6d15bb3d17 |
Persist session IDs across thread resume (#29327)
## Summary
A cold-resumed subagent kept its durable thread ID but could receive a
new session ID, splitting one agent tree across multiple sessions after
a restart.
Persist the root session ID in every rollout `SessionMeta`, carry it
through thread creation, and restore it before initializing the resumed
`Session` and `AgentControl`.
## Behavior
For a nested agent tree:
```text
root session R
parent thread P
child thread C
```
The child rollout stores:
```text
session_id: R
parent_thread_id: P
id: C
```
After a cold resume, the child still belongs to root session `R` while
its immediate parent remains `P`. The integration coverage uses distinct
values for all three IDs so it catches restoring the session from
`parent_thread_id`.
## Legacy rollouts
Previous rollouts have `id` but no `session_id`. `SessionMetaLine`
deserialization treats a missing `session_id` as `id`, keeping those
files readable, listable, and resumable. When a legacy subagent is
resumed through its root, that synthesized child ID no longer overrides
the inherited root-scoped `AgentControl`. New rollouts always persist
the explicit root session ID.
|
||
|
Shijie Rao
|
fc8c6b7384 |
Add per-turn multi-agent mode (#28685)
## Why Multi-agent v2 currently carries an explicit-request-only delegation rule in its static usage hint. That provides a safe default, but it prevents clients from selecting proactive delegation per turn without changing static guidance or rewriting prior model context. This change makes delegation mode a session selection that can be updated through `turn/start`, while deriving the effective model-visible mode separately for each turn. Eligible multi-agent v2 turns remain explicit-request-only unless proactive mode is both selected and enabled. ## What changed - Add the experimental `turn/start.multiAgentMode` parameter with `explicitRequestOnly` and `proactive` values. Omission retains the loaded session's current optional selection. - Add the default-off `features.multi_agent_mode` feature gate. Eligible multi-agent v2 turns use the selected mode when enabled; an unset selection or disabled gate resolves to `explicitRequestOnly`. - Treat mode prompting as inapplicable for multi-agent v1 and other unsupported session configurations, producing no multi-agent mode developer message rather than rejecting the turn. - Move the explicit-request-only rule out of the static v2 usage hint and into a bounded, tagged developer context fragment. - Emit the effective mode in initial context and only when that effective mode changes on later turns. - Persist the effective mode in `TurnContextItem` as the durable baseline for resume and context-update comparisons. Historical rollout items are not rewritten. Later mode developer messages establish the current rule incrementally. ## Not covered - Initial selection through `thread/start` and selected-mode reporting from thread lifecycle/settings APIs; those are isolated in the stacked #28792. - A TUI control or slash command for selecting the mode. - Persisting a preferred mode to `config.toml`; selection remains session/turn scoped. - Changes to multi-agent concurrency limits, tool availability, or model catalog capability declarations. - Rewriting historical rollout prompt items. Cold resume restores the latest persisted effective mode when available while leaving historical developer messages intact. ## Verification - `CARGO_INCREMENTAL=0 just test -p codex-core multi_agent_mode` - Focused app-server coverage verifies that `turn/start.multiAgentMode` produces proactive developer instructions for an eligible v2 turn. ## Stack Followed by #28792, which adds `thread/start` initialization and lifecycle/settings observability. |
||
|
sayan-oai
|
45a133bae0 |
[2/3] core: track starting environments in snapshots (#28683)
## Why Remote environments may still be resolving when Codex creates a session or turn. Waiting for the existing all-or-nothing environment snapshot can hold startup until the selected environment is usable. Behind the default-off `deferred_executor` feature, let callers take a useful snapshot immediately: completed environments remain available normally, while unfinished environments are reported without blocking startup. With the feature disabled, snapshots preserve the existing blocking behavior. Depends on #28674. ## What changed - Store one ordered list of selected environments in `ThreadEnvironments`. Each selection owns one shared resolution that produces its complete `TurnEnvironment`. - Start new resolutions in the background with `remote_handle()`, allowing snapshots and the future wait tool to share the same result while cancellation follows the retained handles. - Make `snapshot()` a read-only operation: nonblocking snapshots collect completed resolutions and retain handles for unfinished ones, while blocking snapshots await every resolution. - Replace completed failed resolutions from the current manager entry and log when failed environments are omitted. - Return attached and starting environments as a point-in-time view, and count starting environments when deciding whether a snapshot is local-only. - Keep existing consumers attached-only. `to_selections()` derives from attached environments, so child threads do not inherit an environment that is still starting. ## Test plan - `just test -p codex-core environment_selection` - `just test -p codex-core deferred_executor_reaches_model_before_remote_environment_is_ready` ## Landing note Keep `deferred_executor` disabled for slow-starting executors until configurable `environment/add` connection timeouts and caller support land. When enabled, an environment that attaches after session startup may remain absent from environment-derived model context, tools, instructions, skills, and related state until follow-up refresh work lands. |
||
|
pakrym-oai
|
f00f93d8c0 |
[codex] Assign response item IDs when recording history (#28814)
## Why Client-created response items enter history without IDs, so their identity is lost across rollout persistence and resume. IDs should be assigned once at the history-recording boundary, while IDs returned by the server must remain unchanged. The Responses API validates item IDs using type-specific prefixes. Locally generated IDs therefore use the matching prefix plus a hyphenated UUIDv7, keeping them valid while distinguishable from server-generated IDs. Because this changes persisted history and provider request shapes, the behavior is opt-in behind the under-development `item_ids` feature. Compaction triggers remain request controls whose API shape does not accept an ID. ## What changed - Register the disabled-by-default `item_ids` feature and expose it in `config.schema.json`. - Make supported optional `ResponseItem` IDs serializable and expose them in the generated app-server schemas. - When `item_ids` is enabled, assign an ID during conversation-history preparation if an item has no ID. - Generate type-prefixed, hyphenated UUIDv7 IDs using the Responses API item conventions. - Preserve existing server IDs without rewriting them. - Persist assigned IDs in rollouts and include them in subsequent Responses requests. - Remove the unsupported ID field from `CompactionTrigger` and document why it has no ID. - Add integration coverage for enabled ID persistence, preservation of server IDs, and omission of generated IDs while the feature is disabled. `prepare_conversation_items_for_history` is the single response-item ID allocation boundary. ## Test plan - `just test -p codex-features` - `just test -p codex-core response_item_ids_persist_across_resume_and_preserve_server_ids` - `just test -p codex-core non_openai_responses_requests_omit_item_turn_metadata` - `just test -p codex-core resize_all_images_prepares_failures_before_history_insertion` - `just test -p codex-protocol` - `just test -p codex-app-server-protocol` - `just test -p codex-api azure_default_store_attaches_ids_and_headers` |
||
|
xl-openai
|
e83b7841b0 |
[codex] Reuse parsed plugin skills during session startup (#28844)
## Summary - Preserve raw plugin skill-root snapshots in the matching loaded-plugin cache entry, keyed by the effective plugin root identity including namespace. - Pass those snapshots through `SkillsLoadInput` as an optional preload, so session startup reuses plugin parsing while ordinary skill loads pass `None`. - Keep plugin skill loading cohesive: the existing loaders accept the optional snapshots directly, and uncached or marketplace-detail paths do not create a cache. ## Why Plugin discovery already parses plugin skills to determine available capabilities. Cold session startup then scanned and parsed the same roots again while building the skills snapshot. This solves the same duplicate-work problem as #28623 while keeping ownership narrow: `PluginsManager` creates and owns `PluginSkillSnapshots` only for its loaded-plugin cache entry; `SkillsService` consumes an optional clone. Entry replacement or clearing naturally drops the snapshots, with no separate generation, capacity policy, or watcher coupling. ## Validation - `cargo clippy -p codex-core-skills --all-targets -- -D warnings` - `just test -p codex-core-plugins skills_service_reuses_skills_parsed_during_plugin_load` - `just test -p codex-core-skills namespaces_plugin_skills_using_provided_namespace` - `just fmt` |
||
|
rka-oai
|
752ed90d78 |
current time reminders impl for system clock (varlatency 2/n) (#28824)
Stacked on #28822. ## Summary - add a host-injectable current-time provider with a built-in system implementation - record UTC developer reminders in history immediately before due model requests - keep cadence state per session and force a refresh after compaction This does NOT include the app server client <-> server clock logic. This PR is only for the reminder message & system clock that will be used in prod. ## Testing - `just test -p codex-core varlatency_` - `just clippy -p codex-core -p codex-app-server -p codex-mcp-server -p codex-thread-manager-sample` - `just fmt` |
||
|
Gabriel Peal
|
21a599fa56 |
Support openai/form extended form elicitations (#27500)
# Summary Allow App Server clients to opt into `openai/form` MCP elicitations. |
||
|
jif
|
0318381762 |
Replace SkillsManager with SkillsService (#28705)
## Why Host skill discovery was still exposed as a manager even though it is a process-owned service shared by sessions, the app-server catalog, and file-watcher invalidation. The skills extension also consumed an ad hoc loaded-skills wrapper instead of a named immutable snapshot. ## What changed - replace `SkillsManager` with concrete `SkillsService` - make the service cache and return immutable `HostSkillsSnapshot` values - migrate the skills extension host provider to the snapshot boundary - migrate app-server catalog, watcher, and invalidation paths to the service This keeps the service limited to host discovery, caching, roots, and invalidation. Catalog rendering and invocation remain extension responsibilities for the next stacked change. |
||
|
Adam Perry @ OpenAI
|
f8850cab1d |
app-server: preserve target-native environment cwd (#28146)
## Why app-server may run on a different OS from the selected exec-server environment. Parsing that environment’s cwd with the Codex host’s path rules prevents thread startup. ## What Carry environment cwd values as `LegacyAppPathString` at the app-server boundary and `PathUri` internally. Existing tool-call schemas and relative-path behavior stay host-native; remaining local-only consumers convert explicitly and leave follow-up TODOs. The Wine integration test verifies app-server can start a thread and complete an ordinary turn with a Windows environment cwd from Linux. ## Validation - `bazel test //codex-rs/core/tests/remote_env_windows:smoke-test --test_output=errors` - focused app-server environment-selection and protocol schema tests - scoped Clippy for `codex-core` and `codex-app-server-protocol` |
||
|
pakrym-oai
|
1e015884c5 |
[codex] Use local environment for user shell commands (#28163)
## Why User shell commands still read the legacy turn cwd and session shell even though execution context is now owned by selected turn environments. App-server also defines `thread/shellCommand` as a local-host escape hatch, so it must use an available local environment even when a remote environment is primary. ## What changed - Add `ResolvedTurnEnvironments::local()` to find the selected local environment. - Resolve the user shell command cwd and shell from that local `TurnEnvironment`. - Emit the standard `shell is unavailable in this session` error when no selected local environment or resolved local shell is available. - Add an integration test covering `/shell` without a local environment. ## Test plan - `just test -p codex-core user_shell_command_without_local_environment_emits_error` |
||
|
pakrym-oai
|
022f1221e8 |
[codex] Bind shell snapshots to retained thread environments (#28421)
## Why Shell snapshots are currently session-scoped even though shell and cwd are properties of a selected turn environment. That makes snapshot refresh depend on separate session-cwd plumbing, prevents retained environments from retaining their snapshot work, and can make snapshot construction use a different shell than command execution. This follows #27955 by making the retained thread-environment service own environment snapshot lifecycles. Session configuration remains the requested selection state, while `ThreadEnvironments` remains the source of successfully resolved environments. ## What changed - Configure the shell-snapshot builder before initial environment resolution. - Start each local environment snapshot task when its `TurnEnvironment` is built and retain that shared task while environment ID and cwd still match. - Inherit retained environment snapshots into spawned child threads. - Carry the selected `TurnEnvironment` through shell runtimes so snapshot construction and command execution use the same environment-specific shell and cwd. - Load project instructions and warm plugins/skills after initial environment resolution. - Continue decoding invalid UTF-8 instruction files lossily without emitting a startup warning. - Keep requested selections in `SessionConfiguration`; failed or duplicate resolutions only affect the resolved environment snapshot. ## Validation - `cargo check -p codex-core --tests` - `just test -p codex-home instructions` (6 passed) - Focused environment, instruction, shell-snapshot, and user-shell tests (84 passed) - Focused shell-snapshot, user-shell, and unified-exec tests (126 passed; two event-timing tests passed on retry) |
||
|
pakrym-oai
|
9728992fab |
[codex] retain resolved environments across turns (#27955)
## Why Selected execution environments are thread-scoped resources, but startup and turn construction repeatedly resolved their IDs and working directories. That discarded existing environment handles and shell metadata even when a selection had not changed. Session configuration updates also need to affect future turns without changing the resolved environment set already captured by a running turn. ## What changed - Create a `ThreadEnvironments` service inside `Codex` from the spawned `EnvironmentManager` and raw environment selections, then store it on `SessionServices`. - Split service construction from `update_selections`, allowing session configuration updates to mutate the resolved set in place. - Retain an existing `TurnEnvironment` when its environment ID and working directory match; resolve only added or changed selections and remove selections that are no longer present. - Normalize duplicate IDs by keeping the first selection and skip individual selections that fail to resolve instead of rejecting the entire update. - Give each `TurnContext` a cloned `TurnEnvironmentSnapshot`, so later session configuration updates affect future turns without rewriting an active turn. - Reuse the service-owned environment manager and resolved snapshot for startup work, MCP initialization, and child-thread spawning instead of flowing resolved environments through spawn arguments. ## Test plan - `cargo check -p codex-core --tests` - `just test -p codex-core environment_selection` - `just test -p codex-core turn_environments` - `just test -p codex-core session_update_settings_does_not_rewrite_sticky_environment_cwds` - `just test -p codex-core default_turn_does_not_overlay_legacy_fallback_cwd_onto_stored_thread_environments` |
||
|
mchen-oai
|
af99f6a72f |
core: cache the tool search handler per session (#27258)
## Why Tool router construction rebuilds the deferred-tool BM25 index during session initialization and before each sampling continuation, even when the searchable tool metadata is unchanged. Local profiling measured `append_tool_search_executor` at roughly 113 ms per continuation, making repeated index construction the largest measured router-building cost. ## What changed - Add a session-scoped `ToolSearchHandlerCache` so continuations and user turns can reuse the existing handler. - Key reuse on the complete ordered `Vec<ToolSearchInfo>`, rebuilding when searchable text, loadable tool specs, source metadata, or ordering changes. - Build handlers outside the cache lock and recheck before publishing them, avoiding holding the mutex during index construction. ## Verification - `cache_reuses_identical_search_infos_and_rebuilds_changed_inputs` covers exact cache reuse and invalidation when the ordered search metadata changes. - Local rollout profiling showed the initial router build populating the cache and unchanged later continuations reusing it: - uncached: 118 ms median across 14 spans from 3 rollouts - cached: 4 ms median across 12 spans from 3 rollouts |
||
|
pakrym-oai
|
1aa3e432cc |
[codex] simplify shell snapshot ownership (#27756)
## Why Shell snapshot lifecycle state was split between `Shell` and `SessionServices`: `Shell` carried the receiver while session code exposed and forwarded the raw sender. That coupled shell identity to mutable snapshot state and made refresh, inheritance, and file lifetime harder to reason about. ## What changed - make each `Arc<ShellSnapshot>` represent one cwd-specific snapshot generation - store the active generation in `SessionServices` with `ArcSwapOption` - have construction start the background build and expose only a cwd-validated snapshot path - use `ShellSnapshotFile` ownership to delete snapshot files automatically - pass snapshot paths explicitly to shell runtimes instead of storing snapshot state on `Shell` - preserve inherited and in-flight generations by pinning their `Arc` while they are in use ## Test plan - `cargo check -p codex-core --lib` - `just test -p codex-core 'shell_snapshot::tests'` - `just test -p codex-core shell_command_snapshot_still_intercepts_apply_patch` - `just test -p codex-core shell_snapshot_deleted_after_shutdown_with_skills` |
||
|
jif
|
6d9df687a5 |
skills: hide orchestrator skills with a local executor (#28333)
## Why App-server threads without a local executor need orchestrator-owned skills from the hosted `codex_apps` MCP server. Threads with the local executor already discover installed skills from the local filesystem. After the orchestrator skill provider was enabled for every app-server thread, local-executor threads also received the hosted skill catalog and the `skills.list` and `skills.read` tools. This changed the existing local behavior and could expose a second hosted copy of a skill that was already installed locally. ## What changed - Expose the thread's selected execution environments to extensions at thread startup. - Enable orchestrator skills only when the reserved local environment is not selected. - Apply that decision consistently to hosted skill catalog discovery, explicit skill injection, and the `skills.list` and `skills.read` tools. ## Verification - The existing no-executor app-server test continues to verify hosted skill discovery, invocation, and child-resource reads. - A new app-server test verifies that local-executor threads do not receive hosted skill context or `skills.*` tools. |
||
|
Celia Chen
|
9915d34684 |
feat: use encrypted local secrets for MCP OAuth (#27541)
## Summary - store MCP OAuth credentials in the configured auth credential backend - support encrypted-local OAuth storage, including legacy keyring migration - propagate the credential backend through MCP refresh, session, CLI, and app-server paths ## Stack 1. #27504 — config and feature flag 2. #27535 — auth-specific secret namespaces 3. #27539 — encrypted CLI auth storage 4. this PR — encrypted MCP OAuth storage This is a parallel review stack; the original #17931 remains unchanged. ## Tests - `just test -p codex-rmcp-client` (the transport round-trip test passed after building the required `codex` binary and retrying) - `just test -p codex-mcp` - `just test -p codex-app-server refresh_config_uses_latest_auth_keyring_backend` - `just test -p codex-core refresh_mcp_servers_is_deferred_until_next_turn` - `just test -p codex-cli mcp` - `just fix -p codex-rmcp-client -p codex-mcp -p codex-core -p codex-cli -p codex-app-server -p codex-protocol` - `just bazel-lock-check` |
||
|
Adam Perry @ OpenAI
|
52a50aec70 |
sandboxing: migrate cwd inputs to PathUri (#27816)
## Why Sandbox cwd values can cross app-server and exec-server host boundaries. They should retain URI semantics until the receiving host validates them instead of being interpreted early as native paths. ## What - Carry `PathUri` through filesystem sandbox contexts, sandbox commands, and transform inputs. - Convert command and policy cwd once in `SandboxManager::transform`, then keep launch requests native. - Preserve sandbox cwd over remote filesystem transport and reject non-native URIs without fallback. - Cache paired native/URI turn-environment cwd values during migration, with immutable access to keep them synchronized. - Extend existing protocol, forwarding, transform, and core runtime tests. |
||
|
jif
|
693082f3c4 |
Make MCP server contributions thread-scoped (#27670)
## Why `selectedCapabilityRoots` belongs to one thread, but MCP contributors previously received only the global Codex config. That left no clean way for a selected executor capability to contribute MCP servers to its own thread. ## What this PR does - Gives MCP contributors a small context containing the config and, for a running thread, its frozen host-seeded inputs. - Uses the same thread inputs during startup, status queries, refreshes, and skill dependency checks. - Keeps threadless MCP operations and the existing hosted Apps behavior unchanged. - Adds coverage showing that two threads resolve independent registrations and that later lifecycle mutations do not change the frozen MCP inputs. This PR does not discover plugin manifests, add MCP servers, or launch anything new. It only establishes the thread-scoped registration boundary. ## Follow-ups - Resolve selected executor plugin roots through their owning environment filesystem. - Convert their stdio MCP declarations into environment-bound registrations and add an executor MCP end-to-end test. ## Verification - `just fmt` - `cargo check --tests -p codex-protocol -p codex-extension-api -p codex-mcp-extension -p codex-core -p codex-app-server` Tests and Clippy were not run. |
||
|
pakrym-oai
|
be338ee9a2 |
[codex] resolve environment shell metadata eagerly (#27709)
## Why Turn construction passed resolved environments through several layers while leaving the environment shell unresolved. As a result, model-visible environment context could fall back to the session shell instead of reporting the selected remote environment's shell. Resolve environment metadata at the turn-context boundary so each turn carries the shell that belongs to its selected environment. Keep request validation in app-server, where invalid selections can be returned as straightforward JSON-RPC errors without coupling core turn construction to that policy. ## What changed - resolve environment selections eagerly in `new_turn_context_from_configuration` - store the full resolved `Shell` on each `TurnEnvironment` - simplify the now-redundant resolved-environment constructor plumbing - keep duplicate and unknown-environment validation as a small app-server preflight - add a remote-environment integration test that runs a full `test_codex` turn and verifies the model-visible environment message reports `bash` ## Testing - `cargo check -p codex-core --test all -p codex-app-server` - `remote_test_env_exposes_bash_shell_to_model` on the Linux remote-executor harness |
||
|
Tom
|
b2cc01fc7c |
[codex] Move persistence policy application into ThreadStore (#27318)
Move the application of the persistence policy into the thread store, so thread stores can get raw append items rather than canonical append items. This will enable store-specific projections over the raw input items. |
||
|
Owen Lin
|
14df0e8833 |
core: Consolidate Responses API Codex metadata (#27122)
## What Introduce a `CodexResponsesMetadata` struct that defines all the core metadata we send to Responses API. Example fields are `thread_id`, `turn_id`, `window_id`, etc. Going forward, `client_metadata["x-codex-turn-metadata"]` will be the canonical way Codex sends metadata to Responses API across both HTTP and websocket transports. For now, we continue to emit the existing top-level HTTP headers and top-level `client_metadata` fields from the same `CodexResponsesMetadata` struct for compatibility reasons. Also, app-server clients who specify additional `responsesapi_client_metadata` via `turn/start` and `turn/steer` will have those fields merged into `client_metadata["x-codex-turn-metadata"]`, but cannot override the reserved fields that core uses (i.e. the fields in `CodexResponsesMetadata`). ## Why Responses API request instrumentation is the source of truth for downstream Codex analytics that join requests by Codex IDs such as session, thread, turn, and context window. Before this change, those values were assembled through several request-specific paths: HTTP request bodies, websocket handshake headers, websocket `response.create` payloads, compaction requests, and the rich `x-codex-turn-metadata` envelope all had their own wiring. That made metadata propagation easy to drift across API-key/direct Responses API requests, ChatGPT-auth/proxied requests, websocket requests, and compaction requests. It also made additions like `window_id` error-prone because a field could be added to one transport projection but missed in another. ## What changed - Added `CodexResponsesMetadata` as the core-owned snapshot for Codex metadata sent to ResponsesAPI. - Render `client_metadata["x-codex-turn-metadata"]`, flat `client_metadata` projections, and direct compatibility headers from that same snapshot. - Include the known Codex-owned fields in the turn metadata blob, including installation/session/thread/turn/window IDs, request kind, lineage, sandbox/workspace metadata, timing, and compaction details. - Treat app-server `responsesapi_client_metadata` as enrichment for the Codex turn metadata blob while preventing those extras from overriding Codex-owned fields. - Use the same metadata path for normal turns, websocket prewarm, local compaction, remote v1 compaction, and remote v2 compaction. - Keep websocket connection-only preconnect metadata separate so handshakes carry compatibility identity headers without inventing a fake turn metadata blob. ## Verification - `cargo check -p codex-core` - `just fix -p codex-core` |
||
|
jif
|
4a5a676499 |
Resolve MCP server registrations through a catalog (#27634)
## Why MCP servers currently come from user config, local plugins, compatibility Apps synthesis, and host extensions. Those sources were composed by mutating a shared map, leaving registration identity, precedence, removal, and provenance implicit in assembly order. Before adding executor-owned MCPs, Codex needs one durable resolution boundary above `McpConnectionManager`. This PR introduces that boundary while preserving current server configuration, policy, and runtime behavior. Executor-scoped registrations and explicit policy layers remain follow-ups. ## What changed - Add typed `McpServerRegistration` inputs and an immutable `ResolvedMcpCatalog` in `codex-mcp`. - Retain each registration's complete `McpServerConfig`, including its environment binding, while recording its source and provenance. - Preserve the existing structural precedence between plugin, config, compatibility, and ordered extension sources. - Resolve equal-precedence actions by contribution order; provenance IDs are used only for diagnostics and cannot affect the winner. - Preserve extension removals and the existing name-scoped `enabled = false` veto. - Report same-tier conflicts with every contender and the final catalog outcome, including whether the winning action registers or removes the server. - Require MCP contributors to provide a stable diagnostic identity. - Derive materialized server maps and plugin ownership from the resolved catalog. `McpConnectionManager`, transport startup, tool calls, and resource routing continue to consume the same effective `McpServerConfig` values. ## Scope This PR does not add new MCP capabilities or change user-visible behavior. It does not add executor plugin discovery, thread-scoped registrations, dynamic refresh generations, or new user/managed policy semantics. ## Verification - Added focused catalog coverage for source precedence, complete configuration preservation, disabled vetoes, plugin ownership, contribution-order tie breaking, removal outcomes, and conflict diagnostics. - Extended hosted Apps coverage for ordered extension removal and Apps-disabled hosts with and without the hosted extension installed. - `cargo check -p codex-mcp --tests -p codex-extension-api -p codex-core` |