## What changed
- Stream V3 agent message deltas to `delegation.context.append` at 200 ms intervals instead of waiting for each completed message.
- Send commentary without a prefix and label final or phase-less V1/V3 output with `"Agent Final Message":`.
- Bound streamed output while preserving both its beginning and final tail when truncation is required.
- Remove `codexResponseHandoffPrefix` from `thread/realtime/start`; unknown fields from older clients remain ignored.
## Testing
- Cover incremental V3 commentary and final output, Unicode-safe truncation, V1/V3 phase labeling, and legacy request compatibility.
GitOrigin-RevId: 633c18b5eb9a4f5756a264b63b7bfc185b073951
## Why
Changes to the selected working directory can select different `AGENTS.md`
instructions. These updates need to reach the model even when the deferred
executor feature is disabled.
## What changed
- Refresh environment readiness and `AGENTS.md` before capturing each step.
- Record model-visible world-state changes for every session instead of gating
them on the deferred executor feature.
- Preserve the initial environment context without duplicating it.
## Testing
- Cover `AGENTS.md` replacement after a working-directory change.
- Verify that initial environment context remains unique with the deferred
executor both enabled and disabled.
GitOrigin-RevId: ea5dbe590cc049170259e7eccb6a3346fb2ed8e6
## Why
With deferred execution enabled, rebuilding a step context from the latest
thread environment snapshot could apply a settings update to a turn that was
already in progress.
## What changed
- Preserve each turn's captured environment selections when creating later
step contexts.
- Refresh only the readiness of environments that were already starting, while
retaining their original selection order.
- Apply updated environment selections to the next turn.
## Testing
Add coverage for changing the selected workspace while a turn is paused,
verifying that the active turn keeps its original working directory and the
next turn uses the new one.
GitOrigin-RevId: c6f4388c843384ef045b7abdbc303dcfca11d230
## What changed
- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.
## Testing
- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.
GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
## Why
Queue-only child-agent updates that arrive around a final answer should not
restart sampling, but they must remain available to the next turn. Explicitly
injected response items still need to reopen the current turn.
## What changed
- Defer queue-only inter-agent mail after a final answer without treating it as
same-turn pending input.
- Re-enable current-turn mailbox delivery when the model requests a follow-up,
a stop hook continues the turn, or a response item is explicitly injected.
- Check for turn-triggering mailbox work after clearing the completed active
turn so pending work can start safely.
## Testing
Added coverage that queued child mail waits for the next user turn and that an
injected response item after a final answer triggers a follow-up request.
GitOrigin-RevId: aa81d00707943284b7ab069b60970e547908c895
## Why
Token-budget sessions need a final opportunity to preserve important state before the current context is compacted.
## What changed
- When `auto_compact_fallback_prompt` is configured, reserve `auto_compact_fallback_buffer_tokens` beyond the base auto-compaction limit.
- At the base limit, inject the fallback prompt once as a developer message while keeping the normal tool surface available. Continue reporting zero base-window tokens during this reserve.
- Roll over when the reserve or full model context is exhausted, and skip the fallback when a new context was already requested.
## Testing
Added coverage for using the fallback reserve, rolling over after it is exhausted, and bypassing the fallback for an explicit `new_context` request.
GitOrigin-RevId: 32983ecadc5cb4927e15ecd7b1e896cb52a9ad3b
## What changed
- Track connected and disconnected states across initial remote exec-server connections and reconnection attempts.
- Emit experimental `thread/environment/connected` and `thread/environment/disconnected` app-server notifications for each thread selecting the environment. Each payload identifies the thread and environment; current state is not replayed when a thread starts.
- Stop forwarding connection events when an environment selection is removed or replaced.
## Testing
- Cover connection, disconnection, reconnection, shared-environment notifications, and replacement of a selected environment.
GitOrigin-RevId: 5dd767372363c4a2a8319fc16164be117d5bd20c
## What changed
- Convert filtered MCP tool metadata into `CoreToolRuntime` instances before constructing the tool router.
- Pass those runtimes through the shared tool-planning path instead of maintaining separate direct and deferred MCP tool lists.
- Preserve MCP visibility, app policy filtering, and direct versus deferred exposure behavior.
GitOrigin-RevId: 67f7fbeecc2992318c859e038cd97938833911db
## Why
Clients that account for individual upstream Responses API calls need the exact
usage for each completion rather than accumulated or estimated thread usage.
## What changed
- Add the opt-in `rawResponse/completed` app-server notification with the
`threadId`, `turnId`, `responseId`, and upstream `usage` payload.
- Emit the transient event for normal turns and compaction requests when
`experimentalRawEvents` is enabled. The event is not persisted or replayed,
and `usage` is `null` when the upstream response omits it.
- Export the notification in the app-server schemas and document its behavior.
## Testing
- Cover exact token breakdowns for turn and compaction completions.
GitOrigin-RevId: 49040a060bf1784c1ff1dd62112efb4a778428c4
## Why
`TurnContext` stored model and reasoning settings both as resolved turn fields and inside `CollaborationMode`, requiring the copies to stay synchronized when the model changed.
## What changed
- Store the mode and collaboration-mode developer instructions directly on `TurnContext`.
- Reconstruct `CollaborationMode` from those values plus the turn's current model and reasoning effort when callers need it.
- Read the mode directly for plan-mode checks, lifecycle events, and telemetry.
GitOrigin-RevId: b8fccafdd5d5cbccc84bc70d542588087ddf0093
## What changed
- Add a `ResponseItemId` type that generates item-specific prefixes with UUIDv7 suffixes and use it across response items.
- Keep deserialization permissive for legacy histories, but omit empty or unprefixed item IDs from HTTP and WebSocket requests.
- Export the new type in the generated TypeScript protocol schema.
## Testing
- Cover prefixed ID generation, legacy deserialization, prefix recognition, and outbound request filtering.
GitOrigin-RevId: 0209fe430c826d0ae88bc4648652c4ebf390c4a4
## Description
This PR makes new threads with `history_mode = "paginated"` persist
`ItemCompleted(item: <turn_item>)` in their rollout JSONL file.
Legacy threads keep persisting the existing legacy events. Because the
format is selected per thread, a rollout is either legacy or paginated;
we do not need to support mixed rollouts containing both
representations.
This PR depends on [#31473](https://github.com/openai/codex/pull/31473).
## Why
Paginated thread history needs stable turn/item IDs and completed item
snapshots so the later SQLite projector can materialize appended rollout
JSONL without rebuilding the whole thread.
Keeping the legacy persistence policy unchanged avoids changing
historical rollouts or the readers that still consume them.
## What changed
- Made rollout filtering history-mode aware. Paginated threads keep
completed canonical `ItemCompleted` events and drop their redundant
legacy projections; legacy threads keep the existing event set.
- Made forks inherit the source thread history mode, so copied legacy
history is never filtered as paginated.
- Made paginated threads assign IDs to locally-created response items
even when `Feature::ItemIds` is off, and reject streamed output items
that arrive without server IDs.
- Updated legacy turn replay, rollout list/search, and SQLite metadata
extraction to understand completed canonical user-message items.
## Description
This PR moves hook prompts onto the canonical `TurnItem` lifecycle in
core.
Stop hooks now record their `ResponseItem` through the existing
lifecycle path, which emits `ItemStarted` and `ItemCompleted`.
App-server consumes those events directly instead of deriving a hook
prompt from `RawResponseItem`.
## Why
Hook prompts were the only `ThreadItem` app-server synthesized from
`RawResponseItem`. This brings them in line with other core-owned turn
items while preserving legacy rollout replay.
## What changed
- Route stop-hook prompts through
`record_response_item_and_emit_turn_item`.
- Materialize canonical hook prompts in `ThreadHistoryBuilder`.
- Remove `RawResponseItem` to `ThreadItem` synthesis while preserving
legacy rollout replay.
- Add focused coverage for lifecycle emission and canonical and legacy
history materialization.
Follow-up to #30226.
## Why
#30226 makes Apps World State inspect the MCP tool list, while
tool-router construction reads the same list again later in the sampling
request. `list_all_tools()` walks the MCP clients and may reconnect or
wait for tools, so doing that work twice adds latency and lets context
and tool construction observe different MCP states for one request.
## What
- Add a lazy MCP tool snapshot to `StepContext`.
- Reuse that snapshot for Apps World State and tool-router construction.
- Let each new `StepContext` refresh naturally for the next sampling
request, without manager-level caching or invalidation.
## Testing
- `just test -p codex-core apps_instructions`
- `just test -p codex-core
apps_guidance_appears_after_background_recovery_within_a_turn`
## Why
Pre-sampling compaction intentionally uses the previous turn's model
when the compaction compatibility hash changes or when switching to a
model with a smaller context window. This keeps compaction aligned with
the settings that produced the history, but it can block the next turn
when a resumed ChatGPT thread still references a model slug that has
since been retired. The Codex backend rejects that compaction request
before the user's currently selected model gets a chance to sample.
This PR lets those threads recover without changing previous-model
compaction behavior for API-key authentication or custom providers. It
is stacked on #31316, which is a behavior-preserving extraction of the
individual remote compaction attempts; this PR contains the fallback
behavior.
## What changed
- For automatic previous-model compaction, capture the selected model's
request context when using ChatGPT authentication with the OpenAI
provider and the selected model differs from the previous model.
- If the previous-model attempt returns an `InvalidRequest`, retry
compaction once with the selected model for both `/responses/compact`
and Responses Compaction V2.
- Complete history processing, lifecycle events, and token accounting
with the context of the model that successfully compacted the thread.
- If the fallback also fails, return the original previous-model error
so the retry does not change the user-visible failure.
- Record fallback attempts with reason, implementation, and outcome
telemetry.
- Leave API-key authentication, custom providers, same-model turns, and
non-`InvalidRequest` failures on their existing paths.
## Testing
- `just test -p codex-core -E 'test(pre_sampling_compact) |
test(model_unavailable_error)'` (10 tests)
- Added integration coverage for a resumed thread whose model was
renamed, a model downshift using Responses Compaction V2, and API-key
authentication with a custom provider.
## Summary
When enabled for the OpenAI provider, Codex sends
`stream_options.reasoning_summary_delivery = "sequential_cutoff"` on
HTTP and
WebSocket requests, including prewarm, and renders completed summary
sections
from `reasoning_summary_text.done`. Flag-off and non-OpenAI behavior is
unchanged.
## Expected rollout
```text
reasoning 0 added
summary 0 done
summary 1 done
summary 2 starts
summary 2 cancelled / incomplete
reasoning 0 done <-- cancel summary 2 work and mark it incomplete
message 1 added
message 1 text streams
message 1 completed
```
Depends on
[openai/openai#1096660](https://github.com/openai/openai/pull/1096660).
## Summary
- preserve reasoning item IDs on summary part and text delta events
- track streamed response items by ID so reasoning summaries can
continue after later items begin
- keep TUI output complete and deduplicated when reasoning and
final-answer events interleave
## Stack
- **1/2: this PR — interleaved item support**
- 2/2: #30752 — wire reasoning summary delivery configuration through
the CLI and app-server
## Validation
- just test -p codex-api preserves_reasoning_summary_item_ids
- just test -p codex-core
interleaved_reasoning_summary_events_keep_reasoning_item_metadata
- just test -p codex-tui
live_reasoning_summary_is_not_rendered_twice_when_item_completes
- just fix -p codex-api -p codex-core -p codex-tui
- just fmt
## Summary
- Preserve the optional namespace on custom tool calls during response
deserialization and app-server replay.
- Use the namespaced tool identifier for streaming argument handling and
tool dispatch.
- Regenerate app-server protocol schemas.
- Add regression tests covering namespace serialization and routing.
## Testing
- Ran affected protocol and app-server test suites.
- Ran the full core test suite; two load-sensitive timing tests passed
when rerun individually.
- Ran Clippy and formatting checks.
- Verified with a local end-to-end app-server replay that the namespace
is preserved through the complete request/response flow.
## Why
Remote diff-root discovery is independent of world-state construction,
but it ran afterward and added filesystem metadata latency before the
first model request. Overlap the independent work so thread-cold turns
do not pay those waits serially.
## What
- Run `record_context_updates_and_set_reference_context_item` and
`turn_diff_display_roots` with `tokio::join!`.
- Reuse the same resolved display roots when constructing
`TurnDiffTracker`; no cache or behavior lifecycle changes are
introduced.
## Validation
A synthetic executor-skill benchmark with artificial network delay:
thread-cold model-request p50 improved from about 1.79 s to 1.58 s.
## Why
Selected plugin metadata is stable, but MCP processes are live runtime
state. They need different lifetimes:
- the MCP extension caches manifest, MCP, and connector declarations for
each stable selected root;
- each model step projects that cached metadata through the roots that
resolved as ready for that exact step;
- the MCP manager is rebuilt only when that availability projection
changes.
This matches executor skills: both features consume the same resolved
step roots instead of inferring readiness from the turn's selected
environments.
## Behavior
```text
E1 not ready for this step
-> no E1 MCP servers or connectors
-> cached plugin metadata stays in ext/mcp
E1 becomes ready
-> reuse cached metadata
-> publish one MCP runtime containing E1 capabilities
same ready roots on the next step
-> reuse the exact runtime; no rediscovery and no MCP restart
resume
-> create new extension thread state and a new MCP runtime
```
All model-facing consumers use the same step snapshot:
```text
resolved selected roots
|
v
extension MCP/connector projection
|
v
{ MCP config, connector snapshot, MCP manager }
|
+-> advertise model tools
+-> build app/connector tools
+-> execute MCP calls
```
## Cache contract
The existing MCP extension owns a cache keyed by the full
`SelectedCapabilityRoot`:
```rust
let state = thread_store.get_or_init(SelectedExecutorPluginMcpState::default);
```
The cache lives with extension thread state. Environment availability
filters projection but does not invalidate metadata. Resume creates new
thread state. There is no file watcher or executor generation because
contents behind a stable environment/root are assumed stable.
## What changes
- Keeps executor plugin discovery and cached metadata in `ext/mcp`.
- Caches MCP and connector declarations together per selected root.
- Uses the step's already-resolved capability roots, including lazy
environments that are not turn environments.
- Reuses the current MCP runtime when the ready-root projection is
unchanged.
- Uses the same step MCP manager and connector snapshot for
model-visible tools and execution.
- Resolves direct thread-scoped MCP requests from the current
selected-root projection.
## Deliberately out of scope
- `app/list` remains based on the latest global host-plugin state; this
PR does not make its response or notifications thread-specific.
- `required = true` startup semantics do not apply to delayed executor
MCP activation.
- No filesystem/content invalidation.
- No transport-disconnect watcher.
- No executor generations or environment replacement semantics.
- No client sharing across complete manager replacements.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. Pin one MCP runtime to each model step.
4. **This PR:** project selected MCP and connector state from
extension-owned metadata.
5. Integration coverage for selected capability availability and resume.
## Verification
-
`selected_plugin_servers_use_managed_requirements_for_the_selected_root_id`
- The stacked integration PR covers unavailable to ready activation,
unchanged-runtime reuse, skills, MCP tools, connector attribution, and
cold resume.
## Why
An MCP refresh can replace the session's current manager while a model
step is still running. The step must execute calls through the same
manager whose tools it advertised.
## Boundary
```text
current session MCP runtime
|
| capture once for this model step
v
StepContext.mcp
- exact MCP config
- exact connection manager
- exact runtime environment context
```
```rust
pub struct McpRuntimeSnapshot {
config: Arc<McpConfig>,
manager: Arc<McpConnectionManager>,
runtime_context: McpRuntimeContext,
}
```
## Example
```text
step A captures runtime A and advertises A's tools
refresh publishes runtime B
step A tool call -> runtime A
next step -> runtime B
```
Capturing the snapshot is only an `Arc` clone. It does not restart MCPs
or make an RPC.
## What changes
- Captures one MCP runtime in `StepContext`.
- Uses it for tool planning, tool calls, resources, approvals, connector
attribution, and elicitation.
- Publishes replacement runtimes atomically.
- Lets an old runtime live only while an in-flight step or request still
holds its `Arc`.
Most of this diff is mechanical routing from the session-global manager
to `step_context.mcp`; it does not introduce selected-plugin discovery
yet.
## What does not change
- No plugin or extension migration.
- No new MCP cache policy.
- No environment file watching.
- No client sharing between separate managers.
## Stack
1. Extension-owned World State sections.
2. Project executor skills through World State.
3. **This PR:** pin one MCP runtime to each model step.
4. Project selected MCP/app/connector metadata by environment
availability.
5. One end-to-end integration scenario.
## Why
With deferred executors, a turn can begin before a remote environment
attaches. AGENTS.md discovery previously ran only during session setup,
so instructions from a later environment never reached the model or the
session instruction sources.
WorldState persistence has now landed, so this uses the durable
model-visible baseline directly instead of carrying a temporary
resume/fork compatibility path.
## What
- Add an `AgentsMdManager` in `SessionServices` to own host
instructions, loaded state, and refresh caching.
- When `DeferredExecutor` is enabled, refresh AGENTS.md when attached
environment selections change and freeze the result in the corresponding
`StepContext`.
- Represent AGENTS.md as a persisted WorldState section for every
session, with bounded initial, replacement, and removal updates.
- Remove duplicate AGENTS.md state and rendering from
`SessionConfiguration` and `TurnContext`.
- Build initial context, per-request updates, and compaction context
from the same step-scoped value.
- On resume and fork, compare current instructions with the restored
WorldState baseline and inject a replacement exactly once when they
differ.
Builds on #29833, #29835, and #29837.
## Tests
- Covers a remote environment becoming ready mid-turn, with AGENTS.md
appearing on the next request exactly once and updating canonical
instruction sources.
- Covers full, unchanged, replaced, and removed AGENTS.md WorldState
rendering.
- Covers changed instructions across cold resume and fork without
duplicate reinjection.
- Covers remote-v2 compaction retaining creation-time instructions in
the live session and cold resume appending one replacement when the
source changed.
- Ran focused `codex-core` AGENTS.md, WorldState, and context-update
test suites.
## Why
Connector declarations currently enter Codex through broad plugin
capability summaries, then MCP setup, turn tooling, and `app/list` each
reconstruct the same information. That makes executor-selected
connectors difficult to add without coupling connector behavior to the
host plugin loader.
This PR introduces a small connector-owned value that later stack layers
can populate before thread startup.
## What changed
- Move the pure app-declaration parser into `codex-connectors`,
preserving declaration order and category cleanup while leaving
host-side validation and deduplication unchanged.
- Add an immutable `ConnectorSnapshot` with ordered connector IDs and
plugin display-name provenance.
- Adapt the existing local-plugin capability summaries into that
snapshot at current consumer boundaries.
- Use the snapshot for MCP tool provenance, turn connector inventory,
and `app/list`.
- Keep the crate API narrow: no test-only snapshot accessors are
exposed.
The externally visible behavior is unchanged. Connector tools still come
from the orchestrator-owned `/ps/mcp` server, and local plugin
enablement remains owned by the existing plugin loader.
## Stack scope
This is the foundation only. It does not read selected executor packages
or change thread startup. #29852 adds the executor-backed declaration
reader, and #29856 composes selected declarations into a thread
snapshot.
## Why
Work(TPP) threads can be launched from the Desktop app, but if they all
keep the Desktop app's default originator then downstream attribution
cannot distinguish local Work launches from cloud-backed Work launches.
`thread/start.serviceName` already carries that launch signal, while
`SessionMeta.originator` is the durable thread-level value that survives
resume and fork.
This change converts the Desktop Work service names into an effective
originator at thread creation time, persists that originator with the
thread, and keeps using it for later model requests and memory writes.
## What changed
- Map `CODEX_WORK_LOCAL` and `CODEX_WORK_CLOUD` service names to
per-thread originators, while preserving
`CODEX_INTERNAL_ORIGINATOR_OVERRIDE` as the highest-precedence override.
- Persist the effective originator in `SessionMeta.originator`, read it
back on resume/fork, and inherit the parent originator for subagent
spawns when there is no persisted session metadata.
- Handle truncated `SpawnAgentForkMode::LastNTurns` forks by falling
back to the live parent originator when the forked history no longer
includes `SessionMeta`.
- Thread the per-thread originator through Responses headers,
websocket/compaction request paths, thread-store creation, rollout
metadata, and memory stage-one telemetry.
## Verification
- `just test -p codex-core
agent::control::tests::spawn_thread_subagent_inherits_parent_originator_without_fork
agent::control::tests::spawn_thread_subagent_fork_last_n_turns_inherits_parent_originator_without_session_meta
thread_manager::tests::originator_override_precedes_service_name_remapping`
- `just test -p codex-core
agent::control::tests::resume_thread_subagent_restores_stored_metadata_and_effective_multi_agent_mode`
- `just test -p codex-memories-write`
- `just fix -p codex-core -p codex-memories-write`
- `git diff --check`
## Why
When `Feature::TokenBudget` is enabled, compaction should behave like
`new_context`: start a fresh context window with the standard injected
context, without asking the server to summarize old history and without
carrying prior user or assistant messages into the next model request.
This is still a compaction operation from the client lifecycle
perspective. Manual `/compact` and auto-compaction should keep the same
observable side effects that clients and hooks expect, including compact
hooks and `TurnItem::ContextCompaction`.
## What changed
- Added `compact_token_budget` to run token-budget manual and inline
auto-compaction through a shared compaction lifecycle.
- Split pending `new_context` requests from forced context-window
startup: `take_new_context_window_request()` consumes pending requests,
and `start_new_context_window()` installs a fresh context window.
- Routed token-budget manual `/compact` and inline auto-compaction to
install a fresh context window locally instead of calling server/local
summarization.
- Preserved compact lifecycle side effects for token-budget compaction
by running pre/post compact hooks and emitting `ContextCompaction` item
start/completion events.
- Updated token-budget tests to assert fresh window IDs, absence of
server-side compaction calls, dropped prior transcript messages/tool
output after reset, and compact hook/item lifecycle behavior.
## Testing
- `just test -p codex-core
token_budget_context_uses_new_window_after_compaction`
- `just test -p codex-core token_budget_compaction_runs_compact_hooks`
- `just test -p codex-core
token_budget_mid_turn_auto_compaction_resets_before_active_follow_up`
---------
Co-authored-by: pakrym-oai <pakrym@openai.com>
## Why
With `model_auto_compact_token_limit_scope = "body_after_prefix"`, the
persistent prefix should not count against the active body window.
`get_context_remaining` and the token-budget reminder should report the
same usable body-after-prefix window that auto-compaction uses, rather
than the total token count since the session began.
This is stacked on #29664 so the mechanical move from `turn.rs` is
isolated from the behavior fix.
## What
- Extends `ContextWindowTokenStatus` with `context_remaining_tokens`.
- Updates `get_context_remaining` to use the shared context-window
accounting.
- Adds integration coverage for body-after-prefix reminder timing and
`get_context_remaining` output.
## Testing
- `just test -p codex-core body_after_prefix_window`
- `just test -p codex-core auto_compact_body_after_prefix`
- `just fix -p codex-core`
## Why
This PR keeps the mechanical helper extraction separate from the
behavior change in #29665. The follow-up needs the token-window
accounting from `turn.rs` in another call path, but reviewing that is
much easier when the helper extraction is separate from the semantic
change.
## What
- Adds `session/context_window.rs` with `ContextWindowTokenStatus`.
- Moves the existing auto-compaction token-status calculation out of
`session/turn.rs`.
- Replaces the duplicated inline remaining-token calculation in
`turn.rs` with `tokens_until_compaction()`.
This PR is intended to be behavior-preserving. The
`get_context_remaining` behavior change is stacked separately in #29665.
## Testing
- `just test -p codex-core auto_compact_body_after_prefix`
---
[//]: # (BEGIN SAPLING FOOTER)
Stack created with [Sapling](https://sapling-scm.com). Best reviewed
with [ReviewStack](https://reviewstack.dev/openai/codex/pull/29664).
* #29665
* __->__ #29664
## Why
With deferred executors, an environment can become ready between two
sampling requests in the same turn. The model-visible environment
update, advertised tools, and eventual tool execution must all describe
the same request-time view.
Otherwise, a request built while only environment B is ready can
advertise a tool without an `environment_id`; if higher-priority
environment A becomes ready before execution, that call could silently
run in A instead.
This PR is stacked on #29527.
## Design
`run_turn` captures one `Arc<StepContext>` at each sampling-request
boundary. That step owns the request's `TurnContext` and environment
snapshot.
- World-state environment updates and tool planning borrow that same
step.
- `ToolCallRuntime` retains the `Arc` while asynchronous tool calls
execute.
- `ToolInvocation` carries the step to handlers; its temporary `turn`
compatibility field is derived from the same object.
- `ToolRouter` does not retain `StepContext`; it only uses it while
constructing the request's tool set.
- With `DeferredExecutor` disabled, step capture keeps using the
environments frozen at turn start.
Simply: every sampling request gets one consistent picture of its
environments, from what the model sees through where its tool calls run.
## What changed
- Build environment-dependent tool specs from the current request's
`StepContext`.
- Use that same step for unified exec, legacy shell, `apply_patch`,
`view_image`, and `request_permissions` execution.
- Hide environment-backed tools, including `request_permissions`, while
no environment is attached.
- Resolve legacy shell paths and metadata from the selected step
environment instead of the stale turn-start environment.
- Capture explicit steps at non-turn-loop boundaries such as compaction,
prompt debug, and startup prewarm.
- Reconcile prompt-debug history from the same step used to build its
tools.
## Follow-up
- Bind yielded code-mode cells to the tool runtime that created them, so
nested calls made after yielding continue to use the originating
request's `StepContext`.
## Test plan
- `just test -p codex-core
deferred_executor_updates_context_and_tools_after_startup`
- `just test -p codex-core
environment_count_controls_environment_backed_tools`
- `just test -p codex-core
build_prompt_input_includes_context_and_user_message`
## Why
Follow-up to #29249 and its [compaction review
thread](https://github.com/openai/codex/pull/29249#discussion_r3455055101).
During a turn, environment readiness can change between sampling
requests. Inline compaction must render the same model-visible
`WorldState` used by the request it follows. Rebuilding that state
during compaction can observe a newer environment, make replacement
history disagree with what the model saw, and suppress the next
environment update.
## What changed
- Make `run_turn` own the current `Arc<WorldState>` and replace it only
between sampling requests.
- Build each state from an explicitly chosen environment snapshot, diff
deferred-executor steps against the turn-owned state, and retain the
latest state in `ContextManager` only for cross-turn and resume
tracking.
- Pass the exact turn-owned state into inline compaction and explicit
new-context-window replacement.
- Carry that state with
`InitialContextInjection::BeforeLastUserMessage`, so replacement context
and its stored baseline cannot come from different snapshots.
- Remove obsolete state-recapture helpers and ambiguous TurnContext-only
WorldState builders.
- Add an integration test that moves an environment from starting to
ready during a paused turn, triggers compaction, and verifies the next
request receives the readiness update exactly once.
## Test plan
- `just test -p codex-core
deferred_executor_compaction_preserves_then_updates_environment_once`
- `just test -p codex-core process_compacted_history`
- `just test -p codex-core mid_turn_continuation_compaction`
- `just test -p codex-core build_initial_context`
- `just test -p codex-core
ignores_session_prefix_messages_when_truncating`
When using Responses Lite, we should all use `additional_tools` and a
developer item instead of the top level tools array & instructions
field. This keeps things 1-to-1.
Forced namespacing for _all_ tools will land in a following PR after
some coordination & fixes in Responses API (around collisions & return
items).
The goal is to eventually expand the scope of this to _all_ requests
from codex, but that will require larger coordination across providers &
slower rollout.
## Summary
- read the request-scoped safety-buffering treatment from HTTP response
headers and per-turn WebSocket metadata through one shared header parser
- combine that treatment with Responses API safety-buffering signals
- propagate `showBufferingUi` and nullable `fasterModel` through the
existing `model/safetyBuffering/updated` app-server notification
- update the app-server documentation and generated JSON and TypeScript
schemas
The public implementation contains no model mapping or real model
identifier. Tests and protocol examples use generic `current-model` and
`faster-model` placeholders only.
## Dependencies
- server-side treatment evaluation:
https://github.com/openai/openai/pull/1060247
- initial Responses API safety-buffering propagation:
https://github.com/openai/codex/pull/29371
- Codex App UI: https://github.com/openai/openai/pull/1057789
## Validation
- Codex API tests: 129 passed
- focused Codex core safety-buffering integration test passed
- app-server protocol tests passed after regenerating schema fixtures
- Clippy fix and repository formatting completed successfully
The broader app-server run compiled all changed crates and completed
with 1,269 passing tests. Its remaining failures were unrelated
environment limitations: macOS sandbox application was denied, one
expected test binary was unavailable, and several existing subprocess
tests timed out as a result.
This PR moves construction of `PluginTelemetryMetadata` from loader and
model helpers into `PluginsManager`, which already owns installed plugin
state and will eventually perform remote identity enrichment. The
metadata type remains in `codex-plugin`, and serialized analytics events
remain unchanged.
## Before
```mermaid
flowchart LR
subgraph Events["Analytics event paths"]
direction TB
Lifecycle["Local install / uninstall"]
Config["Enable / disable"]
Remote["Remote install"]
Used["Plugin used"]
end
subgraph Construction["Metadata construction"]
direction TB
Loader["Loader telemetry helpers"]
Summary["PluginCapabilitySummary::telemetry_metadata"]
Override["Caller adds remote_plugin_id"]
end
Metadata["PluginTelemetryMetadata"]
Lifecycle --> Loader
Config --> Loader
Remote --> Loader
Loader -->|"local events"| Metadata
Loader -->|"remote install"| Override
Override --> Metadata
Used --> Summary
Summary --> Metadata
```
Telemetry metadata was constructed through loader helpers, a
capability-summary method, and a remote-install call-site override.
## After
```mermaid
flowchart LR
subgraph Events["Analytics event paths"]
direction TB
Lifecycle["Local install / uninstall"]
Config["Enable / disable"]
Remote["Remote install"]
Used["Plugin used"]
end
Manager["PluginsManager — single construction owner"]
Metadata["PluginTelemetryMetadata"]
Lifecycle --> Manager
Config --> Manager
Remote -->|"authoritative remote ID"| Manager
Used -->|"capability summary"| Manager
Manager --> Metadata
```
Every analytics path delegates metadata construction to
`PluginsManager`. Remote install still supplies its authoritative
backend ID explicitly.
## What Changes
- Make loader code return a focused plugin capability summary instead of
constructing analytics metadata.
- Centralize immutable plugin telemetry metadata construction in
`PluginsManager`.
- Route local install/uninstall, remote install, enable/disable, and
plugin-used emitters through the manager.
- Preserve the current serialized analytics contract exactly.
Normal metadata still has no remote override. Remote install continues
to provide its authoritative backend ID explicitly, so the existing
serializer continues reporting that ID through `plugin_id`.
Snapshot-based enrichment is intentionally deferred to the final PR.
## Testing
- `just test -p codex-core-plugins` (238 tests passed)
- `just test -p codex-plugin` (3 tests passed)
- Scoped Clippy/compile checks passed for `codex-plugin`,
`codex-core-plugins`, `codex-app-server`, and `codex-core`.
## Split Overview
```text
main
├── #27093 Debug analytics capture (merged)
├── #27099 Non-mutating plugin smoke (merged)
├── #27100 Remote install/uninstall smoke (merged)
└── #27102 Plugin telemetry metadata refactor ← you are here
└── #27669 Persist remote plugin identity
After #27102 and #27669 merge:
└── Final PR: add explicit local and remote IDs to plugin analytics
```
Review order and dependencies:
1. [#27093 Add debug-only analytics event
capture](https://github.com/openai/codex/pull/27093) (merged)
2. [#27099 Add a plugin analytics smoke
workflow](https://github.com/openai/codex/pull/27099) (merged)
3. [#27100 Add a remote plugin analytics mutation smoke
workflow](https://github.com/openai/codex/pull/27100) (merged)
4. This metadata refactor, independent and based on `main`
5. [#27669 Persist remote plugin
identity](https://github.com/openai/codex/pull/27669), stacked on this
PR
6. Final remote-ID behavior PR, created after the prerequisites merge
The original [#26281](https://github.com/openai/codex/pull/26281)
remains open as the aggregate reference until the final replacement PR
is published.
## Why
Nonblocking environment snapshots allow a turn to reach the model while
a remote environment is still starting. The initial context can describe
that environment as still loading, but nothing currently refreshes the
model-visible environment context when startup finishes during the same
turn.
This adds the first request-scoped reconciliation slice on top of
#28683. It is gated by `DeferredExecutor` and intentionally updates only
model-visible environment context; tools and other environment-derived
state will migrate separately.
## What
- Add a minimal `StepContext` containing the environment snapshot
captured before each sampling request.
- Render attached environments with their resolved shell and starting
environments with `still loading`.
- Track the latest environment state recorded in model history and
append a bounded update only when it changes.
- Seed that baseline from full initial context so ready-at-start
environments are not duplicated.
- Clear the in-memory baseline when history is rewritten so replacement
history can be refreshed safely.
## Testing
- `just test -p codex-core deferred_executor`
- `just test -p codex-core
environment_context_baseline_deduplicates_until_history_is_replaced`
The integration coverage verifies that a pending environment reaches the
first request, the ready state reaches the next request, later requests
do not duplicate it, and ready-at-start environments remain
single-injected.
<details>
<summary>Live verification</summary>
- Connected to a real remote executor with startup deliberately delayed
and forced three sampling requests in one turn.
- Inspected the raw model inputs: request 1 showed the remote
environment as `still loading`, request 2 appended its ready shell and
cwd, and request 3 contained no duplicate ready update.
- With the feature disabled, startup waited for the delayed executor and
the first request contained only the ready environment.
- With a synchronously ready environment and the feature enabled, the
first request contained one environment context with no duplicate.
- Executed `pwd` and read a marker file through the remote process
runner; the command exited successfully and returned the remote cwd and
marker contents.
</details>
## Summary
- read the `AutoCompaction` feature flag through `TurnContext::config`
- fix both the mid-turn and pre-sampling compaction checks
## Why
#28260 was validated against an older base where `TurnContext` exposed a
direct `features` field. It was then merged after that field had moved
under `config`, leaving the merge result unable to compile with `E0609`
on `turn_context.features`.
This restores compilation for Bazel, SDK, and argument-comment-lint jobs
that build `codex-core`. Behavior is unchanged: disabling
`auto_compaction` still skips automatic compaction.
## Validation
- `just fmt`
- `CODEX_HOME=/private/tmp/codex-fix-auto-compaction-test-home just test
-p codex-core auto_compaction_feature_disabled` — 4 passed
- `just test -p codex-core` — `codex-core` compiled; 2,722 passed and 89
unrelated local-environment failures remained because the sandbox could
not write the default Codex SQLite/proxy paths and some first-party test
binaries were unavailable
Responses API safety buffering metadata currently stops at the transport
boundary, so app-server clients cannot render the in-progress safety
review state.
This change:
- decodes and deduplicates `safety_buffering` metadata from Responses
API SSE and WebSocket events without suppressing the original response
event
- emits a typed core event containing the requested model plus backend
use cases and reasons
- forwards that event as `turn/safetyBuffering/updated` through
app-server v2 and updates generated protocol schemas
- keeps the side-channel event out of persisted rollouts and turn timing
This supports the Codex Apps buffering UX and depends on the Responses
API backend work in https://github.com/openai/openai/pull/1044569 and
https://github.com/openai/openai/pull/1044571.
Validation:
- focused `codex-core` safety-buffering integration test passes
- `cargo check -p codex-core -p codex-app-server -p
codex-app-server-protocol`
- `just fix -p codex-api -p codex-protocol -p codex-core -p
codex-app-server-protocol -p codex-app-server -p codex-rollout -p
codex-rollout-trace -p codex-otel`
- `just fmt`
- broad package test run: 4,430/4,492 passed; 62 unrelated
local-environment/concurrency failures involved unavailable test
binaries, MCP subprocess setup, and app-server timeouts
## Summary
- add a default-on `auto_compaction` feature flag as an internal escape
hatch
- skip pre-turn, model-switch/hash, and mid-turn automatic compaction
when the flag is disabled
- preserve manual `/compact` behavior and surface the existing
context-window error when the provider runs out of room
- add integration coverage for disabled pre-turn and mid-turn compaction
## Motivation
Long-running SPO optimization rollouts need the option to preserve their
full context and fail on context exhaustion instead of entering another
compaction window. This deliberately uses the existing feature-flag
mechanism rather than adding a dedicated public config or app-server
API.
Disable it with:
```sh
codex --disable auto_compaction
```
## Testing
- `just test -p codex-features` — 51 passed
- `just test -p codex-core auto_compaction_feature_disabled` — 2 passed
- `just fix -p codex-core -p codex-features`
- `just write-config-schema`
- `just test -p codex-core` — the new compaction tests passed; the
overall local run had 54 unrelated environment failures, primarily
missing first-party test binaries and shell-snapshot timeouts
## Why
The token-budget feature currently adds remaining-token messages
whenever usage crosses the 25%, 50%, and 75% thresholds. Those periodic
inserts create prompt churn without requiring action, while the
near-compaction reminder and explicit `get_context_remaining` tool
already cover actionable and on-demand budget information.
The context-window lineage block is also easier to scan as plain labeled
text than as a `<token_budget>`-wrapped fragment.
## What changed
- Stop recording automatic remaining-token messages at percentage
thresholds.
- Render context-window lineage in `First`, `Current`, `Previous` order
with colon-separated labels.
- Omit the `Previous` line for the first context window.
- Remove `<token_budget>` wrappers from newly rendered lineage,
near-compaction reminders, and `get_context_remaining` output.
- Keep recognizing legacy wrapped fragments so existing rollouts remain
compatible.
- Remove the post-sampling token snapshot that was only needed by the
periodic threshold path.
## Testing
- `just test -p codex-core token_budget` (11 tests passed)
## Why
The token-budget feature reports coarse remaining-context milestones,
but it does not give the model a configurable wrap-up prompt before
automatic compaction. A strict threshold-crossing check can also miss
resumed or reconfigured windows that are already inside the threshold.
## What changed
- Add structured `[features.token_budget]` configuration for an absolute
`reminder_threshold_tokens` and bounded `reminder_message_template`;
`{n_remaining}` is expanded when the reminder is delivered.
- Compute remaining tokens against the next effective auto-compaction
boundary, including scoped `body_after_prefix` accounting and the full
context-window limit.
- Make reminder delivery level-triggered before and after sampling, with
one-shot state owned by `AutoCompactWindow` and re-armed on compaction,
`new_context`, restore, or history replacement.
- Leave the existing initial full-window token-budget context, 25/50/75%
notices, and token-budget tools unchanged.
- Persist the resolved feature configuration in the session config lock
and regenerate the config schema.
## Validation
- `just test -p codex-core token_budget`
- `just test -p codex-core
token_budget_reminder_emits_after_crossing_compaction_threshold`
- `just test -p codex-core auto_compact_window`
- `just test -p codex-core
lock_contains_prompts_and_materializes_features`
- `just test -p codex-features`
- `just test -p codex-config`
## Stack
Depends on #28494.
## Description
This PR propagates shared rollout-budget exhaustion through the existing
`CodexErr::TurnAborted` task result.
Each thread records its model usage against the same ledger. Once the
ledger is exhausted, that usage update and all later usage updates
return `TurnAborted`. The task wrapper emits the normal aborted-turn
event and lifecycle instead of completing the turn.
This is intentionally a soft boundary: there is no cross-thread
`Op::Interrupt` fanout. An in-flight thread can finish its current
response before it observes the exhausted ledger, but every thread
aborts at its next usage-accounting boundary.
## Tests
The integration coverage verifies that:
- the response that exhausts the budget aborts its turn;
- a later response also aborts because the shared ledger remains
exhausted; and
- sub-agent usage draws from the same shared ledger; and
- local and remote-v2 compaction abort without retrying or emitting a
generic error.
Local checks:
- `just test -p codex-core
exhausted_budget_aborts_current_and_later_turns`
- `just test -p codex-core subagent_usage_draws_from_the_shared_budget`
- `just test -p codex-core
abort_regular_task_emits_marker_before_turn_aborted`
- `just test -p codex-core
compaction_budget_exhaustion_aborts_without_error_or_retry`
- `just fix -p codex-core`
- `just fmt`
- `git diff --check`
The full workspace test suite was not run locally.
Stacked on #28822.
## Summary
- add a host-injectable current-time provider with a built-in system
implementation
- record UTC developer reminders in history immediately before due model
requests
- keep cadence state per session and force a refresh after compaction
This does NOT include the app server client <-> server clock logic. This
PR is only for the reminder message & system clock that will be used in
prod.
## Testing
- `just test -p codex-core varlatency_`
- `just clippy -p codex-core -p codex-app-server -p codex-mcp-server -p
codex-thread-manager-sample`
- `just fmt`
## Stack
Depends on #28746. This PR implements shared rollout-budget accounting
and model-visible reminders using the configuration defined in #28746.
# Description / Main changes to Core:
`AgentControl` will now be the area where "rollout level" features &
accounting will have to live. It is incorrectly named for this
responsibility, but I think it can hold all the necessary shared state &
features (rollout token budget, mutliple thread interruption
responsibilitym etc)
In this PR, we have one "token ledger" that each thread will subtract
from when sampling. The "charge" will occur when response.completed() is
done and the calculation will be done on the responses api usage
carrier. The calculation will weigh sampling and pre-fill tokens as
specified.
Every time the budget crosses the configured reminder threshold, a
developer message is appended before the thread's next request
This remaining budget will _always_ be restated/reminded after a
compaction event.
Expiration and fan-out interruption will be in the stacked follow-up
(and also live in Agent Control).
## Reminders
"You have weighted {session_tokens_left} tokens left in the shared
session token budget."
The first request in each thread context receives the current remainder.
Later reminders are emitted after aggregate weighted usage crosses a
configured interval. If several intervals are crossed before a thread
sends another request, Core inserts one reminder with the latest
remainder.
Compaction response usage is charged before the next context starts. The
next reminder is appended after the compaction summary, leaving the
initial context content stable.
## Tests
Integration coverage verifies:
- weighted output and non-cached input accounting
- initial and periodic reminders
- shared accounting between a root and sub-agent
- post-compaction remainder and message placement
Local checks:
- `just fmt`
- `just test -p codex-core rollout_budget`
- `git diff --check`
The full workspace test suite was not run locally.
## What
Built on the realtime speech-control plumbing merged in #27917.
- Add optional `codexResponseHandoffPrefix` to `thread/realtime/start`.
- Apply that prefix only to automatic V1 commentary sent through
`conversation.handoff.append`; final answers remain unprefixed.
- Add opt-in `clientManagedHandoffs`. When true, core suppresses
automatic response handoffs and completion output so delivery is
controlled by explicit client append APIs.
- Preserve existing automatic behavior by default.
`codexResponsesAsItems: true` continues to select item routing when
client-managed mode is disabled.
## Why
Voice clients need two delivery policies: automatic background context
with silent commentary instructions and fully client-owned handoffs.
Phase-aware prefixing keeps routine commentary silent without
suppressing the final answer, while client-managed mode lets an app
decide exactly which updates to append.
## Validation
- `just fmt`
- `cargo test -p codex-app-server-protocol
serialize_thread_realtime_start`
- `RUST_MIN_STACK=16777216 cargo test -p codex-core --test all
conversation_handoff_persists_across_item_done_until_turn_complete`
- `RUST_MIN_STACK=16777216 cargo test -p codex-app-server --test all
webrtc_v1_client_managed_handoffs_disable_automatic_output`
- `RUST_MIN_STACK=16777216 cargo test -p codex-app-server --test all
webrtc_v1_final_automatic_handoff_omits_silent_prefix`
- `cargo build -p codex-cli --bin codex`
- Local Codex Apps compatibility check: 43 focused webview tests passed,
and a live voice session routed through the source-built app-server.
The explicit `RUST_MIN_STACK` avoids a macOS Tokio test-worker stack
overflow seen with the default test environment.
## Why
Host skill discovery was still exposed as a manager even though it is a
process-owned service shared by sessions, the app-server catalog, and
file-watcher invalidation. The skills extension also consumed an ad hoc
loaded-skills wrapper instead of a named immutable snapshot.
## What changed
- replace `SkillsManager` with concrete `SkillsService`
- make the service cache and return immutable `HostSkillsSnapshot`
values
- migrate the skills extension host provider to the snapshot boundary
- migrate app-server catalog, watcher, and invalidation paths to the
service
This keeps the service limited to host discovery, caching, roots, and
invalidation. Catalog rendering and invocation remain extension
responsibilities for the next stacked change.
## Why
`TurnContext` had accumulated dead fields and cached projections of
values already owned by its per-turn `Config` or `ModelInfo`. Keeping
both copies made ownership unclear and allowed artificial split-brain
states, such as a compatibility hash differing from the model metadata
it came from.
`Prompt` similarly carried a write-only personality after personality
selection had already been materialized into its base instructions.
This makes the canonical owner explicit: configuration-backed values
come from `config`, model-derived values come from `model_info`, and
prompts contain only data consumed by request construction.
## What changed
- Remove the unused `ghost_snapshot`, `codex_self_exe`, and
`thread_source` fields.
- Remove duplicate `comp_hash`, `truncation_policy`, `features`,
`shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`,
and `tool_mode` fields.
- Read those values directly from `TurnContext::config` or
`TurnContext::model_info` at their consumers.
- Remove the write-only `Prompt::personality` field and its constructor
assignments.
- Preserve review-turn inheritance of the parent turn's shell policy,
Linux sandbox executable, and compact prompt through the review config.
## Testing
- `cargo check -p codex-core --tests`
## Why
#28152 jumped the gun on moving the rollout format to store URIs, and
would likely break compat with some features that don't go through the
same types as the core logic.
## What
Make `TurnContextItem.cwd` an `AbsolutePathBuf` again, remove test added
for `PathUri` serialization in rollouts. Also drops a bunch of error
paths that are no longer needed.
jiayuhuang-openai