## What changed
- Add provider-owned remote compaction capabilities for unsupported, v1, and v2 protocols.
- Mark Amazon Bedrock as v1-only so manual and automatic compaction use `/v1/responses/compact`, even when the v2 feature is enabled.
- Preserve v2 support for OpenAI and Azure Responses providers and local compaction for unsupported providers.
## Testing
- Cover Amazon Bedrock manual and automatic remote compaction, including endpoint selection, authentication headers, model selection, and reuse of the compacted summary.
GitOrigin-RevId: d58093178605d1f8155b93d9a161408177f6ac72
## What changed
- Retain bounded, non-completion agent messages in remote v2 compaction history so delegated tasks remain available on follow-up turns.
- Account for encrypted agent-message content when estimating token usage and place restored initial context before the latest real user or agent message.
- Exclude child completion messages from retained follow-up history and strip inherited parent agent messages when forking a child.
## Testing
- Extend compaction, history-estimation, and agent-fork tests to cover encrypted delegated tasks, child completions, context ordering, and fork sanitization.
GitOrigin-RevId: ba4392e9cfdf4ecbcdc9c7d8581d58a8d2d78260
## What changed
- Wrap `CodexErrorDetails` and an optional retry delay in `CodexErr`, allowing any mapped error to preserve server-provided retry timing.
- Generate the payload-free `CodexErrKind` classification alongside the error details and reuse it for analytics.
- Update error handling sites to inspect `CodexErr::details()` while preserving existing display, debug, protocol mapping, and retryability behavior.
## Testing
- Add coverage for legacy debug formatting, error-specific retryability, and retry-delay propagation through API error mapping.
GitOrigin-RevId: d3ab8a305f2a2ee21d0c0a8c8c388b06dda9c59a
## What changed
- Make `McpRuntime` own the published MCP configuration, connections, elicitation routing, and selected capability roots for a thread.
- Capture immutable MCP bindings for model steps and tool calls so in-flight work keeps a consistent connection set and approval authority while refreshed state is published atomically.
- Mark MCP state dirty when relevant configuration, plugins, environments, authentication, or elicitation capabilities change, then rebuild it before the next sampling request or out-of-band MCP operation.
- Separate config reloads from server invalidation: `ReloadMcpConfig` applies resolved MCP inputs, while `RefreshMcpServers` requests reinitialization from the thread's latest state.
## Testing
- Cover refreshed state visibility for existing turns, stable step bindings, current approval authority, cancelled refresh retries, resource-client reconciliation, and Apps recovery between sampling requests.
GitOrigin-RevId: 59eabb1aa8dc083426bd18ef4d3630508f376401
## Why
Interrupting a turn while MCP tools were still starting could abort before the
submitted user input was recorded in conversation history.
## What changed
- Build and retain the MCP tool list and tool router as part of each step
snapshot, using the turn cancellation token during tool discovery.
- Record turn hooks and submitted input when cancellation occurs during startup
prewarming, pre-sampling compaction, or first-step capture.
- Reuse the captured router for sampling, compaction, and prompt debugging so
advertised and executable tools share the same request-scoped view.
## Testing
Add coverage that interrupts each affected MCP startup phase and verifies that
the user prompt remains in history before the turn-aborted marker.
GitOrigin-RevId: 6e96c6814357a55545d69b580e28f35933cfadf7
## What changed
- Remove the step-scoped `ExtensionData` argument from context, turn-input, and tool contributors.
- Pass the host's optional `McpResourceClient` through `ThreadStartInput` so extensions can retain session capabilities explicitly.
- Keep the MCP resource client in skills-owned session state for catalog loading and skill tools.
GitOrigin-RevId: bafa77bcd998aff408d6a396c5fd9ac268c4cce4
## What changed
- Assign IDs to client-created response items in every session, including streamed items, forked history, compaction results, and non-OpenAI provider requests.
- Preserve server-provided IDs and stable generated IDs across persisted and resumed conversations while continuing to omit unprefixed server IDs from outbound requests.
- Retire `features.item_ids` as a configurable feature. Existing configuration and CLI toggles remain accepted as compatibility no-ops.
## Testing
- Cover default ID assignment, persistence and resume behavior, remote compaction, provider requests, WebSocket requests, and removed-feature compatibility.
GitOrigin-RevId: 7a09e2d07d2606c2ef1e8e2283d36cb555557eae
## What changed
- Pass only compaction message and window metadata into
`Session::replace_compacted_history`.
- Build the persisted `CompactedItem` there after assigning any missing response
item IDs, so its replacement history matches the live history without callers
cloning or assembling it themselves.
GitOrigin-RevId: a6106c33b6d3eef39ac0b563a50e17f6782970dc
## What changed
- Add an `ExtensionData` store to each `StepContext`.
- Pass the step store to context, world-state, turn-input, and tool contributors so extensions can use capabilities bound to the current sampling step.
- Preserve the captured step context when rebuilding initial context during compaction.
GitOrigin-RevId: 51f3105d3a793f60534f122f9be8247c131c2ca5
## What changed
- Add remote and local audio variants to core and app-server user inputs, including generated JSON and TypeScript schemas.
- Preserve audio fields when converting app-server inputs and response content into thread items.
- Keep unsupported audio out of model requests, TUI rendering, text extraction, and truncated tool output, using explicit text placeholders where audio is submitted as a model input.
## Testing
- Cover app-server media conversion, unsupported-input placeholders, TUI omission, and tool-output truncation.
GitOrigin-RevId: 16152bf3af674575ce569764a8b472e032e7569d
## What changed
- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.
## Testing
- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.
GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
## Why
Clients that account for individual upstream Responses API calls need the exact
usage for each completion rather than accumulated or estimated thread usage.
## What changed
- Add the opt-in `rawResponse/completed` app-server notification with the
`threadId`, `turnId`, `responseId`, and upstream `usage` payload.
- Emit the transient event for normal turns and compaction requests when
`experimentalRawEvents` is enabled. The event is not persisted or replayed,
and `usage` is `null` when the upstream response omits it.
- Export the notification in the app-server schemas and document its behavior.
## Testing
- Cover exact token breakdowns for turn and compaction completions.
GitOrigin-RevId: 49040a060bf1784c1ff1dd62112efb4a778428c4
## Why
`TurnContext` stored model and reasoning settings both as resolved turn fields and inside `CollaborationMode`, requiring the copies to stay synchronized when the model changed.
## What changed
- Store the mode and collaboration-mode developer instructions directly on `TurnContext`.
- Reconstruct `CollaborationMode` from those values plus the turn's current model and reasoning effort when callers need it.
- Read the mode directly for plan-mode checks, lifecycle events, and telemetry.
GitOrigin-RevId: b8fccafdd5d5cbccc84bc70d542588087ddf0093
## Summary
- rename the rollout-budget exhaustion error from
`RolloutBudgetExceeded` to `SessionBudgetExceeded`
- expose the matching app-server v2 wire value as
`sessionBudgetExceeded`
- regenerate JSON/TypeScript schema fixtures and update the app-server
docs and focused tests
This is a naming-only follow-up to #29715 based on [Pavel's review
suggestion](https://github.com/openai/codex/pull/29715#discussion_r3463183480).
Runtime behavior is unchanged.
## Tests
- `just test -p codex-core rollout_budget`
- `just test -p codex-app-server-protocol`
- `just fmt`
- `just write-app-server-schema`
## Summary
- surface shared rollout-budget exhaustion as
`CodexErr::RolloutBudgetExceeded` instead of a generic interrupted turn
- map it through the existing `CodexErrorInfo` and app-server v2
`codexErrorInfo` path
- keep local compaction from retrying after the shared rollout budget is
exhausted
This gives app-server clients a stable `rolloutBudgetExceeded` error
they can classify without guessing from `status="interrupted"`.
## Tests
- `just test -p codex-core rollout_budget`
## Why
Follow-up to #29249 and its [compaction review
thread](https://github.com/openai/codex/pull/29249#discussion_r3455055101).
During a turn, environment readiness can change between sampling
requests. Inline compaction must render the same model-visible
`WorldState` used by the request it follows. Rebuilding that state
during compaction can observe a newer environment, make replacement
history disagree with what the model saw, and suppress the next
environment update.
## What changed
- Make `run_turn` own the current `Arc<WorldState>` and replace it only
between sampling requests.
- Build each state from an explicitly chosen environment snapshot, diff
deferred-executor steps against the turn-owned state, and retain the
latest state in `ContextManager` only for cross-turn and resume
tracking.
- Pass the exact turn-owned state into inline compaction and explicit
new-context-window replacement.
- Carry that state with
`InitialContextInjection::BeforeLastUserMessage`, so replacement context
and its stored baseline cannot come from different snapshots.
- Remove obsolete state-recapture helpers and ambiguous TurnContext-only
WorldState builders.
- Add an integration test that moves an environment from starting to
ready during a paused turn, triggers compaction, and verifies the next
request receives the readiness update exactly once.
## Test plan
- `just test -p codex-core
deferred_executor_compaction_preserves_then_updates_environment_once`
- `just test -p codex-core process_compacted_history`
- `just test -p codex-core mid_turn_continuation_compaction`
- `just test -p codex-core build_initial_context`
- `just test -p codex-core
ignores_session_prefix_messages_when_truncating`
## Description
This PR is a followup to https://github.com/openai/codex/pull/28355 and
starts assigning `internal_chat_message_metadata_passthrough.turn_id` to
durable Responses API items created during a turn.
The goal is that those items keep the `turn_id` that introduced them
when Codex resends stateless HTTP context, reconstructs history for
resume/fork paths, or reuses websocket response state.
## What changed
- Set `internal_chat_message_metadata_passthrough.turn_id` when missing
as response items enter durable history, initial/replacement history,
inter-agent communication history, and local compaction summaries.
- Preserve existing item turn IDs instead of overwriting them during
persistence, resume reconstruction, compaction, forked history, and
websocket incremental reuse.
- Keep `compaction_trigger` fieldless because it is a request control,
not a durable response item.
- Update focused history/request assertions and fixtures for stateless
requests, websocket incrementals, compaction, thread injection, prompt
debug, and related CI coverage.
## Description
This PR cuts Codex over from generic `ResponseItem.metadata` (introduced
here: https://github.com/openai/codex/pull/28355) to
`ResponseItem.internal_chat_message_metadata_passthrough`, which is the
blessed path and has strongly-typed keys.
For now we have to drop this MAv2 usage of `metadata`:
https://github.com/openai/codex/pull/28561 until we figure out where
that should live.
## Why
The rendered `<token_budget>` fragment identifies the thread and current
context window, but it does not expose enough lineage to identify the
first window in the thread or the immediately preceding window. Those
IDs also need to remain stable across compaction, resume, and rollback.
## What changed
- Track first, previous, and current UUIDv7 context-window IDs in
auto-compaction state.
- Render `thread_id`, `first_window_id`, `previous_window_id`, and the
current window ID in the full `<token_budget>` fragment.
- Persist the first and previous window IDs in compacted rollout
checkpoints and restore them during rollout reconstruction.
- Preserve compatibility with older compacted records that do not
contain the new optional fields.
- Update focused state, rendering, reconstruction, rollback, and
serialization coverage.
## Validation
- `just test -p codex-core token_budget`
- `just test -p codex-protocol compacted_item::tests`
- `just test -p codex-core tracks_prefill_and_window_boundaries`
- `just test -p codex-core
reconstruct_history_uses_replacement_history_verbatim`
- `just test -p codex-core
thread_rollback_restores_cleared_reference_context_item_after_compaction`
## Stack
Depends on #28494.
## Description
This PR propagates shared rollout-budget exhaustion through the existing
`CodexErr::TurnAborted` task result.
Each thread records its model usage against the same ledger. Once the
ledger is exhausted, that usage update and all later usage updates
return `TurnAborted`. The task wrapper emits the normal aborted-turn
event and lifecycle instead of completing the turn.
This is intentionally a soft boundary: there is no cross-thread
`Op::Interrupt` fanout. An in-flight thread can finish its current
response before it observes the exhausted ledger, but every thread
aborts at its next usage-accounting boundary.
## Tests
The integration coverage verifies that:
- the response that exhausts the budget aborts its turn;
- a later response also aborts because the shared ledger remains
exhausted; and
- sub-agent usage draws from the same shared ledger; and
- local and remote-v2 compaction abort without retrying or emitting a
generic error.
Local checks:
- `just test -p codex-core
exhausted_budget_aborts_current_and_later_turns`
- `just test -p codex-core subagent_usage_draws_from_the_shared_budget`
- `just test -p codex-core
abort_regular_task_emits_marker_before_turn_aborted`
- `just test -p codex-core
compaction_budget_exhaustion_aborts_without_error_or_retry`
- `just fix -p codex-core`
- `just fmt`
- `git diff --check`
The full workspace test suite was not run locally.
## Why
Remote v2 compaction can return replacement-history items without IDs.
Because replacement history is installed directly, those items bypass
normal history preparation and remain ID-less in later Responses
requests even when the `item_ids` feature is enabled.
## What changed
- Pass the active `TurnContext` into `replace_compacted_history`.
- When `item_ids` is enabled, assign missing IDs before installing and
persisting replacement history.
- Rebuild `CompactedItem` from the prepared history so live and
persisted replacement histories match.
- Add integration coverage requiring IDs on every ID-capable input item
in the initial, remote v2 compaction, and post-compaction requests.
## Test plan
- `just test -p codex-core response_item_ids`
- `just test -p codex-core websocket_v2_test_codex_shell_chain`
- `just test -p codex-core remote_compaction_parity_pre_turn_auto`
- `just test -p codex-app-server
thread_inject_items_adds_raw_response_items_to_thread_history`
## Why
The token-budget context currently identifies a context window by its
thread-local sequence number. A UUIDv7 gives the model a stable opaque
identity that remains fixed for a window and rotates when compaction or
`new_context` starts the next one.
## What changed
- Preserve the existing monotonic value as `window_number` and add a
UUIDv7 `window_id` to `CompactedItem`.
- Generate and rotate the UUID with auto-compaction window state,
persist it alongside the number, and reconstruct it on resume and
rollback.
- Accept legacy compacted rollout records where the numeric `window_id`
represented the window number.
- Use the UUID only in token-budget context; existing request headers
and metadata continue using `thread_id:window_number`.
## Testing
- `just test -p codex-protocol compacted_item::tests`
- `just test -p codex-core token_budget`
## Why
`TurnContext` had accumulated dead fields and cached projections of
values already owned by its per-turn `Config` or `ModelInfo`. Keeping
both copies made ownership unclear and allowed artificial split-brain
states, such as a compatibility hash differing from the model metadata
it came from.
`Prompt` similarly carried a write-only personality after personality
selection had already been materialized into its base instructions.
This makes the canonical owner explicit: configuration-backed values
come from `config`, model-derived values come from `model_info`, and
prompts contain only data consumed by request construction.
## What changed
- Remove the unused `ghost_snapshot`, `codex_self_exe`, and
`thread_source` fields.
- Remove duplicate `comp_hash`, `truncation_policy`, `features`,
`shell_environment_policy`, `codex_linux_sandbox_exe`, `compact_prompt`,
and `tool_mode` fields.
- Read those values directly from `TurnContext::config` or
`TurnContext::model_info` at their consumers.
- Remove the write-only `Prompt::personality` field and its constructor
assignments.
- Preserve review-turn inheritance of the parent turn's shell policy,
Linux sandbox executable, and compact prompt through the review config.
## Testing
- `cargo check -p codex-core --tests`
## Description
This PR adds an optional `metadata` field to `ResponseItem` for
Responses API calls. Only mechanical plumbing, no actual values
populated and sent yet. Turns out just adding a new field to
`ResponseItem` has quite a large blast radius already.
This change is backwards compatible because `metadata` is optional and
omitted when absent, so existing response items and rollout history
without it still deserialize and requests that do not set it keep the
same wire shape. For provider compatibility, we strip out `metadata`
before non-OpenAI Responses requests so Azure and AWS Bedrock never see
this field.
My followup PR here will actually make use of it to start storing and
passing along `turn_id`: https://github.com/openai/codex/pull/28360
## What changed
- Added `ResponseItemMetadata` with optional `turn_id`, plus optional
`metadata` on Responses API item variants and inter-agent communication.
- Preserved item metadata through response-item rewrites such as
truncation, missing tool-output synthesis, compaction history
rebuilding, visible-history conversion, rollout/resume, and generated
app-server schemas/types.
- Strip item metadata from non-OpenAI Responses requests while
preserving it for OpenAI-shaped requests.
- Updated the mechanical fixture/test construction churn required by the
new optional field.
## What
Introduce a `CodexResponsesMetadata` struct that defines all the core
metadata we send to Responses API. Example fields are `thread_id`,
`turn_id`, `window_id`, etc.
Going forward, `client_metadata["x-codex-turn-metadata"]` will be the
canonical way Codex sends metadata to Responses API across both HTTP and
websocket transports.
For now, we continue to emit the existing top-level HTTP headers and
top-level `client_metadata` fields from the same
`CodexResponsesMetadata` struct for compatibility reasons.
Also, app-server clients who specify additional
`responsesapi_client_metadata` via `turn/start` and `turn/steer` will
have those fields merged into
`client_metadata["x-codex-turn-metadata"]`, but cannot override the
reserved fields that core uses (i.e. the fields in
`CodexResponsesMetadata`).
## Why
Responses API request instrumentation is the source of truth for
downstream Codex analytics that join requests by Codex IDs such as
session, thread, turn, and context window. Before this change, those
values were assembled through several request-specific paths: HTTP
request bodies, websocket handshake headers, websocket `response.create`
payloads, compaction requests, and the rich `x-codex-turn-metadata`
envelope all had their own wiring.
That made metadata propagation easy to drift across API-key/direct
Responses API requests, ChatGPT-auth/proxied requests, websocket
requests, and compaction requests. It also made additions like
`window_id` error-prone because a field could be added to one transport
projection but missed in another.
## What changed
- Added `CodexResponsesMetadata` as the core-owned snapshot for Codex
metadata sent to ResponsesAPI.
- Render `client_metadata["x-codex-turn-metadata"]`, flat
`client_metadata` projections, and direct compatibility headers from
that same snapshot.
- Include the known Codex-owned fields in the turn metadata blob,
including installation/session/thread/turn/window IDs, request kind,
lineage, sandbox/workspace metadata, timing, and compaction details.
- Treat app-server `responsesapi_client_metadata` as enrichment for the
Codex turn metadata blob while preventing those extras from overriding
Codex-owned fields.
- Use the same metadata path for normal turns, websocket prewarm, local
compaction, remote v1 compaction, and remote v2 compaction.
- Keep websocket connection-only preconnect metadata separate so
handshakes carry compatibility identity headers without inventing a fake
turn metadata blob.
## Verification
- `cargo check -p codex-core`
- `just fix -p codex-core`
## Summary
- replace raw compaction `error` analytics with `codex_error_kind` and
`codex_error_http_status_code`
- derive compaction error telemetry from `CodexErr` using the same
`CodexErrKind` mapping and HTTP status helper used by turn events
- remove the pre-compact hook stop reason from the internal compaction
outcome now that it is no longer emitted as raw analytics text
## Why
Compaction `error` was a raw `CodexErr::to_string()` value, which can
carry free-form provider or user-derived text. Structured Codex error
fields preserve useful low-cardinality telemetry without sending the raw
string.
## Validation
- `just fmt`
- `just test -p codex-analytics`
- `just test -p codex-core
compact::tests::build_token_limited_compacted_history_appends_summary_message`
Attempted `just test -p codex-core`; the changed crate compiled, but the
full target failed in unrelated environment-dependent tests such as
missing helper binaries and shell snapshot timeouts.
## Summary
- add nullable `cached_input_tokens` to the compaction analytics event
- populate it from response usage for compaction v2
- leave it `null` for other compaction implementations
This adds visibility into prompt-cache usage for v2 compaction without
changing compaction behavior.
## Testing
- `just test -p codex-analytics`
- `just test -p codex-core
collect_compaction_output_accepts_additional_output_items`
## Why
The model should be able to see bounded context-window budget metadata
when the `token_budget` feature is enabled. The full-window message is
only injected with full context, while normal turns get a smaller
follow-up only when reported usage first crosses a budget threshold.
## What changed
- Added the `TokenBudget` feature flag.
- Added `<token_budget>` developer fragments for full context-window
metadata and current-window remaining tokens.
- Inserted the threshold message during normal turn handling by
comparing token usage before and after sampling, avoiding persistent
threshold bookkeeping.
- Added core integration coverage for full-context-only metadata and
25/50/75 percent threshold messages.
## Verification
- `just test -p codex-core token_budget`
- `git diff --check`
## Why
Compaction window identity is part of session history, not model-client
transport state. Persisting it with the compacted rollout item lets
resumed threads continue from the reconstructed window without keeping
mutable window state on `ModelClient`.
## What changed
- Added `window_id` to `CompactedItem` and stamp it when
`replace_compacted_history` installs compacted history.
- Moved auto-compact window id ownership into `AutoCompactWindow` /
`SessionState`; `ModelClient` now receives the request window id from
callers instead of storing it.
- Returned `window_id` from rollout reconstruction for resume.
Reconstruction uses the newest surviving compacted item's stored
`window_id` when present, and falls back to the legacy compacted-item
count when it is absent.
- Kept fork startup at the fresh default window id and updated direct
model-client tests to pass explicit test window ids.
## Validation
- `cargo check -p codex-core --tests`
## Why
Compaction analytics adds retained image count and compaction summary
output tokens for v1.5 specifically.
## What changed
- Add nullable `retained_image_count` and `compaction_summary_tokens`
fields to `codex_compaction_event`.
- Populate them only for `responses_compaction_v2`: retained images come
from the retained v2 compacted history, and summary tokens come from
`response.completed.token_usage.output_tokens`.
- Leave local and legacy remote compaction events as `null` for these
detail fields.
## Verification
- `just fmt`
- `just fix -p codex-core`
- `just test -p codex-core
build_v2_compacted_history_counts_retained_input_images`
- `git diff --check`
## Summary
- accept non-empty model-defined reasoning effort values while
preserving built-in effort behavior
- propagate the non-Copy effort type through core, app-server, TUI,
telemetry, and persistence call sites
- preserve string wire encoding and expose an open-string schema for
clients
- update model selection and shortcut behavior for model-advertised
effort values
## Root cause
`ReasoningEffort` gained a string-backed custom variant, so it could no
longer implement `Copy` or rely on derived closed-enum serialization.
Existing consumers still moved effort values from shared references and
assumed a fixed built-in value set.
## Validation
- `just fmt`
- Local tests and compilation were not run per request; relying on CI.
## Why
Compaction analytics need token counts that better represent the request
being compacted. The existing session snapshot can diverge from the
actual remote compaction request after output rewriting, and remote v2
can use server-side Responses usage when available.
## What changed
- Add an optional `active_context_tokens_before` override to
`CompactionAnalyticsAttempt::track(...)` for remote compaction when it
has a better before-token value than the begin-time session snapshot.
The local `/compact` path passes no override.
- For remote v1 `responses_compact`, subtract the estimated token delta
from pre-compaction output rewriting from the session snapshot, capped
by locally-added tokens since the last successful API response.
- For remote v2 `responses_compaction_v2`, use the same bounded
output-rewrite fallback as remote v1, then overwrite
`active_context_tokens_before` with server `token_usage.input_tokens`
from the `response.completed` event when present.
- Keep the existing v2 compaction-output validation while carrying the
completed response token usage through `collect_compaction_output`.
## Verification
- `just fmt`
- `just test -p codex-core
collect_compaction_output_accepts_additional_output_items`
- `git diff --check`
## Summary
- add analytics-only `CodexErr` telemetry to `codex_turn_event` while
leaving existing `turn_error` unchanged
- record terminal `CodexErr` facts from core immediately before the
existing turn error event is sent
- emit source-truth `codex_error_*` fields for downstream analytics,
including the raw `CodexErr::InvalidRequest(String)` message as
`codex_error_subreason`
## Validation
- `just test -p codex-analytics`
- attempted `just test -p codex-core`, but the local run timed out
across unrelated integration suites in this environment and is not being
used as validation
## Why
`codex_core` is consistently a bottleneck for incremental builds during
iteration. The simplest fix is to make the crate smaller.
## Summary
`codex-core` owns several reusable prompt renderers and static prompt
assets, which makes the crate harder to split apart.
Rename `codex-review-prompts` to `codex-prompts` and move shared review,
goal, permissions, compaction, realtime, hierarchical AGENTS.md, and
`apply_patch` prompts into it. Move prompt-only tests and update
consumers and `CODEOWNERS`.
## Validation
- `just test -p codex-prompts -p codex-apply-patch`
- `just test -p codex-core prompt_caching`
- Bazel builds for the affected crates
## Summary
- Add `request_kind` values for foreground turn, startup prewarm,
compaction, and detached memory model requests.
- Attach compaction dispatch metadata to local Responses, legacy
`/v1/responses/compact`, and remote v2 compact requests.
- Add the existing logical context-window identifier as `window_id` on
turn-owned model request metadata.
- Keep identity fields optional for detached memory requests, while
still emitting `request_kind="memory"` in non-git/no-sandbox workspaces.
## Root Cause
`x-codex-turn-metadata` has more than one producer. Foreground turns and
compaction requests own a real turn and should carry that turn identity.
Detached memory stage-one requests do not own a foreground turn, so
absent identity fields are valid rather than missing data. Startup
websocket prewarm is also a model request, but it has `generate=false`
and must not be counted as a foreground turn.
`thread_source` or session source identifies where a thread came from
(for example review, guardian, or another subagent). `request_kind`
identifies what the current outbound model request is doing (`turn`,
`prewarm`, `compaction`, or `memory`). A review or guardian thread can
issue either a normal turn request or a compaction request, so source
cannot replace request kind.
## Behavior / Impact
- Ordinary foreground requests send `request_kind="turn"`, their real
identity fields, and `window_id="<thread_id>:<window_generation>"`.
- Startup websocket warmup requests send `request_kind="prewarm"` so
they are not counted as foreground turns.
- Compaction requests send `request_kind="compaction"`, their real
owning turn identity, the existing `window_id`, and
`compaction.{trigger,reason,implementation,phase,strategy}`.
- Detached memory stage-one requests send `request_kind="memory"`
without `session_id`, `thread_id`, `turn_id`, or `window_id`; when no
workspace metadata exists, the kind-only header is still emitted.
- `session_id`, `thread_id`, `turn_id`, and `window_id` remain optional
in the header schema because detached memory requests do not own a
foreground turn or context window.
- `window_id` is not a new ID system: it is copied from the already-sent
`x-codex-window-id` / WS client metadata value at model-request dispatch
time.
- Existing `x-codex-window-id` HTTP/WS emission, value format,
generation advancement, resume behavior, and fork reset behavior are
unchanged.
- `request_kind`, `window_id`, and upstream turn-owned identity fields
remain schema-owned; input `responsesapi_client_metadata` cannot replace
their canonical values.
- No table, DAG, export, app-server API, or MCP `_meta` schema changes
are included.
A compaction attempt stopped by a pre-compact hook issues no model
request and therefore has no request header; its outcome remains in
analytics events. Status, error, duration, and token deltas also remain
analytics fields rather than request-header fields.
Future detached-memory attribution using a real initiating turn ID as
`trigger_turn_id` is intentionally not part of this PR.
## Sync With Main
- Final pushed head `716342e79` is rebased onto `origin/main@0d37db4b2`.
- The metadata conflict came from upstream `#24160`, which added
`forked_from_thread_id` on the same `turn_metadata` surface. Resolution
preserves that field and its protection from client metadata override
alongside this PR's request-kind, compaction, and window-id fields.
- While resolving the overlapping commits, I removed an accidental
recursive model-request overlay and a duplicate detached-memory header
builder before completing the rebase.
## Latency / User Experience Boundary
- Foreground turns perform no new filesystem, git, or network work. New
fields are inserted into metadata already serialized for outgoing
requests.
- Compaction issues the same model/HTTP requests with the same prompt,
model, service tier, and sampling settings; only metadata bytes change.
- Startup prewarm already sent metadata; it is now correctly classified
as `prewarm`.
- Non-git detached memory now sends a small kind-only metadata header
rather than no header.
- This client diff adds no user-visible latency mechanism beyond
negligible serialization and header bytes on already-existing requests.
## Validation
On conflict-resolved head `1d35c2cfb` based on `origin/main@487521733`:
- `just fmt` (passed)
- `just fix -p codex-core` (passed)
- `git diff --check origin/main...HEAD` (passed)
- `just test -p codex-core -E 'test(turn_metadata) |
test(websocket_first_turn_uses_startup_prewarm_and_create) |
test(responses_stream_includes_turn_metadata_header_for_git_workspace_e2e)
|
test(responses_websocket_forwards_turn_metadata_on_initial_and_incremental_create)
| test(remote_compact_v2_retries_failures_with_stream_retry_budget) |
test(window_id_advances_after_compact_persists_on_resume_and_resets_on_fork)'`
(`23 passed`; `bench-smoke` passed)
- `just test -p codex-app-server -E
'test(turn_start_forwards_client_metadata_to_responses_request_v2) |
test(turn_start_forwards_client_metadata_to_responses_websocket_request_body_v2)
| test(auto_compaction_remote_emits_started_and_completed_items)'` (`3
passed`; `bench-smoke` passed)
- `just test -p codex-memories-write` (`29 passed`; `bench-smoke`
passed)
## Why
[Recent PR](https://github.com/openai/codex/pull/22709) removed
`trace_id` from `TurnContextItem`.
## What changed
- Add to `TurnStartedEvent` so rollout consumers can correlate turns
with telemetry traces.
- Note that the branch name is out of date because I originally re-added
to `TurnContextItem`, but we decided to move it to `TurnStartedEvent`.
## Verification
- `cargo test -p codex-protocol`
- `cargo test -p codex-core --lib
regular_turn_emits_turn_started_without_waiting_for_startup_prewarm`
- `cargo test -p codex-core --test all
emits_warning_when_resumed_model_differs`
- `cargo test -p codex-rollout`
- `cargo test -p codex-state`
## Why
Compaction now installs replacement history inside the session, but the
turn and compaction callers were still reaching into
`ModelClientSession` to reset websocket transport state after that
install. That made a transport-level reset part of the compaction API
even though websocket incremental request selection already checks
whether the next request is a strict extension of the previous one and
falls back to a full `response.create` when it is not.
## What changed
- Removed the compaction-side calls to `reset_websocket_session` from
`compact.rs` and `session/turn.rs`.
- Simplified pre-sampling and mid-turn compaction helpers so they return
`CodexResult<()>` instead of carrying a reset flag.
- Made `ModelClientSession::reset_websocket_session` private to
`client.rs`, leaving only the websocket timeout recovery path inside the
client as a caller.
## Validation
- `cargo test -p codex-core --test all
responses_websocket_creates_on_non_prefix`
- `cargo test -p codex-core --test all
steered_user_input_waits_for_model_continuation_after_mid_turn_compact`
- `cargo test -p codex-core --test all
pre_sampling_compact_runs_on_switch_to_smaller_context_model`
## Why
Dogfooder feedback exposed two correctness gaps in normal-loop overflow
recovery:
1. a sampling request that hit `ContextWindowExceeded` could keep
re-entering auto-compaction indefinitely if the compacted retry still
did not fit, and
2. local compact-history rebuilds flattened user messages down to text,
so an overflowing `[image, "what is this?"]` turn could be retried
without the image after compaction.
That means recovery could either fail to terminate cleanly or proceed
with a materially weakened version of the user request.
## What changed
- Move normal-loop `ContextWindowExceeded` handling into the sampling
retry loop, so successful rescue compaction consumes the provider retry
budget instead of creating an unbounded outer-turn loop.
- Keep compacted user-history rebuilds structured:
`collect_user_messages` now carries user `UserInput` content rather than
flattened strings, and `build_compacted_history` reconstructs full user
messages from that structured representation.
- Preserve image inputs while retaining the existing text-budget
truncation behavior for compacted user history.
- Preserve existing compaction-task failure handling and client-session
reset behavior while bounding repeated overflow retries.
- Add focused regression coverage for:
- recovery after a normal-loop overflow,
- retry-budget exhaustion after repeated overflow,
- local recovery preserving image + text input,
- remote recovery preserving image + text input,
- remote compaction v2 preserving image + text input, and
- compaction failure still terminating cleanly.
The main behavior changes are in `codex-rs/core/src/session/turn.rs` and
`codex-rs/core/src/compact.rs`.
## Verification
- Not run locally; relying on PR CI for this update.
---------
Co-authored-by: Codex <noreply@openai.com>
Based on work from Vincent K -
https://github.com/openai/codex/pull/19060
<img width="1836" height="642" alt="CleanShot 2026-04-29 at 20 47 40@2x"
src="https://github.com/user-attachments/assets/b647bb89-65fe-40c8-80b0-7a6b7c984634"
/>
## Why
Compaction rewrites the conversation context that future model turns
receive, but hooks currently have no deterministic lifecycle point
around that rewrite. This adds compact lifecycle hooks so users can
audit manual and automatic compaction, surface hook messages in the UI,
and run post-compaction follow-up without overloading tool or prompt
hooks.
## What Changed
- Added `PreCompact` and `PostCompact` hook events across hook config,
discovery, dispatch, generated schemas, app-server notifications,
analytics, and TUI hook rendering.
- Added trigger matching for compact hooks with the documented `manual`
and `auto` matcher values.
- Wired `PreCompact` before both local and remote compaction, and
`PostCompact` after successful local or remote compaction.
- Kept compact hook command input to lifecycle metadata: session id,
Codex turn id, transcript path, cwd, hook event name, model, and
trigger.
- Made compact stdout handling consistent with other hooks: plain stdout
is ignored as debug output, while malformed JSON-looking stdout is
reported as failed hook output.
- Added integration coverage for compact hook dispatch, trigger
matching, post-compact execution, and the audited behavior that
`decision:"block"` does not block compaction.
## Out of Scope
- Hook-specific compaction blocking is not implemented;
`decision:"block"` and exit-code-2 blocking semantics are intentionally
unsupported for `PreCompact`.
- Custom compaction instructions are not exposed to compact hooks in
this PR.
- Compact summaries, summary character counts, and summary previews are
not exposed to compact hooks in this PR.
## Verification
- `cargo test -p codex-hooks`
- `cargo test -p codex-core
manual_pre_compact_block_decision_does_not_block_compaction`
- `cargo test -p codex-app-server hooks_list`
- `cargo test -p codex-core config_schema_matches_fixture`
- `cargo test -p codex-tui hooks_browser`
## Docs
The developer documentation for Codex hooks should be updated alongside
this feature to document `PreCompact` and `PostCompact`, the
`manual`/`auto` matcher values, and the compact hook payload fields.
---------
Co-authored-by: Vincent Koc <vincentkoc@ieee.org>
## Why
This adds the `remote_compaction_v2` client path so remote compaction
can run through the normal Responses stream and install a
`context_compaction` item that trigger a compaction.
The goal is to migrate some of the compaction logic on the client side
We keeps the v2 transport behind a feature flag while letting follow-up
requests reuse the compacted context instead of falling back to the
legacy compaction item shape.
## What changed
- add `ResponseItem::ContextCompaction` and refresh the generated
app-server / schema / TypeScript fixtures that expose response items on
the wire
- add `core/src/compact_remote_v2.rs` to send compaction through the
standard streamed Responses client, require exactly one
`context_compaction` output item, and install that item into compacted
history
- route manual compact and auto-compaction through the v2 path when
`remote_compaction_v2` is enabled, while keeping the existing remote
compaction path as the fallback
- preserve the new item type across history retention, follow-up request
construction, telemetry, rollout persistence, and rollout-trace
normalization
- add targeted coverage for the feature flag, `context_compaction`
serialization, rollout-trace normalization, and remote-compaction
follow-up behavior
## Verification
- added protocol tests for `context_compaction`
serialization/deserialization in `protocol/src/models.rs`
- added rollout-trace coverage for `context_compaction` normalization in
`rollout-trace/src/reducer/conversation_tests.rs`
- added remote compaction integration coverage for v2 follow-up reuse
and mixed compaction output streams in
`core/tests/suite/compact_remote.rs`
---------
Co-authored-by: Codex <noreply@openai.com>
## Why
Several legacy `EventMsg` variants were still emitted or mapped even
though clients either ignored them or had moved to item/lifecycle
events. `Op::Undo` had also degraded to an unavailable shim, so this
removes that dead task path instead of preserving a command that cannot
do useful work.
`McpStartupComplete`, `WebSearchBegin`, and `ImageGenerationBegin` are
intentionally kept because useful consumers still depend on them: MCP
startup completion drives readiness behavior, and the begin events let
app-server/core consumers surface in-progress web-search and
image-generation items before the final payload arrives.
## What Changed
- Removed weak legacy event variants and payloads from `codex-protocol`,
including legacy agent deltas, background events, and undo lifecycle
events.
- Kept/restored `EventMsg::McpStartupComplete`,
`EventMsg::WebSearchBegin`, and `EventMsg::ImageGenerationBegin` with
serializer and emission coverage.
- Updated core, rollout, MCP server, app-server thread history,
review/delegate filtering, and tests to rely on the useful replacement
events that remain.
- Removed `Op::Undo`, `UndoTask`, the undo test module, and stale TUI
slash-command comments.
- Stopped agent job/background progress and compaction retry notices
from emitting `BackgroundEvent` payloads.
## Verification
- `cargo check -p codex-protocol -p codex-app-server-protocol -p
codex-core -p codex-rollout -p codex-rollout-trace -p codex-mcp-server`
- `cargo test -p codex-protocol -p codex-app-server-protocol -p
codex-rollout -p codex-rollout-trace -p codex-mcp-server`
- `cargo test -p codex-core --test all suite::items`
- `just fix -p codex-protocol -p codex-app-server-protocol -p codex-core
-p codex-rollout -p codex-rollout-trace -p codex-mcp-server`
- Earlier coverage on this PR also included `codex-mcp`, `codex-tui`,
core library tests, MCP/plugin/delegate/review/agent job tests, and MCP
startup TUI tests.
## Summary
- Remove `ghost_snapshot` / `GhostCommit` from the Responses API surface
and generated SDK/schema artifacts.
- Keep legacy config loading compatible, but make undo a no-op that
reports the feature is unavailable.
- Clean up core history, compaction, telemetry, rollout, and tests to
stop carrying ghost snapshot items.
## Testing
- Unit tests passed for `codex-protocol`, `codex-core` targeted undo and
compaction flows, `codex-rollout`, and `codex-app-server-protocol`.
- Regenerated config and app-server schemas plus Python SDK artifacts
and verified they match the checked-in outputs.
## Summary
Wires rollout trace recording into `codex-core` session and turn
execution. This records the core model request/response, compaction, and
session lifecycle boundaries needed for replay without yet tracing every
nested runtime/tool boundary.
## Stack
This is PR 2/5 in the rollout trace stack.
- [#18876](https://github.com/openai/codex/pull/18876): Add rollout
trace crate
- [#18877](https://github.com/openai/codex/pull/18877): Record core
session rollout traces
- [#18878](https://github.com/openai/codex/pull/18878): Trace tool and
code-mode boundaries
- [#18879](https://github.com/openai/codex/pull/18879): Trace sessions
and multi-agent edges
- [#18880](https://github.com/openai/codex/pull/18880): Add debug trace
reduction command
## Review Notes
This layer is the first live integration point. The important review
question is whether trace recording is isolated from normal session
behavior: trace failures should not become user-visible execution
failures, and recording should preserve the existing turn/session
lifecycle semantics.
The PR depends on the reducer/data model from the first stack entry and
only introduces the core recorder surface that later PRs use for richer
runtime and relationship events.
rka-oai