304 Commits
Author SHA1 Message Date
Dmitriy Kovalenko 9c84e89ce0 Skip SQLite page reclamation on Linux NFS mounts (#54417)
## Why

SQLite page reclamation on NFS creates I/O load and contention.

## What changed

Detect whether `sqlite_home` is on NFS on Linux and skip reclamation before opening a maintenance connection. Cache successful filesystem probes across `SqliteConfig` clones while allowing failed probes to retry.

## Testing

Add tests for shared probe caching and recovery after a missing directory is created. Update reclamation coverage to expect no reclaimed pages or file shrinkage on NFS.

GitOrigin-RevId: 4f627cabc6ad712171654c3f8b8659d2f632bbba
2026-10-15 21:07:40 +00:00
Dmitriy Kovalenko 1ef30bcbe7 Preserve existing SQLite journal modes on startup (#54389)
## Why

Opening an initialized database in `DELETE` journal mode previously attempted to convert it to `WAL`, which could fail while another connection held a write lock.

## What changed

- Initialize `WAL` and incremental auto-vacuum only for databases without a schema, including files left by interrupted initialization.
- Preserve journal and vacuum modes for initialized databases, use `synchronous = EXTRA` for `DELETE` connections, and retain `NORMAL` for `WAL`.
- Close failed initialization pools before retrying and restrict background page reclamation to `WAL` databases.

## Testing

Extend SQLite tests to cover interrupted initialization, opening `DELETE` databases under a write lock across all pool connections, persistence after reopening, and lock release after failed initialization.

GitOrigin-RevId: 23acecfe472618dde8545815e9a1058a51591235
2026-10-15 19:15:04 +00:00
willwang-openai 7ae8319917 Retry SQLite operations on transient writer contention (#54369)
## Why

Competing SQLite writers can cause database opening and writes to fail with `SQLITE_BUSY`. Deferred write transactions can also encounter contention after taking a read snapshot.

## What changed

- Add up to two retries with 100 ms and 200 ms delays for SQLite busy errors during pool opening, writer admission, and standalone writes.
- Use a shared `BEGIN IMMEDIATE` helper across state, thread history, and agent message board writes to acquire the writer slot before reading.
- Preserve bound arguments when retrying standalone writes, and commit writes using `RETURNING` before exposing their results. Retry transaction admission without replaying an already-started transaction.

## Testing

Add tests using real SQLite writer contention to verify recovery after a competing writer releases its lock. Queue tests cover enqueue, update, and delete, checking preserved payloads, single revision increments, and no duplicate messages or unintended deletions.

GitOrigin-RevId: ac5bd31a87c67cc47fb385bf3fa5435a54ff70f0
2026-10-15 18:06:57 +00:00
pakrym-oai 5ef96ab278 Add opt-in output token replay for OpenAI requests (#52742)
## What changed

- Add the disabled-by-default `output_token_replay` feature to request `output.encrypted_content` from OpenAI providers.
- Preserve encrypted message and tool-call output, message and function-call status, and output-text `annotations` and `logprobs` through history and replay. Update the generated protocol schemas and SDK types.
- Strip output ciphertext from requests to other providers while retaining it in stored history, including after replay is disabled. Remove encrypted content from memory extraction inputs.

## Testing

Add integration coverage for the feature/provider combinations, metadata replay across follow-up requests, and history preservation when resuming with replay disabled.

GitOrigin-RevId: 77729a021b6771933402aebe74ece8eb39c057e2
2026-10-10 01:25:00 +00:00
Brent Traut 0ada5d8806 Expose experimental durable thread read state in the app server (#52350)
## What changed

- Add experimental `readState` to `thread/read` and a thread-ID-keyed `readStates` map to `thread/list` for eligible durable local user threads. Each receipt contains `firstUnread` and an opaque `revision`.
- Publish new visible terminal results before `turn/completed`, after flushing the rollout. Reading snapshots leaves activity unread; replayed history does not publish new activity.
- Clear unread positions removed by a paginated thread revert while preserving explicit thread-start marks and concurrent receipt updates.

## Testing

Add an integration test covering completed-turn receipts, repeated reads without acknowledgement, matching list receipts, and revert cleanup with a changed revision.

GitOrigin-RevId: e7f6ddfbb1fc2fa95d1ef2876d7f0f7b9f99ff65
2026-10-09 01:22:30 +00:00
Brent Traut b6e8a2e207 Add durable thread read state with revision-checked updates (#52337)
## Why

A stale read acknowledgement must not clear unread results published after the client’s snapshot or overwrite another read-state edit. Read state also needs to survive thread metadata rebuilds.

## What changed

- Add `thread_read_receipts` and state APIs to query read state, publish thread attention, and mark threads read or unread.
- Track the first unread turn and require matching revisions for explicit edits, returning the current state on conflicts. Allow manually marking even an empty thread unread.
- Retain the last published turn across acknowledgements to suppress duplicate delivery. Keep receipts independent of rebuildable metadata and remove them on thread deletion.
- Return initial read states without creating receipt rows.

## Testing

Add tests covering stale edits, thread-specific initial revisions, repeated unread marks, duplicate publication, persistence across metadata rebuilds and restarts, clearing a reverted turn’s unread position, and deleted-thread handling.

GitOrigin-RevId: fee8c05c20903b1d7aea7e8b61c9248c7f623a72
2026-10-09 00:41:27 +00:00
zbarsky-openai 9dd798805f Prevent recursive Tokio tracing in telemetry and SQLite logs (#51775)
## Why

Tokio runtime tracing can cause SQLite log writes to enqueue more writes and exporter activity to generate further OTLP exports.

## What changed

- Exclude `tokio` and `runtime` spans from OTLP export alongside `h2` spans.
- Limit `tokio::` and `runtime::` targets to `INFO` in the default log filter, and reject their `TRACE` events directly in the SQLite sink even without an outer filter.
- Box and pin the session-loop future before spawning it.
- Reorder tracing and Tokio test attributes, restrict the skills concurrency test's subscriber to the catalog-refresh span, and use the default log filter in the SQLite formatter comparison test.

GitOrigin-RevId: f545013a2469088d7f9890c1f46bb93eb5446d2d
2026-10-07 17:28:32 +00:00
jif 30bdfec59d Persist Guardian review failures for reports across restarts (#51651)
## Why

Failed Guardian review evidence was process-local, so restarting Codex lost the context needed for later diagnostic reports.

## What changed

- Persist failed-review records in SQLite before reviewer cleanup, retaining at most 64 records globally, eight per thread, and 8 MiB of evidence. Delete persisted evidence when its reviewed thread is deleted.
- Recover and deduplicate persisted and in-memory records for the reported task tree when uploading `auto-review-failures.jsonl`.
- Bound persistence attempts to 250 ms and retain an in-memory fallback when storage is unavailable. Keep capture gated by user opt-in and non-ephemeral sessions, and export evidence only when logs are included.

## Testing

Add coverage for persistence across reopen and process restart, retention limits, cascading deletion, busy-storage fallback, and report uploads with and without logs, including SQLite log failures.

GitOrigin-RevId: ea2da1818d4a5da3da29f211042ca45e834342bc
2026-10-07 11:18:05 +00:00
pakrym-oai 28cd3e2501 Remove obsolete fields from persisted turn context (#51492)
## What changed

Remove `workspace_roots`, `current_date`, `timezone`, `network`, `personality`, and `multi_agent_mode` from `TurnContextItem` and stop recording them in rollouts. Update the precomputed protocol schema accordingly.

Make `summary` optional when reading turn context and write `"none"` as a compatibility placeholder for older readers, rather than persisting the active reasoning-summary setting.

## Testing

Update protocol tests to cover omitted `summary` fields and compatibility-placeholder serialization. Extend compaction tests to verify that model requests retain the updated reasoning-summary setting after compaction while the rollout checkpoint records the placeholder.

GitOrigin-RevId: d49eabaa1b340be1d79f7f09115281a307ea8a64
2026-10-06 22:39:06 +00:00
Eddie Chen cfc946f4e1 Expose and persist turn lineage across the app server (#51415)
## Why

Clients starting delegated work need to attribute descendant turns to the original turn and retain that attribution through completion and history reads.

## What changed

- Accept optional `parentTurnId` and `rootTurnId` in `turn/start`, and expose `rootTurnId` on returned turns and lifecycle notifications.
- Default new turns to their own root when none is supplied. Preserve the active turn's root when steering, ignoring incoming lineage fields.
- Carry root IDs through completion, interruption, recovery, and fork boundaries, and persist them for paginated history reads. Leave unknown roots null in older history.
- Update protocol schemas and generated TypeScript and Python bindings.

## Testing

Extend coverage for default and explicit roots, steering without changing lineage, interrupted and recovered turns, history reads after restart in both history modes, nested fork pagination, and compatibility with older database writers.

GitOrigin-RevId: f1a570a12bd11742929fd2e8b017e5d35aa07dcb
2026-10-06 16:56:39 +00:00
Eric Traut 80e0b51c9e Make session lookup pagination stable and report listing failures (#51230)
## Why

Session activity can move unread threads ahead of a pagination cursor, and timestamp-only cursors can skip threads with equal creation times. Either can prevent a session label lookup from detecting duplicate labels. Database failures must also remain distinguishable from an exhausted listing.

## What changed

- Resolve session labels using database-only listings ordered by `CreatedAt`, with thread IDs breaking timestamp ties. Accept unique matches across pages with valid timestamp-and-ID cursors, while retaining ambiguity errors for older cursors that cannot establish uniqueness.
- Preserve timestamp index ranges for database continuation queries and timestamp-only cursors for listings that may fall back to the filesystem.
- Propagate database-only listing and name-resolution failures. Keep best-effort name enrichment for ordinary listings and retry latest-session lookup with scan-and-repair after a database-only error.
- Keep loaded sessions selectable in the resume and fork pickers when a later page fails, stop further pagination, and display `Could not load more sessions`.

## Testing

Add regression coverage for creation-time continuation after session activity, cursor query plans, filesystem fallback, database and name-query errors, cross-page label uniqueness, older server cursors, and selection after picker pagination failures.

GitOrigin-RevId: 264ab2e69b8ad12d9edaa31fd253cb9805a46a90
2026-10-06 02:14:22 +00:00
Benjamin Carlsson 09bced5ad9 Populate thread previews from delegated task inputs (#50462)
## Why

Delegated turns can begin without a user message, leaving threads without a task preview. Populate the preview from delegated input so these threads are discoverable before a user follow-up.

## What changed

- Extract previews from completed `create_thread` and `send_message_to_thread` outputs in the `codex_app` and `codex_tui` namespaces.
- Unwrap recognized `<codex_delegation>` inputs and decode their escaped task text for display, preserving model content. Keep plain or unrecognized wrappers as text and ignore blank inputs.
- Fill missing previews in state metadata extraction and live thread metadata updates without replacing existing previews or setting the title or `first_user_message`.

## Testing

Add tests for delegation decoding and tool filtering, preview-only metadata updates, preservation of later user-message metadata, and database listing before a user follow-up in both legacy and paginated history modes.

GitOrigin-RevId: f7b3c60ef75e08c76b4084d043e3967bf2bade22
2026-10-02 23:34:11 +00:00
joeytrasatti-openai 26a49b3882 Add paginated reverse lookup for thread attachments (#50083)
## What changed

Add `StateRuntime::list_thread_attachment_threads` to find threads by exact `attachment_type` and `identity_key`, including threads without previews. Return thread IDs and archive state, with filters for all, archived, or non-archived threads.

Order results by thread ID and bind pagination cursors to the attachment identity and archive filter. Add a database index on `(attachment_type, identity_key, thread_id)` to support the lookup.

## Testing

Add coverage for exact identity matching, archive filters, pagination, rejection of cursors reused with different lookup parameters, and membership updates after attachment removal or thread deletion.

GitOrigin-RevId: 21f32eb0cb1845a82d740d67b374080b6e12eb03
2026-10-01 19:55:36 +00:00
Dmitriy Kovalenko 596ae94fb0 Classify SQLite corruption using typed error codes (#49710)
## What changed

Preserve the underlying error in `LocalStateDbStartupError` and use its error chain to decide whether CLI startup failures qualify for automatic database backup and recovery.

Replace corruption checks against error-message text with SQLite result-code classification, including SQLx extended result codes and directly wrapped `SqliteError` values.

## Testing

Update the embedded startup corruption test to verify that the startup error retains a detectable SQLite corruption cause and the failed database path.

GitOrigin-RevId: 6e8ecd0ee67f3c239183ddea3fc75c4e4cd5744a
2026-09-30 19:00:00 +00:00
Dmitriy Kovalenko 3620b2caf8 Detect SQLite corruption during startup and preserve recovery backups (#49701)
## Why

A database can open successfully while containing corruption that startup previously missed. Detecting it before runtime initialization completes allows recoverable databases to be rebuilt while preserving the damaged files.

## What changed

- Run `PRAGMA quick_check(1)` when opening writable databases, with a 100 ms scan budget. Share attempts across cloned `SqliteConfig` instances by file identity so repeated opens skip validation and replacement files are checked again.
- Back up and rebuild runtime databases that support recovery. Report corruption without rebuilding thread history or other databases without an automatic recovery policy. Treat interrupted or locked checks as incomplete rather than confirmed corruption.
- Collect startup backups across recovery retries and include their locations in the app server warning. Explain that saved conversations can restore the thread list and history, while some database-only metadata may be unavailable.
- Record confirmed corruption in `codex.sqlite.corruption.count`.

## Testing

Add tests for recovery policies, backup preservation, validation caching and concurrency, cancelled and interrupted checks, and locked databases. Add app server coverage for recovery warnings and saved conversation restoration, a TUI warning snapshot, and message board post preservation.

GitOrigin-RevId: 43e26328888e0301def642d69bd324acee867566
2026-09-30 18:29:46 +00:00
Dmitriy Kovalenko 8ea2c0e0d4 Prune diagnostic logs periodically by age and database size (#49425)
## Why

Startup-only cleanup leaves expired logs in long-running sessions, and age-based retention alone does not limit database usage.

## What changed

Run cleanup in the background immediately after initialization and every 30 minutes. Start with the existing 10-day retention window, then halve it until occupied SQLite pages fit within a 64 MiB budget or the window reaches one second. Preserve logs within that final second even when they exceed the budget.

Exclude free pages from the budget so SQLite can reuse them, and retain the passive checkpoint after the first sweep. Stop the task when the runtime is dropped or the log pool is closed.

## Testing

Add tests for the 10-day boundary, repeated retention-window halving, free-page accounting, the one-second retention floor, immediate startup cleanup, and periodic cleanup without new writes or a restart. Update existing log tests to use current timestamps so background retention does not remove their fixtures.

GitOrigin-RevId: c6eeb5ec13006d4a03354e08322827429be53ce8
2026-09-30 01:34:12 +00:00
Dmitriy Kovalenko 16a7c0f0eb Filter graceful shutdown guard and trigger traces from SQLite logs (#49414)
## What changed

Set the default SQLite log filter to `DEBUG` for `tokio_graceful::guard` and `tokio_graceful::trigger`, dropping their `TRACE` events while retaining debug and higher-level diagnostics.

## Testing

Extend the SQLite sink filter test to verify that guard and trigger traces are dropped, their debug events are retained, and `tokio_graceful::shutdown` trace events remain available.

GitOrigin-RevId: c98495c834db35f3d8799ab4f0a1f7a82a53c8ef
2026-09-30 00:50:50 +00:00
jif c2837d8ece Batch metadata reads when resolving thread names (#49305)
## Why

Thread name resolution performs a separate SQLite metadata lookup for each thread. Batch these reads to reduce database queries for large result sets.

## What changed

Add `StateRuntime::get_threads` and use it in `resolve_thread_names`. Fetch metadata in batches of at most 999 IDs to stay within SQLite's bind limit, skipping missing or invalid records independently so valid metadata remains available.

## Testing

Add regression coverage for reads spanning multiple batches, invalid and missing records, duplicate IDs, and empty input.

GitOrigin-RevId: a403416f1da5adea56705c70839850760d3eb550
2026-09-29 16:36:07 +00:00
Benjamin Carlsson c2d2f422e6 Preserve SQLite vacuum modes and surface pool initialization errors (#49102)
## Why

Changing an existing database from `FULL` to `INCREMENTAL` auto-vacuum requires a writer lock and can block pool initialization. SQLx also retries connection initialization failures, masking the original error as `PoolTimedOut`.

## What changed

Preserve existing auto-vacuum modes when opening writable pools, while continuing to initialize empty databases with incremental auto-vacuum. Return the first connection initialization error directly to the caller, retaining SQLx's normal error handling for later lazy connections.

## Testing

Add regression tests for fresh database settings, opening an existing `FULL` auto-vacuum database under a write lock, and preserving the WAL conversion lock error while allowing a subsequent open to succeed after the lock is released.

GitOrigin-RevId: b586c68e43a5a0d3b47268378716919d01b749cf
2026-09-28 23:44:39 +00:00
Dmitriy Kovalenko 33a0f766a6 Reclaim unused SQLite log database pages in the background (#49069)
## Why

Deleting logs leaves free SQLite pages available for reuse without shrinking the database file. Reclaim excess free space while preserving a reserve for future writes.

## What changed

- Start a background incremental-vacuum worker with `StateRuntime`, enabling reclamation only for the logs database. Use a per-home file lock to select one worker and wait for it to finish during shutdown.
- Reclaim pages when at least 64 MiB and 25% of the database are free, preserving a reserve of at least 16 MiB or 10% of the database.
- Use short transactions with page and cooperative time budgets, defer work on contention, and reduce batch sizes with exponential backoff when interrupted passes make no progress.
- Record reclamation counts, durations, and reclaimed pages in telemetry.

## Testing

Add a regression test that interrupts a vacuum commit, verifies foreground writes can resume without data loss, then confirms a resumed pass shrinks the database and passes `PRAGMA integrity_check`.

GitOrigin-RevId: 8421f87b0aeef6540eda5690c84216906673c72e
2026-09-28 20:19:10 +00:00
jif 46fdd5ef39 Avoid SQLite stalls from connection setup and stderr span logging (#49032)
## Why

Setting `auto_vacuum` on every connection can wait for a writer lock even when the mode is unchanged. Logging every span entry and exit can also block SQLx workers on undrained stderr while they hold a write transaction.

## What changed

- Set incremental vacuum only for empty databases or databases using full auto-vacuum, before enabling WAL. Preserve existing nonempty databases with auto-vacuum disabled.
- Emit span creation and closure events in app-server text and JSON stderr logs, preserving busy/idle timings and explicit events while omitting span entry and exit records.

## Testing

Add regression tests for opening and expanding writable pools while another connection holds a writer lock, and for transaction completion with undrained stderr in both log formats. Verify vacuum modes and retained log diagnostics.

GitOrigin-RevId: 40d4ef32c7efc49f4240b987277ba45551259335
2026-09-28 17:00:25 +00:00
acrognale-oai 7596ab19a6 Keep archived threads with empty previews visible in listings (#48199)
## Why

Archived threads without previews were filtered out of thread listings, even though they had valid session metadata.

## What changed

- Include archived threads with empty previews in both SQLite and rollout-file listings.
- Add archive-specific indexes for creation, update, and recency ordering so these listings can use indexed pagination.

## Testing

Add regression coverage for paginated archive listings with and without SQLite, archive query plans that avoid temporary sorting, and metadata reads that preserve source and file timestamps for empty archived threads.

GitOrigin-RevId: e29d0257b2e029bea5fdf94eed6dab1006d2a8f2
2026-09-25 20:11:47 +00:00
Dmitriy Kovalenko 3d6428510f Warn users when SQLite diagnostic log writes fail (#47887)
## Why

Failed SQLite writes leave diagnostic logs incomplete. Notify users before they close Codex so they can submit logs or run `codex doctor`.

## What changed

- Add a best-effort warning on the first log write failure in standalone and in-process app servers, while continuing to retain failure diagnostics in memory.
- Handle failures that occurred before the in-process warning reporter was attached.
- Tailor the warning's guidance to whether submitting logs is enabled.

## Testing

Add coverage for warning deduplication, both guidance variants, retained diagnostics after the notification sender is dropped, and SQLite flush failures before and after reporter attachment. Extend log upload integration coverage and add a TUI warnings snapshot.

GitOrigin-RevId: 12a74126d13c3568430e539a180cca47fa931fa9
2026-09-24 17:04:50 +00:00
Dmitriy Kovalenko a285066737 Preserve diagnostic logs when SQLite logging fails (#47886)
## Why

SQLite write failures can leave persisted logs incomplete, while log snapshots taken before collection miss errors raised during that collection.

## What changed

- Report SQLite flush failures directly to the in-memory log buffer without re-entering tracing, using an error category and batch size instead of raw error details.
- Track whether the log writer has lost a batch and make app-server diagnostic uploads use the memory buffer when SQLite logs are incomplete.
- Refresh log bytes after flushing and querying SQLite so the same submission includes collection failures, while preserving captured metadata and thread identity.

## Testing

Add regression tests for failure reporting without recursion, subsequent write recovery, metadata preservation, and healthy and failing SQLite collection. Add an upload integration test that captures attachments locally and checks failure diagnostics, memory-buffer fallback, and omission of raw SQLite error details.

GitOrigin-RevId: 51b0c4b69c977d9f4e74f2f245a9fd90fdb13761
2026-09-24 17:04:24 +00:00
Charlie Marsh 44b857c00e Clean up legacy Guardian thread metadata in SQLite (#47303)
## Why

Guardian prompts are synthetic review context, but legacy thread metadata can duplicate them in titles, previews, and first messages. Keep SQLite metadata small while retaining rollout JSONL as the canonical record.

## What changed

Add a migration for Guardian threads that:

- Replaces blank or prompt-derived titles with `Guardian review`.
- Preserves nonblank names and distinct nonblank titles, using the title as the name when no name exists.
- Sets `preview` to `Approval review` and clears `first_user_message`.

## Testing

Add a migration test covering prompt-derived and blank titles, custom titles and names, and unchanged metadata for a non-Guardian worker thread.

GitOrigin-RevId: 65b5ff5200a39a19fb11f494de6dca7710ff6cc6
2026-09-22 15:05:27 +00:00
Charlie Marsh 4f6c106cdd Use compact display metadata for Guardian review threads (#47302)
## Why

Guardian review threads use synthetic approval prompts as user messages. Deriving thread titles and previews from those prompts exposes large, unhelpful summaries in thread listings.

## What changed

- Default Guardian threads to the title "Guardian review" and preview "Approval review" across metadata creation, rollout reads, and resume.
- Skip synthetic user messages and goal updates when deriving Guardian display metadata, leaving `first_user_message` unset for new threads.
- Preserve custom names, including legacy indexed names, when listing, reading, resuming, and migrating threads to paginated history.

## Testing

Add regression coverage for compact metadata, resume without session metadata, custom names before and after migration, and app-server persistence, listing, and rename/resume behavior.

GitOrigin-RevId: 42aa707db20c3e57ffa5fd4c91ee313710be10d7
2026-09-22 15:02:54 +00:00
chiam-oai 772abc9425 Preserve and expose thread item lifecycle timestamps (#47114)
## What changed

- Persist producer-recorded item start and completion times in thread history, preserving existing non-null values on updates.
- Return nullable `startedAtMs` and `completedAtMs` Unix millisecond timestamps from `thread/items/list`, and update the generated schemas and TypeScript and Python types.
- Leave timestamps unknown for existing rows and older writers.

## Testing

Add coverage for exact timestamp persistence, protocol round trips with present and missing timestamps, paginated item responses, and compatibility with older database writers.

GitOrigin-RevId: 894853481085782ff54db1704799e41446d15e51
2026-09-21 22:23:26 +00:00
Jeremy Rose 23ae65bb06 Persist thread creator identity in rollouts and SQLite (#47113)
## What changed

- Capture optional `creator_user_id` and `creator_account_id` from authentication when creating a thread, including forks.
- Carry these fields through thread storage, rollout metadata, and a SQLite migration. Leave them absent when unavailable or when reading older threads.
- Preserve recorded creator values across resume and metadata updates. On revert, recover missing rollout values from SQLite before writing replacement metadata.

## Testing

Add coverage for ChatGPT and API-key sessions, resume under another account, forks using current authentication, metadata extraction and upserts, and repeated reverts after creator fields are removed from a rollout.

GitOrigin-RevId: 9839fa42f9392f12226002e2c27d05d5656f5695
2026-09-21 21:22:01 +00:00
Ahmed Ibrahim 3627bb91b2 Avoid building a host SQLite driver for SQLx macros (#47084)
## Why

Row derives and migration embedding do not need query macros or a host SQLite driver.

## What changed

- Remove the `macros` feature from `sqlx` and depend directly on `sqlx-macros` with only `derive` and `migrate` enabled.
- Update state and thread-store derives and migration embedding to use `sqlx_macros`.
- Enable `sqlx-sqlite`'s `offline` feature directly to preserve query-description support in the target driver.
- Pin `sqlx`, `sqlx-macros`, and `sqlx-sqlite` to `=0.9.0` to keep their versions aligned.

GitOrigin-RevId: 8a8ef691ff2862a7dad978a578c5206b52dfc0b6
2026-09-21 18:30:31 +00:00
joeytrasatti-openai 19286b8819 Copy current thread attachments into non-ephemeral forks (#45579)
## What changed

Copy the source thread's current attachments when creating a non-ephemeral fork, including forks at an earlier turn. Copies receive new attachment IDs and creation timestamps while preserving resource identities and payloads. Attachment membership can then change independently on either thread; referenced resources are not copied.

Await the atomic copy before publishing the fork. If copying fails, log the error and allow the conversation fork to succeed without attachments. Resuming a fork does not copy attachments again.

Document that clients should use `forkedFromId` on `thread/started` and fetch the fork's attachments with `thread/attachment/list`; copying does not emit per-attachment updates.

## Testing

Add coverage for atomic rollback, independent attachment membership, inheritance across history cutoffs, successful conversation forks after copy failures, and resuming without restoring removed attachments.

GitOrigin-RevId: 939d2c6a3073ccfcd922380c6beda45d3deac606
2026-09-15 02:33:27 +00:00
Matthew Zeng b5544d5732 Persist disabled plugin IDs in thread settings (#44332)
## What changed

- Add `disabled_plugin_ids` to thread startup options, settings overrides, snapshots, and persisted turn contexts. Omitted overrides preserve the current list; an empty list clears it.
- Restore the selection on resume and capture it before truncating history for a fork. Prefer the owning thread's latest settings snapshot, falling back only to the latest turn context so stale history cannot resurrect an older selection.
- Freeze the selection at turn admission and retain compatibility with histories that lack the new fields.

## Testing

Add coverage for owned settings, explicit clears, latest-turn fallback, and legacy snapshots. Extend settings tests to cover plugin selection during runtime restoration and compaction, and verify initial turn persistence without extra settings checkpoints in both history modes.

GitOrigin-RevId: 75ffbff5763cd93cc5ac69d73b08b237f59bc009
2026-09-09 23:23:59 +00:00
joeytrasatti-openai 130d6e4fba Add paginated thread attachment listing to the state runtime (#44330)
## What changed

Add `StateRuntime::list_thread_attachments` with page sizes from 1 to 100 and stable keyset ordering by `created_at` and `id`. Return a continuation cursor when more attachments remain, and reject malformed cursors or cursors belonging to another thread.

## Testing

Add coverage for thread-scoped pagination, cursor validation, and attachment persistence across reopening the database and archiving/unarchiving a thread. Verify listings reflect attachment removal and cascading deletion when a thread is deleted.

GitOrigin-RevId: e07433e1a675669082cc58f86b0dc4535d365ba3
2026-09-09 23:04:22 +00:00
joeytrasatti-openai 589874be81 Add transactional thread attachment mutations to the state runtime (#43949)
## What changed

- Replace the `ThreadArtifact` model and related exports with attachment terminology.
- Add `StateRuntime::add_thread_attachment` and `remove_thread_attachment` using SQLite transactions. Repeated additions for the same thread, attachment type, and identity key return the existing record without changing its payload or creation time.
- Enforce limits of 100 attachments per thread, 64 KiB per serialized payload, and 256 bytes each for nonblank attachment types and identity keys. Removal returns the deleted record or `NotFound` and frees capacity immediately. Both mutations reject unknown threads.

## Testing

Add tests for idempotency, thread isolation, removal outcomes, capacity reuse, invalid inputs, unknown threads, and concurrent additions creating exactly one record.

GitOrigin-RevId: a68fe5076832f071524f1b4fc87d08b96ac270b7
2026-09-09 00:18:35 +00:00
joeytrasatti-openai 9d88e9ae08 Rename thread artifacts to attachments in the state database (#43927)
## What changed

Add a migration that renames `thread_artifacts` to `thread_attachments` and
`artifact_type` to `attachment_type`, and recreates the thread lookup index
under the attachment name.

## Testing

Extend the migration test to verify that existing attachment data survives the
rename, section metadata is preserved, and the version 50 migrator tolerates
the updated database with `ignore_missing` enabled.

GitOrigin-RevId: 43955c4f69b91f92217efc1dc8e6a551d0716059
2026-09-08 22:02:55 +00:00
rhan-oai 6515a72db7 Preserve runtime workspace roots across thread resume (#43848)
## Why

Resuming a thread should retain its selected workspace folders, including additional roots and explicit empty selections. Resume overrides also need to survive a subsequent resume when no turn has run.

## What changed

- Persist `runtime_workspace_roots` in startup metadata and thread settings snapshots, separately from explicit environment selections and permission-profile roots.
- Restore roots from the latest snapshot owned by the resumed thread, falling back to owned startup metadata only when no snapshot exists. Honor explicit `runtimeWorkspaceRoots` overrides, retarget the old `cwd` root when `cwd` changes, deduplicate roots, and validate restored paths for the current host.
- Checkpoint effective settings on resume and restored settings after revert. Reload resume configuration if saved workspace roots change during loading.
- Normalize Windows rollout path spellings when matching thread search results, preserving selection of the correct rollout after revert, including compressed rollouts.

## Testing

Add regression coverage for workspace restoration, empty and explicit overrides, foreign paths, compaction and revert, resume checkpoints without recency changes, concurrent settings persistence, and rollout search path matching.

GitOrigin-RevId: d98d9d34dd63934d441120916c61c12b69e7f062
2026-09-08 16:24:38 +00:00
jif 2cbbf0c9b5 Add memory dual writing and v2 readiness reporting (#43827)
## Why

Allow v2 memories to build in the background while the selected memory version continues supplying context, and let clients check whether v2 has enough consolidated data and a valid summary to use.

## What changed

- Add `memories.dual_write`, disabled by default, to run v1 and v2 extraction and consolidation concurrently with separate stores and directories.
- Add the experimental `memory/status` endpoint, returning `v2ConsolidatedThreads` and `v2Ready`. Readiness requires a valid v2 summary and a consolidation thread count meeting `minConsolidatedThreads` (default 20, supported range 1–4096).
- Persist the largest thread count from a successful consolidation across pruning and clear it on memory reset.
- Share v2 summary validation between consolidation and status reporting, and tag memory metrics with `memory_version`.

## Testing

Add tests for independent dual-write outputs, preserving existing notes without importing them into v2, and readiness behavior across consolidation ownership checks, missing summaries, configurable thresholds, thread deletion, and memory reset.

GitOrigin-RevId: 54417cc60bf419062401355c0da8c79119995f61
2026-09-08 14:23:23 +00:00
jif 3f76e88a48 Add configurable memory versions with isolated storage (#43797)
## What changed

- Add `memories.version` with `v1` as the default and `v2` as an opt-in selection.
- Route memory generation, summaries, and retrieval tools through the selected version. Store `v2` artifacts in `memories_v2` and lazily create a separate SQLite database, keeping jobs and outputs isolated while sharing the thread catalog.
- Preserve the initial memory version across extension config updates so summaries and retrieval tools use the same namespace.
- Clear both versions on memory reset and remove thread memory from both stores on thread deletion. Recognize both artifact roots in shell usage telemetry.

## Testing

Add regression coverage for version defaults and validation, independent job claims and outputs, deletion and reset across versions, and shell usage classification for both memory roots.

GitOrigin-RevId: aa799bf87ab6ec10c8f5668213931e57547d080b
2026-09-08 10:55:48 +00:00
faizan-oai 3921a30d6b Persist Daybreak preferences in thread metadata (#42854)
## Why

Clients need a per-thread Daybreak toggle that survives restarts and can be
restored on resume without changing the access program of an active or future
turn.

## What changed

- Add experimental `daybreakEnabled` support to `thread/metadata/update` and
  expose the saved value on thread responses.
- Store the preference in SQLite, including for threads without a first turn,
  and preserve it through reads, resumes, reconciliation, and restarts.
- Copy the preference when forking a thread while keeping subsequent updates
  independent.
- Keep `daybreakEnabled` separate from the per-turn `cyberAccessProgram` field;
  app-server does not derive one from the other.

## Testing

- Cover persistence across restarts, cold reads, reconciliation, and forks.
- Verify metadata updates do not alter the access program of active or later
  turns.

GitOrigin-RevId: bcf987a9fcd7b0ac8a88b1c8f0df06b33ed8ad6e
2026-09-04 21:17:25 +00:00
Can Sar 728cb12fe5 Expose thread originators through the app-server API (#42458)
## What changed

- Add the creation-time `originator` to app-server thread responses and `thread/started` notifications.
- Persist originators in thread metadata and preserve the first recorded value across list, read, resume, rollout, and SQLite paths.
- Add an `originators` allowlist to `thread/list` for hosted backends. The local app-server accepts omitted or empty values but rejects nonempty filters.

## Testing

- Cover originator persistence across server restarts and list, read, and resume requests.
- Verify local `thread/list` handling for nonempty, omitted, null, and empty originator filters.
- Heap-allocate large TUI dispatcher futures in affected tests to keep them within the Windows test-thread stack.

GitOrigin-RevId: 73ad6e61b8fdd2d93a22d1f971723a19ec7be3fc
2026-09-03 06:02:25 +00:00
felixxia-oai 5971d42847 Preserve verified answers across history compaction (#42293)
## Why

Compaction replaces the model history window, but host-verified user answers may
still be needed for the rest of the owning thread.

## What changed

- Add bounded, model-invisible retained context for verified
  `request_user_input` answers.
- Persist retained context in rollout events and compaction checkpoints, and
  restore it when a thread resumes.
- Remove answers when their source turns are rolled back, and prevent
  parent-local answers from being inherited by child forks.

## Testing

Add lifecycle coverage for deduplication and size limits, compaction and resume,
rollback, rollout migration, and child forks.

GitOrigin-RevId: c32900d3e8120a9899b71f0576683acd67a4b898
2026-09-02 15:07:11 +00:00
Owen Lin 5f79a92e39 Persist response token usage in rollout history (#41912)
## Why

Resumed threads need to continue per-turn and per-thread token totals without scanning arbitrarily far beyond the latest compaction checkpoint.

## What changed

- Add durable `TokenUsageRecord` rollout items with response, turn, thread, session, and root-turn attribution.
- Restore accumulated usage on resume and snapshot the latest record plus the compaction response ID in compaction checkpoints.
- Preserve root-turn lineage in persisted turn context, while ensuring forked child threads start with their own usage totals.

## Testing

- Cover usage accumulation across multiple responses and resumed turns.
- Cover local and remote compaction checkpoints, invalid remote compaction output, rollout reconstruction, and fork isolation.

GitOrigin-RevId: ef9e0c4a9102a08a2c382be4cdac68c84353c90a
2026-08-31 21:15:38 +00:00
rhan-oai f5636bb733 Restore thread cwd from owned settings snapshots (#41567)
## Why

Resuming a thread without an explicit `cwd` should restore that thread's latest
retained setting. Forked history can contain settings copied from another
thread, while compaction can move the latest setting outside the replay window.

## What changed

- Record the owning thread ID on new settings snapshots and only use snapshots
  owned by the resumed thread when restoring `cwd`. Older snapshots without an
  owner remain readable but do not override the startup `cwd`.
- Checkpoint the current settings after compaction and serialize checkpoints
  with settings updates so the retained history contains an accepted, current
  snapshot.
- Cover resume behavior across compaction, forks, reverts, legacy histories,
  and concurrent settings updates.

GitOrigin-RevId: 1efc9cf55472d6b88c465c4efe44672a232dbf64
2026-08-29 19:57:06 +00:00
Anton Panasenko f6726e8986 Add recency sorting to project/list (#41223)
## What changed

- Include `recencyAt`, derived from the newest non-archived assigned thread, in project responses.
- Let `project/list` sort by `position` or `recencyAt` in either direction, with recency as the default descending order for that key and projects without activity placed last.
- Encode the sort key, direction, and millisecond-precision anchor in new cursors while retaining existing ascending-position cursors.
- Add an index for project recency lookups and document the API's sorting and pagination behavior.

## Testing

- Cover recency updates when threads are assigned, archived, unarchived, or deleted.
- Cover ordering, null handling, tie-breaking, cursor pagination, invalid sort combinations, and cursor mismatches.

GitOrigin-RevId: 6165e2b9976b0b45cfa16bcd42978b67811f6df5
2026-08-27 23:43:46 +00:00
Brandon Zhang 0d654e6536 Track window and fork positions in turn metadata (#40987)
## What changed

- Add the zero-based `window_number` to Responses turn metadata and include it in prewarm requests.
- Persist `forked_from_ordinal_exclusive` separately from the physical history base so fork lineage remains accurate after reverts and cold resumes.
- Report context-inheriting subagents through `parent_thread_id` without also emitting fork lineage.
- Reserve the new metadata keys while accepting and filtering previously valid configured values so Codex-owned values take precedence.

## Testing

- Cover window numbering across compaction, resume, and fork operations.
- Cover fork cutoff persistence across direct forks, legacy rollouts, reverts into inherited history, and cold resumes.

GitOrigin-RevId: 288f6cd632254814c1f166668b33828082d799a1
2026-08-26 22:37:59 +00:00
Alexi Christakis 0b94751cc4 Add telemetry for SQLite log persistence (#40726)
## Why

SQLite log persistence needs visibility into batch size, write latency,
failures, and entries dropped before they reach the writer. Exporter
diagnostics must not feed back into the SQLite log sink and keep metric
exports active indefinitely.

## What changed

- Record write count, duration, batch bytes, entry count, largest-entry size,
  and queue drops with bounded outcome tags.
- Add explicit byte histogram boundaries for log batch metrics.
- Filter OTLP transport noise and the benign unsolicited HTTP/2 PING warning
  from SQLite log persistence to prevent export cycles while retaining other
  HTTP/2 warnings.

## Testing

- Cover metric values, tags, queue-drop reasons, and histogram boundaries.
- Verify successful and failed OTLP HTTP and gRPC exports become idle instead
  of generating recurring SQLite log-write metrics.

GitOrigin-RevId: a63224ae15945fe84adc977a8b7d55bb47a4de53
2026-08-25 22:46:49 +00:00
Adam Perry @ OpenAI 1bc02aea50 Sanitize credentials from Git remote metadata (#40713)
## Why

Git remote URLs can embed usernames, passwords, or tokens. Codex carries these
URLs through turn metadata and persisted thread metadata, so credentials must be
removed before a remote enters those paths.

## What changed

- Add `SanitizedGitUrl`, which parses Git URL and SCP-style remotes, strips
  authentication data, and preserves the conventional `git` SSH user.
- Use sanitized remotes when collecting Git metadata, enriching model requests,
  discovering cloud environments, updating thread metadata, and reading legacy
  rollouts. Reject or omit malformed remotes instead of retaining their raw
  contents.
- Keep API, schema, and TypeScript representations as strings while enforcing
  sanitization in Rust.

## Testing

- Cover URL schemes, SCP and IPv6 forms, remote helpers, encoded paths,
  malformed values, and legacy deserialization.
- Verify credentials do not appear in model requests, API responses, SQLite, or
  rollout files.

GitOrigin-RevId: 6435efc4c45bfbfad4723ce7a0457cb00175f25c
2026-08-25 21:34:05 +00:00
chess 32ce703a80 Bound SQLite integrity checks for diagnostic attachments (#40688)
## Why

Diagnostic uploads skip the doctor attachment if report generation exceeds its
overall timeout. SQLite integrity scans can be delayed by large databases or
lock contention.

## What changed

- Give each database integrity scan a one-second deadline when collecting a
  diagnostic attachment.
- Interrupt scans and lock waits at the deadline, preserve any corruption rows
  already returned, and report an incomplete clean scan as a warning.
- Keep direct `codex doctor` integrity checks unbounded.

## Testing

- Cover interrupted scans, lock waits, retries without a deadline, and
  corruption results returned before interruption.

GitOrigin-RevId: 4817fb459ac53953669a9ba5f03397a6fc82c72e
2026-08-25 20:09:55 +00:00
faizan-oai 94967e03e5 Add per-turn cyber access program selection (#40687)
## What changed

- Add the experimental `turn/start.cyberAccessProgram` option with `standard`,
  `daybreakBlue`, and `daybreakRed` values.
- Forward the selection as `access_programs.cyber` on Responses, WebSocket,
  and remote-compaction requests made with ChatGPT authentication, while
  omitting it for API-key and custom-provider requests.
- Preserve the per-turn selection across recovery, compaction, and child-agent
  turns without making it a persistent thread setting.

## Testing

- Cover app-server forwarding, authentication boundaries, WebSocket reuse,
  turn recovery, compaction, and child-agent inheritance.

GitOrigin-RevId: d2eb468f365b2214c5099bc21741cf30f8bd2eb5
2026-08-25 19:54:52 +00:00
Adam Perry @ OpenAI 2f4b105514 Prevent telemetry exporters from feeding back into themselves (#40658)
## Why

Telemetry transport and storage can emit their own diagnostics, causing exports
or SQLite writes to recursively generate more telemetry.

## What changed

- Exclude `h2` spans from trace export so OTLP transport activity cannot trigger
  additional OTLP exports.
- Exclude all `sqlx`, `sqlx_core`, and `sqlx_sqlite` diagnostics from the SQLite
  log sink, including when no external subscriber filter is installed.
- Keep similarly named targets such as `sqlx_application` eligible for storage.

## Testing

- Extend the SQLite log-filter test to cover the broader SQLx exclusion and the
  unrelated-target case.

GitOrigin-RevId: 13a832e1f228e9564a31cc08f00add286342bac1
2026-08-25 17:59:32 +00:00
joeytrasatti-openai 9b2ef38f54 Add persisted thread artifact models (#40509)
## What changed

- Add a `thread_artifacts` SQLite table with per-thread typed identities,
  JSON payloads, cascade deletion, uniqueness enforcement, and an index for
  ordered per-thread reads.
- Export artifact records, attachment and removal outcomes, and paginated
  result models from `codex-state`.

## Testing

- Add a migration test covering preservation of existing thread-section
  metadata and compatibility with the previous migration set.

GitOrigin-RevId: 4e0a8e5bfa168ae2b8871970466e5055e142a4a6
2026-08-25 01:10:10 +00:00