Commit Graph
13 Commits
Author SHA1 Message Date
jif a04940cb12 Reject symbolic links in memory workspaces (#39205)
## What changed

- Reject a symbolic link used as the memory root.
- Recursively remove symbolic links before preparing or seeding the memory workspace.
- Remove and reject symbolic links created during consolidation, including when the worker fails, so they cannot affect files outside the workspace.

## Testing

Add Unix startup tests covering pre-existing extension links and links created by both successful and failed consolidation workers.

GitOrigin-RevId: f96e03812773f7beaa16455dd77370ee47d8a71f
2026-08-18 14:14:57 +00:00
jif 5ad367fb95 Wait for memory consolidation shutdown before finishing jobs (#36140)
## Why

A phase-two memory job could release its lease while its consolidation agent was
still shutting down, allowing another worker to race the existing agent.

## What changed

- Await consolidation-agent shutdown before validating artifacts and completing
  the job.
- Keep the lease until it expires if shutdown fails, and remove the agent from
  the thread manager only after shutdown succeeds.

## Testing

Update startup tests to wait for the phase-two job lifecycle to finish and verify
that the consolidation thread has been removed.

GitOrigin-RevId: 7ac3a7369529822ac95fd2ee9dcb27d39c191d20
2026-07-30 11:52:38 +00:00
jif 687f05cb94 Remove CSV-backed agent jobs (#34413)
## What changed

- Remove the `spawn_agents_on_csv` and `report_agent_job_result` tools and their agent-job runtime and state models.
- Drop the legacy `agent_jobs` and `agent_job_items` tables during state database migration.
- Keep `features.enable_fanout` and `agents.job_max_runtime_seconds` accepted as no-op compatibility settings while omitting them from the generated configuration schema.

## Testing

- Verify upgrades remove both legacy agent-job tables.
- Verify the removed feature and configuration keys still parse without taking effect.

GitOrigin-RevId: 8cc3337da78c67162229f02f40a747f503542646
2026-07-20 21:00:17 +00:00
Eric Traut 2edad72de3 Track prompt cache write token usage (#33454)
## What changed

- Parse `cache_write_tokens` from response input token details and carry it through token usage aggregation.
- Expose `cache_write_input_tokens` in protocol, app-server, exec, and TypeScript SDK usage events, defaulting it to zero for compatibility with older payloads.
- Report cache-write usage in analytics, metrics, traces, and rollout data alongside existing cache-read usage.

## Testing

- Add coverage for response parsing, usage event output, aggregation, analytics, metrics, and telemetry fields.

GitOrigin-RevId: 634e4fbe3086a8923c0ee13b622573e7d9d221bd
2026-07-16 00:22:53 +00:00
jif 54b8f112a3 Preserve parent sandbox enforcement for memory consolidation (#32441)
## What changed

- Pass the parent turn's effective permission profile to the memory consolidation agent, including thread-level permission and legacy sandbox overrides.
- Preserve disabled and externally enforced permission profiles instead of replacing them with a managed sandbox.
- Continue restricting consolidation to the memory root without network access when the parent uses Codex-managed permissions.

## Testing

- Add coverage for disabled, external, and managed parent permission profiles.

GitOrigin-RevId: 9ca3be0e41dc14d858053f6f70e33f4ae7c578e1
2026-07-11 15:20:49 +00:00
jif 601b61a83f Rebind memory consolidation workspace roots (#32197)
## Why

The memory consolidation agent changes its working directory to the memory
root, but applying its sandbox policy directly to the permissions object can
leave workspace roots inherited from the parent configuration.

## What changed

Apply the consolidation sandbox policy through `Config` so its workspace roots
are synchronized with the memory root. Add a test covering the working
directory, workspace roots, and effective legacy sandbox policy.

GitOrigin-RevId: dc6e75aef1ffa84a2f66fec6ed3a6fedcf9aa672
2026-07-10 15:44:56 +00:00
jif 52c9605dc7 Validate memory consolidation artifacts before succeeding (#32193)
## Why

A completed Phase 2 agent run does not guarantee that its required outputs were
created correctly. Treating a clean workspace as success can also preserve a
state where those outputs are missing.

## What changed

- Require `MEMORY.md` to be a file and `memory_summary.md` to start with `v1`
  before marking consolidation successful.
- Fail completed runs with invalid artifacts without resetting the workspace
  baseline, allowing the job to be retried.
- Run consolidation for a clean workspace when its required artifacts are
  invalid instead of taking the no-change success path.

## Testing

Added coverage for rejecting an invalid summary and retrying a clean workspace
whose consolidation artifacts are missing.

GitOrigin-RevId: ac57b2ba9d062c5203ca51afd716795a91814bb3
2026-07-10 15:40:35 +00:00
Celia Chen 51fc4b0559 feat: use provider defaults for memory models (#27129)
## Why

Memory startup used hardcoded OpenAI model slugs for extraction and
consolidation. That works for the default OpenAI-compatible path, but
provider-specific backends can require different model identifiers. In
particular, Amazon Bedrock should use its Bedrock model ID for these
background memory requests instead of the OpenAI `gpt-5.4-mini` /
`gpt-5.4` slugs.

## What Changed

- Added provider-owned preferred memory model methods alongside
`approval_review_preferred_model`.
- Updated memory extraction and consolidation to resolve their default
model through the active `ModelProvider`.
- Added Amazon Bedrock overrides so both memory stages use
`openai.gpt-5.4` through Bedrock’s provider-specific model ID.
- Kept explicit `memories.extract_model` and
`memories.consolidation_model` config overrides taking precedence.
- Added startup coverage for default OpenAI and Bedrock memory model
selection.

#closes #26288
2026-06-09 23:49:09 +00:00
jif-oai aad59a0916 Move memory state to a dedicated SQLite DB (#24591)
## Summary

Generated memory rows and their stage-one/stage-two job state currently
live in `state_5.sqlite` alongside thread metadata. That makes memory
cleanup and regeneration share the main state schema even though those
rows are memory-pipeline data and can be rebuilt independently from the
durable thread records.

This PR moves the memory-owned tables into a dedicated
`memories_1.sqlite` runtime database while keeping thread metadata in
`state_5.sqlite`.

## Changes

- Adds a separate memories DB runtime, migrator, path helpers, telemetry
kind, and Bazel compile data for `state/memory_migrations`.
- Introduces `MemoryStore` behind `StateRuntime::memories()` and moves
memory table/job operations onto that store.
- Drops the old memory tables from the state DB and recreates their
schema in `state/memory_migrations/0001_memories.sql`.
- Updates memory startup, citation usage tracking, rollout pollution
handling, `debug clear-memories`, and app-server `memory/reset` to
operate through the memories DB.
- Preserves cross-DB behavior by hydrating thread metadata from the
state DB when selecting visible memory outputs and checking stage-one
staleness.

## Verification

- Added/updated `codex-state` tests for deleted-thread memory visibility
and already-polluted phase-two enqueue behavior.
- Updated `debug clear-memories`, app-server `memory/reset`, and
memories startup tests to seed and assert memory rows through
`memories_1.sqlite`.
2026-05-26 20:07:25 +02:00
jif-oai 5b7d6f5c4f feat: house-keeping memories 3 (#20005)
Move stuff in memories, no behavioural change expected
2026-04-28 18:13:35 +02:00
jif-oai 21e19912e0 feat: house-keeping memories 2 (#20000)
Just move metrics in a dedicated file
2026-04-28 17:26:44 +02:00
jif-oaiandCodex a9e5c34083 feat: trigger memories from user turns with cooldown (#19970)
## Why

Memory startup was tied to thread lifecycle events such as create, load,
and fork. That can run memory work before a thread receives real user
input, and it makes startup cost scale with thread management instead of
actual turns. Moving the trigger to `thread/sendInput` keeps memory
startup aligned with the first real user turn and lets it use the
current thread config at turn time.

The idea is to prevent ghost cost due to pre-warm triggered by the app

Turn-based startup can also make global phase-2 consolidation easier to
request repeatedly, so this adds a success cooldown and tightens the
default startup scan window.

## What Changed

- Start `codex_memories_write::start_memories_startup_task` after a
non-empty `thread/sendInput` turn is submitted, instead of from thread
create/load/fork paths:
https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/app-server/src/codex_message_processor.rs#L6477-L6487
- Expose `CodexThread::config()` so app-server can pass the live config
into memory startup at turn time.
- Add a six-hour successful-run cooldown for global phase-2
consolidation via `SkippedCooldown`:
https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/state/src/runtime/memories.rs#L963-L966
- Reduce memory startup defaults to at most 2 rollouts over 10 days:
https://github.com/openai/codex/blob/d4a6885b7829e2fd2ec7a09355e4f75ebe1d1fe3/codex-rs/config/src/types.rs#L31-L34

## Verification

Updated the memory runtime coverage around phase-2 reclaim behavior,
including `phase2_global_lock_respects_success_cooldown`.

---------

Co-authored-by: Codex <noreply@openai.com>
2026-04-28 16:23:13 +02:00
jif-oaiandCodex 431ebeaef7 feat: split memories part 2 (#19860)
Keep extracting memories out of core and moving the write trigger in the
app-server
This is temporary and it should move at the client level as a follow-up
This makes core fully independant from `codex-memories-write`

---------

Co-authored-by: Codex <noreply@openai.com>
2026-04-28 13:03:28 +02:00