Commit Graph
36 Commits
Author SHA1 Message Date
Alex Hornby b877a2041e fix unified_exec::tests::unified_exec_timeouts to use a more unique match value (#9414)
Fix unified_exec_timeouts to use a unique variable value rather than
"codex" which was causing false positives when running tests locally
(presumably from my bash prompts). Discovered while running tests to
validate another change.

Fixes https://github.com/openai/codex/issues/9413

Test Plan:

Ran test locally on my fedora 43 x86_64 machine with:
```
cd codex/cargo-rs
cargo nextest run --all-features --no-fail-fast unified_exec::tests::unified_exec_timeouts
```

Before, unified_exec_timeouts fails:
```
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.38s
────────────
 Nextest run ID fa2b4949-a66c-408c-8002-32c52c70ec4f with nextest profile: default
    Starting 1 test across 107 binaries (3211 tests skipped)
        FAIL [   5.667s] codex-core unified_exec::tests::unified_exec_timeouts
  stdout ───

    running 1 test
    test unified_exec::tests::unified_exec_timeouts ... FAILED

    failures:

    failures:
        unified_exec::tests::unified_exec_timeouts

    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 774 filtered out; finished in 5.66s

  stderr ───

    thread 'unified_exec::tests::unified_exec_timeouts' (459601) panicked at core/src/unified_exec/mod.rs:381:9:
    timeout too short should yield incomplete output
    note: run with `RUST_BACKTRACE=1` environment variable to display a backtrace

────────────
     Summary [   5.677s] 1 test run: 0 passed, 1 failed, 3211 skipped
        FAIL [   5.667s] codex-core unified_exec::tests::unified_exec_timeouts
error: test run failed

```

After, works:
```
    Finished `test` profile [unoptimized + debuginfo] target(s) in 0.34s
────────────
 Nextest run ID f49e9004-e30b-4049-b0ff-283b543a1cd7 with nextest profile: default
    Starting 1 test across 107 binaries (3211 tests skipped)
        SLOW [> 15.000s] codex-core unified_exec::tests::unified_exec_timeouts
        PASS [  17.666s] codex-core unified_exec::tests::unified_exec_timeouts
────────────
     Summary [  17.676s] 1 test run: 1 passed (1 slow), 3211 skipped
```
2026-01-17 09:05:53 -08:00
jif-oai 1668ca726f chore: close pipe on non-pty processes (#9369)
Closing the STDIN of piped process when starting them to avoid commands
like `rg` to wait for content on STDIN and hangs for ever
2026-01-16 15:54:32 +01:00
jif-oai 577e1fd1b2 feat: adding piped process to replace PTY when needed (#8797) 2026-01-14 18:44:04 +00:00
jif-oai 32b1795ff4 chore: clamp min yield time for empty write_stdin (#9156)
After evals, 0 impact on performance
2026-01-14 16:25:40 +00:00
jif-oai 7532f34699 fix: drop double waiting header in TUI (#9145) 2026-01-14 09:52:34 +00:00
jif-oai 4cef89a122 chore: rename unified exec sessions (#8822)
Renaming done by Codex
2026-01-07 16:12:47 +00:00
jif-oai 32db8ea5ca feat: add head-tail buffer for unified_exec (#8735) 2026-01-06 15:48:44 +00:00
Michael Bolin 3d4ced3ff5 chore: migrate from Config::load_from_base_config_with_overrides to ConfigBuilder (#8276)
https://github.com/openai/codex/pull/8235 introduced `ConfigBuilder` and
this PR updates all call non-test call sites to use it instead of
`Config::load_from_base_config_with_overrides()`.

This is important because `load_from_base_config_with_overrides()` uses
an empty `ConfigRequirements`, which is a reasonable default for testing
so the tests are not influenced by the settings on the host. This method
is now guarded by `#[cfg(test)]` so it cannot be used by business logic.

Because `ConfigBuilder::build()` is `async`, many of the test methods
had to be migrated to be `async`, as well. On the bright side, this made
it possible to eliminate a bunch of `block_on_future()` stuff.
2025-12-18 16:12:52 -08:00
zhao-oai e0fb3ca1db refactoring with_escalated_permissions to use SandboxPermissions instead (#7750)
helpful in the future if we want more granularity for requesting
escalated permissions:
e.g when running in readonly sandbox, model can request to escalate to a
sandbox that allows writes
2025-12-10 17:18:48 +00:00
jif-oai 0ad54982ae chore: rework unified exec events (#7775) 2025-12-10 10:30:38 +00:00
pakrym-oai ac5fa6baf8 Do not emit start/end events for write stdin (#7561) 2025-12-08 15:23:02 -08:00
jif-oai 9ee855ec57 feat: add warning message for the model (#7445)
Add a warning message as a user turn to the model if the model does not
behave as expected (here, for example, if the model opens too many
`unified_exec` sessions)
2025-12-02 11:56:00 +00:00
jif-oai 457c9fdb87 chore: better session recycling (#7368) 2025-11-30 12:42:26 -08:00
jif-oai 28ff364c3a feat: update process ID for event handling (#7261) 2025-11-25 14:21:05 -08:00
jif-oai b2cddec3d7 feat: unified exec basic pruning strategy (#7239)
LRU + exited sessions first
2025-11-24 17:22:32 +00:00
Ahmed Ibrahim 3de8790714 Add the utility to truncate by tokens (#6746)
- This PR is to make it on path for truncating by tokens. This path will
be initially used by unified exec and context manager (responsible for
MCP calls mainly).
- We are exposing new config `calls_output_max_tokens`
- Use `tokens` as the main budget unit but truncate based on the model
family by Introducing `TruncationPolicy`.
- Introduce `truncate_text` as a router for truncation based on the
mode.

In next PRs:
- remove truncate_with_line_bytes_budget
- Add the ability to the model to override the token budget.
2025-11-18 11:36:23 -08:00
Ahmed Ibrahim 94dfb211af Refactor truncation helpers into its own file (#6683)
That's to centralize the truncation in one place. Next step would be to
make only two methods public: one with bytes/lines and one with tokens.
2025-11-15 06:44:23 +00:00
jif-oai 63c8c01f40 feat: better UI for unified_exec (#6515)
<img width="376" height="132" alt="Screenshot 2025-11-12 at 17 36 22"
src="https://github.com/user-attachments/assets/ce693f0d-5ca0-462e-b170-c20811dcc8d5"
/>
2025-11-14 16:31:12 +01:00
pakrym-oai 7b027e7536 Revert "Revert "Overhaul shell detection and centralize command generation for unified exec"" (#6607)
Reverts openai/codex#6606
2025-11-13 16:45:17 -08:00
pakrym-oai 0792a7953d Update default yield time (#6610)
10s for exec and 250ms for write_stdin
2025-11-13 10:24:41 -08:00
pakrym-oai e6995174c1 Revert "Overhaul shell detection and centralize command generation for unified exec" (#6606)
Reverts openai/codex#6577
2025-11-13 08:43:00 -08:00
pakrym-oaiandPavel Krymets d28e912214 Overhaul shell detection and centralize command generation for unified exec (#6577)
This fixes command display for unified exec. All `cd`s and `ls`es are
now parsed.

<img width="452" height="237" alt="image"
src="https://github.com/user-attachments/assets/ce92d81f-f74c-485a-9b34-1eaa29290ec6"
/>

Deletes a ton of tests that were doing nothing from shell.rs.

---------

Co-authored-by: Pavel Krymets <pavel@krymets.com>
2025-11-13 08:28:09 -08:00
pakrym-oai 807e2c27f0 Add unified exec escalation handling and tests (#6492)
Similar implementation to the shell tool
2025-11-11 08:19:35 -08:00
jif-oai f01f2ec9ee feat: add workdir to unified_exec (#6466) 2025-11-10 19:53:36 +00:00
jif-oai a6b9471548 feat: end events on unified exec (#5551) 2025-10-23 18:51:34 +01:00
jif-oai 00b1e130b3 chore: align unified_exec (#5442)
Align `unified_exec` with b implementation
2025-10-22 11:50:18 +01:00
pakrym-oai 789e65b9d2 Pass TurnContext around instead of sub_id (#5421)
Today `sub_id` is an ID of a single incoming Codex Op submition. We then
associate all events triggered by this operation using the same
`sub_id`.

At the same time we are also creating a TurnContext per submission and
we'd like to start associating some events (item added/item completed)
with an entire turn instead of just the operation that started it.

Using turn context when sending events give us flexibility to change
notification scheme.
2025-10-21 08:04:16 -07:00
jif-oai 5e4f3bbb0b chore: rework tools execution workflow (#5278)
Re-work the tool execution flow. Read `orchestrator.rs` to understand
the structure
2025-10-20 20:57:37 +01:00
jif-oai 3ddd4d47d0 fix: lagged output in unified_exec buffer (#4992)
Handle `Lagged` error when the broadcast buffer of the unified_exec is
full
2025-10-09 16:06:07 +00:00
Jeremy Rose 4a5f05c136 make tests pass cleanly in sandbox (#4067)
This changes the reqwest client used in tests to be sandbox-friendly,
and skips a bunch of other tests that don't work inside the
sandbox/without network.
2025-09-25 13:11:14 -07:00
jif-oai 4c97eeb32a bug: Ignore tests for now (#3777)
Ignore flaky / long tests for now
2025-09-18 10:43:45 +01:00
jimmyfraiture2 d555b68469 fix: race condition unified exec (#3644)
Fix race condition without storing an rx in the session
2025-09-15 06:52:39 -07:00
Fouad Matin 5185d69f13 fix(core): flaky test completed_commands_do_not_persist_sessions (#3596)
Fix flaky test:
```
        FAIL [   2.641s] codex-core unified_exec::tests::completed_commands_do_not_persist_sessions
  stdout ───

    running 1 test
    test unified_exec::tests::completed_commands_do_not_persist_sessions ... FAILED

    failures:

    failures:
        unified_exec::tests::completed_commands_do_not_persist_sessions

    test result: FAILED. 0 passed; 1 failed; 0 ignored; 0 measured; 235 filtered out; finished in 2.63s
    
  stderr ───

    thread 'unified_exec::tests::completed_commands_do_not_persist_sessions' panicked at core/src/unified_exec/mod.rs:582:9:
    assertion failed: result.output.contains("codex")
```
2025-09-14 18:04:05 -07:00
jif-oai ea225df22e feat: context compaction (#3446)
## Compact feature:
1. Stops the model when the context window become too large
2. Add a user turn, asking for the model to summarize
3. Build a bridge that contains all the previous user message + the
summary. Rendered from a template
4. Start sampling again from a clean conversation with only that bridge
2025-09-12 13:07:10 -07:00
jif-oai 44bb53df1e bug: default to image (#3501)
Default the MIME type to image
2025-09-11 23:10:24 +00:00
jif-oai c09ed74a16 Unified execution (#3288)
## Unified PTY-Based Exec Tool

Note: this requires to have this flag in the config:
`use_experimental_unified_exec_tool=true`

- Adds a PTY-backed interactive exec feature (“unified_exec”) with
session reuse via
  session_id, bounded output (128 KiB), and timeout clamping (≤ 60 s).
- Protocol: introduces ResponseItem::UnifiedExec { session_id,
arguments, timeout_ms }.
- Tools: exposes unified_exec as a function tool (Responses API);
excluded from Chat
  Completions payload while still supported in tool lists.
- Path handling: resolves commands via PATH (or explicit paths), with
UTF‑8/newline‑aware
  truncation (truncate_middle).
- Tests: cover command parsing, path resolution, session
persistence/cleanup, multi‑session
  isolation, timeouts, and truncation behavior.
2025-09-10 17:38:11 -07:00