Add experimental turn additional context (#24154)

## Summary

Adds experimental `additionalContext` support to `turn/start` and
`turn/steer` so clients can provide ephemeral external context, such as
browser or automation state, without turning that plumbing into a
visible user prompt or triggering user-prompt lifecycle behavior.

## API Shape

The parameter shape is:

```ts
additionalContext?: Record<string, {
  value: string
  kind: "untrusted" | "application"
}> | null
```

Example:

```json
{
  "additionalContext": {
    "browser_info": {
      "value": "Active tab is CI failures.",
      "kind": "untrusted"
    },
    "automation_info": {
      "value": "CI rerun is in progress.",
      "kind": "application"
    }
  }
}
```

The keys are opaque and caller-defined.

## Context Injection

When provided, accepted entries are inserted into model context as
hidden contextual message items, not as visible thread user-message
items.

`kind: "untrusted"` entries are inserted with role `user`:

```text
<external_${key}>${value}</external_${key}>
```

`kind: "application"` entries are inserted with role `developer`:

```text
<${key}>${value}</${key}>
```

Values are not escaped. Each value is truncated to 1k approximate tokens
before wrapping.

For `turn/start`, accepted additional context is inserted before normal
user input. For `turn/steer`, additional context is merged only when the
steer includes non-empty user input; context-only steers still reject as
empty input.

## Dedupe Strategy

`AdditionalContextStore` lives on session state and stores the latest
complete additional-context map.

Each `turn/start` or non-empty `turn/steer` treats its
`additionalContext` as the current complete set of values. Entries are
injected only when the key is new or the exact entry for that key
changed, including `value` or `kind`. After merging, the store is
replaced with the provided map, so omitted keys are removed from the
retained set and can be injected again later if reintroduced.

Omitting `additionalContext`, passing `null`, or passing an empty object
resets the store to empty and injects nothing.

## What Changed

- Threads experimental v2 `additionalContext` through app-server into
core turn start and steer handling.
- Adds separate contextual fragment types for untrusted user-role
context and application developer-role context.
- Uses pending response input items so additional context can be
combined with normal user input without treating it as prompt text.
- Adds integration coverage for start/steer flow, role routing,
dedupe/reset behavior, deletion/re-add behavior, hook-blocked input
behavior, empty context-only steer rejection, external-fragment marker
matching, and truncation.
This commit is contained in:
pakrym-oai authored and GitHub committed 2026-05-26 13:02:34 -07:00
1 parent cd934c8bcb
commit 768848ab6f
108 files changed
+1583 -57

No files matched your search

+23 -3
View File
@@ -1,3 +1,4 @@
use std::collections::BTreeMap;
use std::collections::HashMap;
use std::collections::HashSet;
use std::fmt::Debug;
@@ -97,6 +98,7 @@ use codex_protocol::openai_models::ModelInfo;
use codex_protocol::openai_models::ModelPreset;
use codex_protocol::permissions::FileSystemSandboxPolicy;
use codex_protocol::permissions::NetworkSandboxPolicy;
use codex_protocol::protocol::AdditionalContextEntry;
use codex_protocol::protocol::FileChange;
use codex_protocol::protocol::HasLegacyEvent;
use codex_protocol::protocol::InterAgentCommunication;
@@ -742,11 +744,17 @@ impl Codex {
pub async fn steer_input(
&self,
input: Vec<UserInput>,
additional_context: BTreeMap<String, AdditionalContextEntry>,
expected_turn_id: Option<&str>,
responsesapi_client_metadata: Option<HashMap<String, String>>,
) -> Result<String, SteerInputError> {
self.session
.steer_input(input, expected_turn_id, responsesapi_client_metadata)
.steer_input(
input,
additional_context,
expected_turn_id,
responsesapi_client_metadata,
)
.await
}
@@ -1086,6 +1094,7 @@ impl Session {
}],
final_output_json_schema: None,
responsesapi_client_metadata: None,
additional_context: Default::default(),
thread_settings: Default::default(),
},
/*mirror_user_text_to_realtime*/ None,
@@ -3152,6 +3161,7 @@ impl Session {
pub async fn steer_input(
&self,
input: Vec<UserInput>,
additional_context: BTreeMap<String, AdditionalContextEntry>,
expected_turn_id: Option<&str>,
responsesapi_client_metadata: Option<HashMap<String, String>>,
) -> Result<String, SteerInputError> {
@@ -3192,6 +3202,11 @@ impl Session {
return Err(SteerInputError::EmptyInput);
}
let additional_context_input = {
let mut state = self.state.lock().await;
state.additional_context.merge(additional_context)
};
if let Some(responsesapi_client_metadata) = responsesapi_client_metadata {
active_task
.turn_context
@@ -3199,10 +3214,15 @@ impl Session {
.set_responsesapi_client_metadata(responsesapi_client_metadata);
}
let mut pending_input = additional_context_input
.into_iter()
.map(TurnInput::ResponseInputItem)
.collect::<Vec<_>>();
pending_input.push(TurnInput::UserInput(input));
self.input_queue
.push_pending_input_and_accept_mailbox_delivery_for_turn_state(
.extend_pending_input_and_accept_mailbox_delivery_for_turn_state(
active_turn.turn_state.as_ref(),
TurnInput::UserInput(input),
pending_input,
)
.await;
Ok(active_turn_id.clone())