Project selected plugin runtime by environment availability (#30093)

## Why

Selected plugin metadata is stable, but MCP processes are live runtime
state. They need different lifetimes:

- the MCP extension caches manifest, MCP, and connector declarations for
each stable selected root;
- each model step projects that cached metadata through the roots that
resolved as ready for that exact step;
- the MCP manager is rebuilt only when that availability projection
changes.

This matches executor skills: both features consume the same resolved
step roots instead of inferring readiness from the turn's selected
environments.

## Behavior

```text
E1 not ready for this step
  -> no E1 MCP servers or connectors
  -> cached plugin metadata stays in ext/mcp

E1 becomes ready
  -> reuse cached metadata
  -> publish one MCP runtime containing E1 capabilities

same ready roots on the next step
  -> reuse the exact runtime; no rediscovery and no MCP restart

resume
  -> create new extension thread state and a new MCP runtime
```

All model-facing consumers use the same step snapshot:

```text
resolved selected roots
        |
        v
extension MCP/connector projection
        |
        v
{ MCP config, connector snapshot, MCP manager }
        |
        +-> advertise model tools
        +-> build app/connector tools
        +-> execute MCP calls
```

## Cache contract

The existing MCP extension owns a cache keyed by the full
`SelectedCapabilityRoot`:

```rust
let state = thread_store.get_or_init(SelectedExecutorPluginMcpState::default);
```

The cache lives with extension thread state. Environment availability
filters projection but does not invalidate metadata. Resume creates new
thread state. There is no file watcher or executor generation because
contents behind a stable environment/root are assumed stable.

## What changes

- Keeps executor plugin discovery and cached metadata in `ext/mcp`.
- Caches MCP and connector declarations together per selected root.
- Uses the step's already-resolved capability roots, including lazy
environments that are not turn environments.
- Reuses the current MCP runtime when the ready-root projection is
unchanged.
- Uses the same step MCP manager and connector snapshot for
model-visible tools and execution.
- Resolves direct thread-scoped MCP requests from the current
selected-root projection.

## Deliberately out of scope

- `app/list` remains based on the latest global host-plugin state; this
PR does not make its response or notifications thread-specific.
- `required = true` startup semantics do not apply to delayed executor
MCP activation.
- No filesystem/content invalidation.
- No transport-disconnect watcher.
- No executor generations or environment replacement semantics.
- No client sharing across complete manager replacements.

## Stack

1. Extension-owned World State sections.
2. Project executor skills through World State.
3. Pin one MCP runtime to each model step.
4. **This PR:** project selected MCP and connector state from
extension-owned metadata.
5. Integration coverage for selected capability availability and resume.

## Verification

-
`selected_plugin_servers_use_managed_requirements_for_the_selected_root_id`
- The stacked integration PR covers unavailable to ready activation,
unchanged-runtime reuse, skills, MCP tools, connector attribution, and
cold resume.
This commit is contained in:
jif authored and GitHub committed 2026-06-26 01:36:44 +01:00
1 parent 5044062704
commit 3095ea9c3d
21 files changed
+455 -163

No files matched your search

+153 -43
View File
@@ -1,7 +1,9 @@
use super::*;
use codex_exec_server::ResolvedSelectedCapabilityRoot;
use codex_mcp::ElicitationReviewRequest;
use codex_mcp::ElicitationReviewer;
use codex_mcp::ElicitationReviewerHandle;
use codex_protocol::capabilities::CapabilityRootLocation;
use codex_protocol::config_types::ApprovalsReviewer;
use codex_protocol::mcp_approval_meta::APPROVAL_KIND_KEY as MCP_ELICITATION_APPROVAL_KIND_KEY;
use codex_protocol::mcp_approval_meta::APPROVAL_KIND_MCP_TOOL_CALL as MCP_ELICITATION_APPROVAL_KIND_MCP_TOOL_CALL;
@@ -75,9 +77,20 @@ impl ElicitationReviewer for GuardianMcpElicitationReviewer {
impl Session {
pub(crate) async fn runtime_mcp_config(&self, config: &Config) -> McpConfig {
let environments = self.services.turn_environments.snapshot().await;
let selected_capability_roots = self
.resolve_selected_capability_roots_for_step(&environments)
.await;
let available_environment_ids =
Self::available_selected_environment_ids(&selected_capability_roots);
self.services
.mcp_manager
.runtime_config_for_thread(config, &self.services.mcp_thread_init)
.runtime_config_for_step(
config,
&self.services.mcp_thread_init,
&self.services.thread_extension_data,
&available_environment_ids,
)
.await
}
@@ -88,6 +101,62 @@ impl Session {
codex_mcp::configured_mcp_servers(&self.runtime_mcp_config(config).await)
}
#[expect(
clippy::await_holding_invalid_type,
reason = "MCP runtime comparison and publication must remain serialized"
)]
pub(crate) async fn mcp_runtime_for_step(
self: &Arc<Self>,
turn_context: &TurnContext,
environments: &TurnEnvironmentSnapshot,
selected_capability_roots: &[ResolvedSelectedCapabilityRoot],
) -> Arc<McpRuntimeSnapshot> {
let available_environment_ids =
Self::available_selected_environment_ids(selected_capability_roots);
let current = self.services.latest_mcp_runtime();
if current.available_environment_ids() == available_environment_ids {
return current;
}
let _guard = self.services.mcp_projection_lock.lock().await;
let current = self.services.latest_mcp_runtime();
if current.available_environment_ids() == available_environment_ids {
return current;
}
let mcp_config = self
.services
.mcp_manager
.runtime_config_for_step(
&turn_context.config,
&self.services.mcp_thread_init,
&self.services.thread_extension_data,
&available_environment_ids,
)
.await;
self.refresh_mcp_servers_inner(
turn_context,
mcp_config,
environments,
&available_environment_ids,
Some(self.mcp_elicitation_reviewer()),
)
.await
}
pub(crate) async fn resolve_selected_capability_roots_for_step(
&self,
environments: &TurnEnvironmentSnapshot,
) -> Vec<ResolvedSelectedCapabilityRoot> {
self.services
.turn_environments
.environment_manager()
.resolve_selected_capability_roots(
&self.services.selected_capability_roots,
&environments.captured_environments(),
)
.await
}
pub(crate) fn mcp_elicitation_reviewer(self: &Arc<Self>) -> ElicitationReviewerHandle {
Arc::new(GuardianMcpElicitationReviewer::new(self))
}
@@ -207,51 +276,22 @@ impl Session {
.await
}
pub async fn read_resource(
&self,
server: &str,
params: ReadResourceRequestParams,
) -> anyhow::Result<ReadResourceResult> {
self.services
.latest_mcp_runtime()
.manager_arc()
.read_resource(server, params)
.await
}
pub async fn call_tool(
&self,
server: &str,
tool: &str,
arguments: Option<serde_json::Value>,
meta: Option<serde_json::Value>,
) -> anyhow::Result<CallToolResult> {
self.services
.latest_mcp_runtime()
.manager_arc()
.call_tool(server, tool, arguments, meta)
.await
}
async fn refresh_mcp_servers_inner(
&self,
turn_context: &TurnContext,
mut mcp_config: McpConfig,
configured_mcp_servers: HashMap<String, McpServerConfig>,
mcp_config: McpConfig,
environments: &TurnEnvironmentSnapshot,
available_environment_ids: &[String],
elicitation_reviewer: Option<ElicitationReviewerHandle>,
) {
mcp_config.mcp_server_catalog = mcp_config
.mcp_server_catalog
.with_materialized_servers(configured_mcp_servers);
let mcp_config = Arc::new(mcp_config);
) -> Arc<McpRuntimeSnapshot> {
let auth = self.services.auth_manager.auth().await;
let mcp_config = Arc::new(mcp_config);
let tool_plugin_provenance = codex_mcp::tool_plugin_provenance(&mcp_config);
let mcp_servers = effective_mcp_servers(&mcp_config, auth.as_ref());
let environment_manager = self.services.turn_environments.environment_manager();
// TODO(anp): Migrate MCP runtime cwd plumbing to PathUri so foreign environment cwd
// values can be used without falling back to the legacy host cwd.
let cwd = turn_context
.environments
let cwd = environments
.primary()
.and_then(|turn_environment| turn_environment.cwd().to_abs_path().ok())
.map(|cwd| cwd.to_path_buf())
@@ -305,10 +345,18 @@ impl Session {
refreshed_manager
.set_elicitations_auto_deny(current_manager.manager().elicitations_auto_deny());
}
self.services
.publish_mcp_runtime(mcp_config, mcp_runtime_context, refreshed_manager);
self.services.publish_mcp_runtime(
mcp_config,
mcp_runtime_context,
available_environment_ids.to_vec(),
refreshed_manager,
)
}
#[expect(
clippy::await_holding_invalid_type,
reason = "MCP runtime refresh and publication must remain serialized"
)]
pub(crate) async fn refresh_mcp_servers_if_requested(
&self,
turn_context: &TurnContext,
@@ -365,9 +413,33 @@ impl Session {
return;
}
let mcp_config = self.runtime_mcp_config(&refresh_config).await;
self.refresh_mcp_servers_inner(turn_context, mcp_config, mcp_servers, elicitation_reviewer)
let _guard = self.services.mcp_projection_lock.lock().await;
let available_environment_ids = self
.services
.latest_mcp_runtime()
.available_environment_ids()
.to_vec();
let mut mcp_config = self
.services
.mcp_manager
.runtime_config_for_step(
&refresh_config,
&self.services.mcp_thread_init,
&self.services.thread_extension_data,
&available_environment_ids,
)
.await;
mcp_config.mcp_server_catalog = mcp_config
.mcp_server_catalog
.with_materialized_servers(mcp_servers);
self.refresh_mcp_servers_inner(
turn_context,
mcp_config,
&turn_context.environments,
&available_environment_ids,
elicitation_reviewer,
)
.await;
}
pub(crate) async fn set_openai_form_elicitation_support(
@@ -398,16 +470,54 @@ impl Session {
Ok(())
}
#[expect(
clippy::await_holding_invalid_type,
reason = "MCP runtime refresh and publication must remain serialized"
)]
pub(crate) async fn refresh_mcp_servers_now(
&self,
turn_context: &TurnContext,
refresh_config: &Config,
elicitation_reviewer: Option<ElicitationReviewerHandle>,
) {
let mcp_config = self.runtime_mcp_config(refresh_config).await;
let mcp_servers = codex_mcp::configured_mcp_servers(&mcp_config);
self.refresh_mcp_servers_inner(turn_context, mcp_config, mcp_servers, elicitation_reviewer)
let _guard = self.services.mcp_projection_lock.lock().await;
let available_environment_ids = self
.services
.latest_mcp_runtime()
.available_environment_ids()
.to_vec();
let mcp_config = self
.services
.mcp_manager
.runtime_config_for_step(
refresh_config,
&self.services.mcp_thread_init,
&self.services.thread_extension_data,
&available_environment_ids,
)
.await;
self.refresh_mcp_servers_inner(
turn_context,
mcp_config,
&turn_context.environments,
&available_environment_ids,
elicitation_reviewer,
)
.await;
}
fn available_selected_environment_ids(
selected_capability_roots: &[ResolvedSelectedCapabilityRoot],
) -> Vec<String> {
let mut available = Vec::new();
for root in selected_capability_roots {
let CapabilityRootLocation::Environment { environment_id, .. } =
&root.selected_root().location;
if !available.contains(environment_id) {
available.push(environment_id.clone());
}
}
available
}
#[cfg(test)]
+8
View File
@@ -10,6 +10,7 @@ pub struct McpRuntimeSnapshot {
config: Arc<McpConfig>,
manager: Arc<McpConnectionManager>,
runtime_context: McpRuntimeContext,
available_environment_ids: Vec<String>,
}
impl McpRuntimeSnapshot {
@@ -17,11 +18,13 @@ impl McpRuntimeSnapshot {
config: Arc<McpConfig>,
manager: Arc<McpConnectionManager>,
runtime_context: McpRuntimeContext,
available_environment_ids: Vec<String>,
) -> Self {
Self {
config,
manager,
runtime_context,
available_environment_ids,
}
}
@@ -41,6 +44,10 @@ impl McpRuntimeSnapshot {
&self.runtime_context
}
pub(crate) fn available_environment_ids(&self) -> &[String] {
&self.available_environment_ids
}
#[cfg(test)]
pub(crate) fn new_uninitialized_for_test(config: &crate::config::Config) -> Arc<Self> {
use codex_exec_server::EnvironmentManager;
@@ -81,6 +88,7 @@ impl McpRuntimeSnapshot {
Arc::new(mcp_config),
Arc::new(manager),
runtime_context,
Vec::new(),
))
}
}
+8 -11
View File
@@ -100,7 +100,6 @@ use codex_protocol::dynamic_tools::DynamicToolResponse;
use codex_protocol::dynamic_tools::DynamicToolSpec;
use codex_protocol::items::TurnItem;
use codex_protocol::items::UserMessageItem;
use codex_protocol::mcp::CallToolResult;
use codex_protocol::models::ActivePermissionProfile;
use codex_protocol::models::AdditionalPermissionProfile;
use codex_protocol::models::BaseInstructions;
@@ -158,8 +157,6 @@ use codex_utils_path_uri::PathUri;
use futures::future::BoxFuture;
use futures::future::Shared;
use futures::prelude::*;
use rmcp::model::ReadResourceRequestParams;
use rmcp::model::ReadResourceResult;
use rmcp::model::RequestId;
use serde_json::Value;
use tokio::sync::Mutex;
@@ -2825,7 +2822,7 @@ impl Session {
/// `run_turn` and pass the result down; standalone request or history boundaries may capture
/// their own step.
pub(crate) async fn capture_step_context(
&self,
self: &Arc<Self>,
turn_context: Arc<TurnContext>,
) -> Arc<StepContext> {
let deferred_executor_enabled = turn_context
@@ -2846,15 +2843,15 @@ impl Session {
}
let loaded_agents_md = self.services.agents_md_manager.get_loaded().await;
let selected_capability_roots = self
.services
.turn_environments
.environment_manager()
.resolve_selected_capability_roots(
&self.services.selected_capability_roots,
&environments.captured_environments(),
.resolve_selected_capability_roots_for_step(&environments)
.await;
let mcp = self
.mcp_runtime_for_step(
turn_context.as_ref(),
&environments,
&selected_capability_roots,
)
.await;
let mcp = self.services.latest_mcp_runtime();
Arc::new(StepContext::new(
turn_context,
environments,
+9 -1
View File
@@ -661,6 +661,7 @@ impl Session {
let config_for_mcp = Arc::clone(&config);
let mcp_manager_for_mcp = Arc::clone(&mcp_manager);
let mcp_thread_init_for_startup = &mcp_thread_init;
let thread_extension_data_for_mcp = &thread_extension_data;
let mcp_runtime_cwd = session_configuration
.environment_selections()
.first()
@@ -673,7 +674,12 @@ impl Session {
let auth_and_mcp_fut = async move {
let auth = auth_manager_clone.auth().await;
let mcp_config = mcp_manager_for_mcp
.runtime_config_for_thread(&config_for_mcp, mcp_thread_init_for_startup)
.runtime_config_for_step(
&config_for_mcp,
mcp_thread_init_for_startup,
thread_extension_data_for_mcp,
/*available_environment_ids*/ &[],
)
.await;
let mcp_servers = codex_mcp::effective_mcp_servers(&mcp_config, auth.as_ref());
let tool_plugin_provenance = codex_mcp::tool_plugin_provenance(&mcp_config);
@@ -1039,6 +1045,7 @@ impl Session {
// setup is straightforward enough and performs well.
mcp_connection_manager,
mcp_runtime: arc_swap::ArcSwapOption::empty(),
mcp_projection_lock: Mutex::new(()),
mcp_startup_cancellation_token: Mutex::new(CancellationToken::new()),
unified_exec_manager: UnifiedExecProcessManager::new(
config.background_terminal_max_timeout,
@@ -1210,6 +1217,7 @@ impl Session {
.install_mcp_connection_manager(
Arc::new(mcp_config),
mcp_runtime_context,
/*available_environment_ids*/ Vec::new(),
mcp_connection_manager,
)
.await?;
+3
View File
@@ -5372,6 +5372,7 @@ pub(crate) async fn make_session_and_context() -> (Session, TurnContext) {
let services = SessionServices {
mcp_connection_manager: Arc::new(arc_swap::ArcSwap::from(mcp_runtime.manager_arc())),
mcp_runtime: arc_swap::ArcSwapOption::from(Some(mcp_runtime)),
mcp_projection_lock: Mutex::new(()),
mcp_startup_cancellation_token: Mutex::new(CancellationToken::new()),
unified_exec_manager: UnifiedExecProcessManager::new(
config.background_terminal_max_timeout,
@@ -7446,6 +7447,7 @@ where
let services = SessionServices {
mcp_connection_manager: Arc::new(arc_swap::ArcSwap::from(mcp_runtime.manager_arc())),
mcp_runtime: arc_swap::ArcSwapOption::from(Some(mcp_runtime)),
mcp_projection_lock: Mutex::new(()),
mcp_startup_cancellation_token: Mutex::new(CancellationToken::new()),
unified_exec_manager: UnifiedExecProcessManager::new(
config.background_terminal_max_timeout,
@@ -7603,6 +7605,7 @@ pub(crate) async fn make_session_and_context_with_rx() -> (
#[tokio::test]
async fn refresh_mcp_servers_keeps_the_previous_runtime_alive() {
let (session, turn_context) = make_session_and_context().await;
let session = Arc::new(session);
let turn_context = Arc::new(turn_context);
let old_runtime = session.services.latest_mcp_runtime();
let step_context = session
+14 -13
View File
@@ -171,8 +171,13 @@ pub(crate) async fn run_turn(
.record_context_updates_and_set_reference_context_item(first_step_context.as_ref())
.await;
let Some((injection_items, explicitly_enabled_connectors)) =
build_skills_and_plugins(&sess, turn_context.as_ref(), &input, &cancellation_token).await
let Some((injection_items, explicitly_enabled_connectors)) = build_skills_and_plugins(
&sess,
first_step_context.as_ref(),
&input,
&cancellation_token,
)
.await
else {
return Ok(None);
};
@@ -504,10 +509,11 @@ async fn run_hooks_and_record_inputs(
#[instrument(level = "trace", skip_all)]
async fn build_skills_and_plugins(
sess: &Arc<Session>,
turn_context: &TurnContext,
step_context: &StepContext,
input: &[TurnInput],
cancellation_token: &CancellationToken,
) -> Option<(Vec<ResponseItem>, HashSet<String>)> {
let turn_context = step_context.turn.as_ref();
// Guardian input embeds the parent transcript as untrusted evidence. Do not interpret skill or
// plugin mentions from that generated prompt as requests to inject additional instructions.
if crate::guardian::is_guardian_reviewer_source(&turn_context.session_source) {
@@ -538,17 +544,14 @@ async fn build_skills_and_plugins(
// enabled plugins, then converted into turn-scoped guidance below.
let mentioned_plugins =
collect_explicit_plugin_mentions(&user_input, loaded_plugins.capability_summaries());
let connector_snapshot = codex_connectors::ConnectorSnapshot::from_plugin_capability_summaries(
loaded_plugins.capability_summaries(),
);
let connector_snapshot = step_context.mcp.config().connector_snapshot.clone();
let mcp_tools = if turn_context.apps_enabled() || !mentioned_plugins.is_empty() {
// Plugin mentions need raw MCP/app inventory even when app tools
// are normally hidden so we can describe the plugin's currently
// usable capabilities for this turn.
match sess
.services
.mcp_connection_manager
.load_full()
match step_context
.mcp
.manager_arc()
.list_all_tools()
.or_cancel(cancellation_token)
.await
@@ -1189,9 +1192,7 @@ pub(crate) async fn built_tools(
.plugins_for_config(&turn_context.config.plugins_config_input())
.instrument(trace_span!("built_tools.load_plugins"))
.await;
let connector_snapshot = codex_connectors::ConnectorSnapshot::from_plugin_capability_summaries(
loaded_plugins.capability_summaries(),
);
let connector_snapshot = step_context.mcp.config().connector_snapshot.clone();
let apps_enabled = turn_context.apps_enabled();
let accessible_connectors =