mirror of
https://github.com/openai/codex.git
synced 2026-11-13 14:46:14 +00:00
feat(core, mcp): cache codex_apps tools in memory (#29003)
## Description This makes Codex Apps tool reads use a shared in-memory snapshot instead of rereading the disk cache every time `list_all_tools()` runs. Disk still seeds the cache on startup and gets updated after successful fetches, but it is no longer the live read path. The core change is that `McpManager` now owns a process-scoped `CodexAppsToolsCache`. Codex threads in the same app-server process now share this Codex Apps in-memory tools snapshot. The snapshot is keyed by the Codex home plus the Codex Apps identity: the active Codex auth user/workspace and the effective Codex Apps MCP source config. There's already code to hard-refresh the cache, so we respect it in this PR. ## Local benchmark I ran a local steady-state microbenchmark of the exact repeated Codex Apps cached-tools read this PR removes, using the same real local cache payload in both trees: `3,678,138` bytes and `381` tools. The cache file was already warm in the OS page cache, so this measures same-process reread/deserialization work rather than cold-disk latency or full turn latency. Each run is 25 iterations (mimicking a turn that makes 25 inference calls). | Version | Run 1 | Run 2 | Avg | |---|---:|---:|---:| | `origin/main` disk read + JSON deserialize + `filter_tools` | `50.755 ms` | `52.894 ms` | `51.825 ms` | | This branch in-memory `current_tools` + `filter_tools` | `0.740 ms` | `0.778 ms` | `0.759 ms` | That removes about `51 ms` from each repeated Codex Apps cached-tools read on this machine, roughly `68x` faster for that subpath. It is useful evidence for the hot path this PR changes, but not a claim that every production turn gets `51 ms` faster; end-to-end impact also depends on the rest of `list_all_tools()` and tool-payload construction. This is on my M2 Max macbook, so with a slower disk this would be much worse (and indeed we did see this really blew up turn runtime with a slow disk).
This commit is contained in:
1 parent
62c7f506d9
commit
703793c22e
15 files changed
+1109
-621
No files matched your search
@@ -44,7 +44,8 @@ use serde_json::Value;
|
||||
use tokio_util::sync::CancellationToken;
|
||||
|
||||
use crate::ResolvedMcpCatalog;
|
||||
use crate::codex_apps::codex_apps_tools_cache_key;
|
||||
use crate::codex_apps_cache::CodexAppsToolsCache;
|
||||
use crate::codex_apps_cache::codex_apps_tools_cache_key;
|
||||
use crate::connection_manager::McpConnectionManager;
|
||||
use crate::runtime::McpRuntimeContext;
|
||||
use crate::server::EffectiveMcpServer;
|
||||
@@ -301,6 +302,7 @@ pub async fn read_mcp_resource(
|
||||
config: &McpConfig,
|
||||
auth: Option<&CodexAuth>,
|
||||
runtime_context: McpRuntimeContext,
|
||||
codex_apps_tools_cache: CodexAppsToolsCache,
|
||||
server: &str,
|
||||
uri: &str,
|
||||
) -> anyhow::Result<ReadResourceResult> {
|
||||
@@ -329,6 +331,7 @@ pub async fn read_mcp_resource(
|
||||
PermissionProfile::default(),
|
||||
runtime_context,
|
||||
config.codex_home.clone(),
|
||||
codex_apps_tools_cache,
|
||||
codex_apps_tools_cache_key(auth),
|
||||
config.prefix_mcp_tool_names,
|
||||
config.client_elicitation_capability.clone(),
|
||||
@@ -361,6 +364,7 @@ pub async fn collect_mcp_server_status_snapshot_with_detail(
|
||||
auth: Option<&CodexAuth>,
|
||||
submit_id: String,
|
||||
runtime_context: McpRuntimeContext,
|
||||
codex_apps_tools_cache: CodexAppsToolsCache,
|
||||
detail: McpSnapshotDetail,
|
||||
) -> McpServerStatusSnapshot {
|
||||
let mcp_servers = effective_mcp_servers(config, auth);
|
||||
@@ -403,6 +407,7 @@ pub async fn collect_mcp_server_status_snapshot_with_detail(
|
||||
PermissionProfile::default(),
|
||||
runtime_context,
|
||||
config.codex_home.clone(),
|
||||
codex_apps_tools_cache,
|
||||
codex_apps_tools_cache_key(auth),
|
||||
config.prefix_mcp_tool_names,
|
||||
config.client_elicitation_capability.clone(),
|
||||
|
||||
Reference in new issue
Block a user