feat(core, mcp): cache codex_apps tools in memory (#29003)

## Description

This makes Codex Apps tool reads use a shared in-memory snapshot instead
of rereading the disk cache every time `list_all_tools()` runs. Disk
still seeds the cache on startup and gets updated after successful
fetches, but it is no longer the live read path.

The core change is that `McpManager` now owns a process-scoped
`CodexAppsToolsCache`. Codex threads in the same app-server process now
share this Codex Apps in-memory tools snapshot. The snapshot is keyed by
the Codex home plus the Codex Apps identity: the active Codex auth
user/workspace and the effective Codex Apps MCP source config.

There's already code to hard-refresh the cache, so we respect it in this
PR.

## Local benchmark

I ran a local steady-state microbenchmark of the exact repeated Codex
Apps cached-tools read this PR removes, using the same real local cache
payload in both trees: `3,678,138` bytes and `381` tools. The cache file
was already warm in the OS page cache, so this measures same-process
reread/deserialization work rather than cold-disk latency or full turn
latency. Each run is 25 iterations (mimicking a turn that makes 25
inference calls).

| Version | Run 1 | Run 2 | Avg |
|---|---:|---:|---:|
| `origin/main` disk read + JSON deserialize + `filter_tools` | `50.755
ms` | `52.894 ms` | `51.825 ms` |
| This branch in-memory `current_tools` + `filter_tools` | `0.740 ms` |
`0.778 ms` | `0.759 ms` |

That removes about `51 ms` from each repeated Codex Apps cached-tools
read on this machine, roughly `68x` faster for that subpath. It is
useful evidence for the hot path this PR changes, but not a claim that
every production turn gets `51 ms` faster; end-to-end impact also
depends on the rest of `list_all_tools()` and tool-payload construction.

This is on my M2 Max macbook, so with a slower disk this would be much
worse (and indeed we did see this really blew up turn runtime with a
slow disk).
This commit is contained in:
Owen Lin authored and GitHub committed 2026-06-25 20:54:48 +00:00
1 parent 62c7f506d9
commit 703793c22e
15 files changed
+1109 -621

No files matched your search

@@ -249,15 +249,12 @@ impl McpRequestProcessor {
}
None => (self.load_latest_config(/*fallback_cwd*/ None).await?, None),
};
let mcp_manager = self.thread_manager.mcp_manager();
let mcp_config = match thread {
Some(thread) => thread.runtime_mcp_config(&config).await,
None => {
self.thread_manager
.mcp_manager()
.runtime_config(&config)
.await
}
None => mcp_manager.runtime_config(&config).await,
};
let codex_apps_tools_cache = mcp_manager.codex_apps_tools_cache();
let auth = self.auth_manager.auth().await;
let environment_manager = self.thread_manager.environment_manager();
// This status path has no turn-selected environment. Use config cwd
@@ -274,6 +271,7 @@ impl McpRequestProcessor {
mcp_config,
auth,
runtime_context,
codex_apps_tools_cache,
)
.await;
});
@@ -287,6 +285,7 @@ impl McpRequestProcessor {
mcp_config: codex_mcp::McpConfig,
auth: Option<CodexAuth>,
runtime_context: McpRuntimeContext,
codex_apps_tools_cache: codex_mcp::CodexAppsToolsCache,
) {
let result = Self::list_mcp_server_status_response(
request_id.request_id.to_string(),
@@ -294,6 +293,7 @@ impl McpRequestProcessor {
mcp_config,
auth,
runtime_context,
codex_apps_tools_cache,
)
.await;
outgoing.send_result(request_id, result).await;
@@ -305,6 +305,7 @@ impl McpRequestProcessor {
mcp_config: codex_mcp::McpConfig,
auth: Option<CodexAuth>,
runtime_context: McpRuntimeContext,
codex_apps_tools_cache: codex_mcp::CodexAppsToolsCache,
) -> Result<ListMcpServerStatusResponse, JSONRPCErrorError> {
let detail = match params.detail.unwrap_or(McpServerStatusDetail::Full) {
McpServerStatusDetail::Full => McpSnapshotDetail::Full,
@@ -316,6 +317,7 @@ impl McpRequestProcessor {
auth.as_ref(),
request_id,
runtime_context,
codex_apps_tools_cache,
detail,
)
.await;
@@ -406,11 +408,9 @@ impl McpRequestProcessor {
}
let config = self.load_latest_config(/*fallback_cwd*/ None).await?;
let mcp_config = self
.thread_manager
.mcp_manager()
.runtime_config(&config)
.await;
let mcp_manager = self.thread_manager.mcp_manager();
let mcp_config = mcp_manager.runtime_config(&config).await;
let codex_apps_tools_cache = mcp_manager.codex_apps_tools_cache();
let auth = self.auth_manager.auth().await;
let environment_manager = self.thread_manager.environment_manager();
// This threadless resource-read path has no turn cwd or turn-selected
@@ -425,6 +425,7 @@ impl McpRequestProcessor {
&mcp_config,
auth.as_ref(),
runtime_context,
codex_apps_tools_cache,
&server,
&uri,
)