mirror of
https://github.com/openai/codex.git
synced 2026-11-18 23:02:23 +00:00
core: cache the tool search handler per session (#27258)
## Why Tool router construction rebuilds the deferred-tool BM25 index during session initialization and before each sampling continuation, even when the searchable tool metadata is unchanged. Local profiling measured `append_tool_search_executor` at roughly 113 ms per continuation, making repeated index construction the largest measured router-building cost. ## What changed - Add a session-scoped `ToolSearchHandlerCache` so continuations and user turns can reuse the existing handler. - Key reuse on the complete ordered `Vec<ToolSearchInfo>`, rebuilding when searchable text, loadable tool specs, source metadata, or ordering changes. - Build handlers outside the cache lock and recheck before publishing them, avoiding holding the mutex during index construction. ## Verification - `cache_reuses_identical_search_infos_and_rebuilds_changed_inputs` covers exact cache reuse and invalidation when the ordered search metadata changes. - Local rollout profiling showed the initial router build populating the cache and unchanged later continuations reusing it: - uncached: 118 ms median across 14 spans from 3 rollouts - cached: 4 ms median across 12 spans from 3 rollouts
This commit is contained in:
1 parent
fbbe7706d6
commit
af99f6a72f
12 files changed
+169
-29
No files matched your search
@@ -4,6 +4,7 @@ use crate::session::turn_context::TurnContext;
|
||||
use crate::tools::context::SharedTurnDiffTracker;
|
||||
use crate::tools::context::ToolInvocation;
|
||||
use crate::tools::context::ToolPayload;
|
||||
use crate::tools::handlers::ToolSearchHandlerCache;
|
||||
use crate::tools::registry::AnyToolResult;
|
||||
use crate::tools::registry::ToolArgumentDiffConsumer;
|
||||
use crate::tools::registry::ToolRegistry;
|
||||
@@ -45,8 +46,12 @@ pub(crate) struct ToolRouterParams<'a> {
|
||||
}
|
||||
|
||||
impl ToolRouter {
|
||||
pub fn from_turn_context(turn_context: &TurnContext, params: ToolRouterParams<'_>) -> Self {
|
||||
build_tool_router(turn_context, params)
|
||||
pub(crate) fn from_turn_context(
|
||||
turn_context: &TurnContext,
|
||||
params: ToolRouterParams<'_>,
|
||||
tool_search_handler_cache: &ToolSearchHandlerCache,
|
||||
) -> Self {
|
||||
build_tool_router(turn_context, params, tool_search_handler_cache)
|
||||
}
|
||||
|
||||
pub(crate) fn from_parts(registry: ToolRegistry, model_visible_specs: Vec<ToolSpec>) -> Self {
|
||||
|
||||
Reference in new issue
Block a user