Share model-visible tool specs across prompts (#37807)

## Why

Building a prompt cloned every model-visible `ToolSpec`, even though the tool
set is immutable for the lifetime of its router.

## What changed

- Store model-visible tool specs as an `Arc<[ToolSpec]>` in `ToolRouter` and
  `Prompt` so prompt construction only clones the shared pointer.
- Keep separately built routers on distinct shared slices so refreshed tool
  sets remain independent.

## Testing

Extend router tests to verify allocation sharing within a router, allocation
separation across rebuilt routers, and the existing deferred-tool filtering.

GitOrigin-RevId: 1e04cd7c4d1b3fa0b494c5c51670232d38ba8ebb
This commit is contained in:
Charlie Marsh authored and copyberry committed 2026-08-10 10:50:49 +00:00
1 parent c8e6e8555c
commit beeba1d2fc
4 files changed
+16 -8

No files matched your search

+3 -2
View File
@@ -8,6 +8,7 @@ use codex_tools::ToolSpec;
use futures::Stream;
use serde_json::Value;
use std::pin::Pin;
use std::sync::Arc;
use std::task::Context;
use std::task::Poll;
use tokio::sync::mpsc;
@@ -21,7 +22,7 @@ pub struct Prompt {
/// Tools available to the model, including additional tools sourced from
/// external MCP servers.
pub(crate) tools: Vec<ToolSpec>,
pub(crate) tools: Arc<[ToolSpec]>,
/// Whether parallel tool calls are permitted for this prompt.
pub(crate) parallel_tool_calls: bool,
@@ -39,7 +40,7 @@ impl Default for Prompt {
fn default() -> Self {
Self {
input: Vec::new(),
tools: Vec::new(),
tools: Arc::default(),
parallel_tool_calls: false,
base_instructions: BaseInstructions::default(),
output_schema: None,