Optimize remote compaction history handling (#34431)

## Why

Remote compaction can process large histories. Repeatedly estimating and replacing the full history, and cloning it when tracing is disabled, adds avoidable CPU and memory overhead.

## What changed

- Estimate each history item's token count once, update the total as trailing tool outputs are rewritten, and replace history only after all rewrites are selected.
- Preserve unclamped token totals while calculating removed tokens so saturated estimates do not hide overflow.
- Snapshot compaction input history only when rollout tracing is enabled.
- Reuse the v2 request input instead of cloning it before adding the compaction trigger.

## Testing

- Cover enabled and disabled compaction trace contexts through `is_enabled()` assertions.

GitOrigin-RevId: 8de9c9704ba29532fba05430eb204bbae2e8bf83
This commit is contained in:
Charlie Marsh authored and copyberry committed 2026-07-20 23:30:30 +00:00
1 parent 56c11cf658
commit fd3c1dc13d
9 files changed
+72 -40

No files matched your search

@@ -142,6 +142,7 @@ fn disabled_thread_context_accepts_trace_calls_without_writing() -> anyhow::Resu
"gpt-test",
"test-provider",
);
assert!(!compaction_trace.is_enabled());
let compaction_attempt =
compaction_trace.start_attempt(&serde_json::json!({ "kind": "compaction" }));
compaction_attempt.record_completed(&[]);
@@ -175,6 +176,7 @@ fn compaction_contexts_share_identity_across_models() -> anyhow::Result<()> {
for model in ["gpt-previous", "gpt-selected"] {
let compaction_trace =
thread_trace.compaction_trace_context("turn-1", "compaction-1", model, "test-provider");
assert!(compaction_trace.is_enabled());
compaction_trace
.start_attempt(&serde_json::json!({ "model": model }))
.record_failed("test failure");