Batch non-user history eviction to preserve Guardian transcript deltas (#43798)

## Why

Evicting entries on every append once history fills invalidates Guardian's transcript cursor, forcing repeated full transcripts instead of deltas.

## What changed

On non-user history overflow, evict at least the oldest half of existing non-user entries, removing more if needed to meet the byte limit. This leaves room for subsequent appends without invalidating the cursor. Preserve the separate user-message retention limits.

## Testing

Update retention assertions and add a regression test that verifies a full transcript after eviction, followed by a delta on the same Guardian thread, while retaining the user's earlier restriction.

GitOrigin-RevId: 7ba18f01e2962d3de35488d981b667ccbb8995c7
This commit is contained in:
jif authored and copyberry committed 2026-09-08 11:03:44 +00:00
1 parent 3f76e88a48
commit 5371951292
3 files changed
+121 -3

No files matched your search

+9 -1
View File
@@ -4,6 +4,8 @@
//! Hosts append original items, restore bounded checkpoints, and trim rolled-back turns.
//! Clones share immutable payloads; eviction changes the generation so readers cannot
//! reuse an offset into a different retained prefix. Prompt selection remains caller-owned.
//! Non-user overflow drops the oldest half, leaving room for appends without more evictions.
//! User messages retain as much history as their separate limits allow.
use std::collections::VecDeque;
use std::io;
@@ -47,6 +49,7 @@ impl TranscriptHistory {
}
/// Appends one original item, evicting only older entries of the same kind.
/// Non-user overflow removes at least half the existing entries; byte limits may need more.
/// Oversized user images fall back to bounded text; other oversized items are skipped.
pub fn record(&mut self, item: &ResponseItem) {
let mut size = BoundedSize {
@@ -117,9 +120,14 @@ impl TranscriptHistory {
(count + 1, bytes + size)
});
if count >= MAX_ITEMS_PER_KIND || bytes > MAX_BYTES_PER_KIND {
let retained_count = if is_user {
MAX_ITEMS_PER_KIND - 1
} else {
count / 2
};
self.items.retain(|(item, size)| {
if item.is_user_message() == is_user
&& (count >= MAX_ITEMS_PER_KIND || bytes > MAX_BYTES_PER_KIND)
&& (count > retained_count || bytes > MAX_BYTES_PER_KIND)
{
count -= 1;
bytes -= size;
@@ -49,7 +49,7 @@ fn each_kind_evicts_its_own_oldest_entries_without_reordering() {
history.items().collect::<Vec<_>>(),
users[..2]
.iter()
.chain(&tools)
.chain(&tools[63..])
.chain(&users[2..])
.collect::<Vec<_>>()
);
@@ -62,7 +62,7 @@ fn each_kind_evicts_its_own_oldest_entries_without_reordering() {
}
assert_eq!(
history.items().collect::<Vec<_>>(),
tools.iter().chain(&newer_users).collect::<Vec<_>>()
tools[63..].iter().chain(&newer_users).collect::<Vec<_>>()
);
assert!(history.generation() > generation);
}