Evaluate recent and character-routed skill selection (#38993)

## What changed

- Add shadow selectors that fuse recently invoked skills with character routing-card matches using reciprocal rank fusion.
- Add a second variant that also incorporates weighted lexical matches.
- Record both variants in the existing shadow-selection experiment, capped at 50 candidates.

## Testing

- Cover cold starts, duplicate removal, short exact matches, deterministic ranking, result limits, and truncation metadata.
- Extend the skills integration test to verify both selectors recover a skill invoked on an earlier turn.

GitOrigin-RevId: 4ff9e91da8cb49ecbce54ab8e760820616e1a1d3
This commit is contained in:
jif authored and copyberry committed 2026-08-17 12:13:20 +00:00
1 parent 8bf50439f2
commit ea10ae7d0f
5 files changed
+348 -2

No files matched your search

+17 -1
View File
@@ -948,7 +948,13 @@ async fn shadow_lru_selector_recovers_a_skill_invoked_on_an_earlier_turn() -> Te
.find(|attribute| attribute.key.as_str() == "method")?
.value
.as_str();
if method != "lru_v1" && method != "lru_plus_lexical_v1" {
if !matches!(
method.as_ref(),
"lru_v1"
| "lru_plus_lexical_v1"
| "lru_plus_character_routing_v1"
| "lru_plus_lexical_character_routing_v1"
) {
return None;
}
let hit = point
@@ -966,6 +972,16 @@ async fn shadow_lru_selector_recovers_a_skill_invoked_on_an_earlier_turn() -> Te
assert_eq!(
vec![
(
"lru_plus_character_routing_v1".to_string(),
"true".to_string(),
2,
),
(
"lru_plus_lexical_character_routing_v1".to_string(),
"true".to_string(),
2,
),
("lru_plus_lexical_v1".to_string(), "true".to_string(), 2),
("lru_v1".to_string(), "false".to_string(), 1),
("lru_v1".to_string(), "true".to_string(), 1),