Recover capability discovery after executor disconnects (#38420)

## Why

Transient executor disconnects could leave capability discovery and skill catalogs
stuck on a cached failure for the rest of a thread, even after the executor
reconnected.

## What changed

- Replay capability discovery after executor recovery and retry transient failures
  on later requests while continuing to cache permanent failures.
- Avoid caching skill catalogs produced from failed discovery so a later step can
  load the recovered catalog.
- Mark the MCP runtime dirty when recovered manifests change the projected MCP
  servers, and allow discovery to be cancelled with the turn.

## Testing

- Cover same-request recovery after a disconnect and recovery on a later request.
- Cover retry classification through connection-attempt errors and skill catalog
  caching after discovery recovers.

GitOrigin-RevId: a57f90844351e73ea831931f72a9ddc4e4f3335c
This commit is contained in:
hesham-oai authored and copyberry committed 2026-08-13 19:14:30 +00:00
1 parent 9946da9af1
commit 588e18aae5
10 files changed
+285 -31

No files matched your search

+10 -4
View File
@@ -103,10 +103,9 @@ impl SkillsThreadState {
/// Returns catalogs for stable selected roots.
///
/// The first catalog returned for a root remains cached until this thread state is dropped.
/// Environment availability only controls whether the root is projected into the current
/// step; it never invalidates the cache. There is intentionally no filesystem watcher or
/// content-based invalidation because selected environment roots are treated as stable.
/// Successful catalogs, including empty or warning-bearing catalogs, remain cached until
/// this thread state is dropped. Catalogs backed by failed discovery are not cached, so
/// later steps can recover. There is no filesystem watcher because selected roots are stable.
#[tracing::instrument(
name = "skills.executor.catalog_snapshot",
level = "info",
@@ -140,6 +139,10 @@ impl SkillsThreadState {
providers: &SkillProviders,
query: SkillListQuery,
) -> SkillCatalog {
let discovery_failed = query
.executor_capability_discovery
.as_ref()
.is_some_and(|discovery| discovery.roots().iter().any(|root| root.result.is_err()));
let sandbox_contexts = query
.executor_capability_discovery
.as_ref()
@@ -158,6 +161,9 @@ impl SkillsThreadState {
}
let roots = query.executor_roots.clone();
let discovered = providers.list_executor_for_turn(query).await;
if discovery_failed {
return discovered;
}
let mut cache = self
.executor_discovery_cache
.lock()