[codex] Add turn profiling analytics (#26484)

## Summary

Add flat profiling fields to `codex_turn_event` so analytics can explain
where turn wall-clock time is spent without changing tool execution
behavior.

The profile reports:
- time before the first sampling request
- sampling time across all attempts and follow-ups
- overhead between sampling requests
- time blocked in the post-sampling tool drain
- time after the final sampling request
- sampling request and retry counts

## Implementation

- Extend the existing turn timing state with constant-memory phase
accounting and one RAII phase guard.
- Observe sampling and the existing post-sampling drain only at turn
orchestration boundaries.
- Keep tool runtime, tool futures, response item handling, and turn
lifecycle values unchanged.
- Add the profiling fields directly to the existing analytics turn event
without changing app-server protocol or rollout persistence.
- Use the existing turn `status` to distinguish completed, failed, and
interrupted profiles.

Exact sampling/tool overlap is intentionally omitted because measuring
tool completion accurately would require hooks in the tool execution
path.

## Validation

- Add app-server end-to-end coverage for a single-sampling turn with no
blocking tool work.
- Add app-server end-to-end coverage for `request_user_input` blocking
followed by a second sampling request.
- CI is running on the PR; tests were not executed locally per
repository guidance.
This commit is contained in:
Ahmed Ibrahim authored and GitHub committed 2026-06-05 11:27:10 -07:00
1 parent 82b15b65e2
commit 8d72fb6de9
11 files changed
+530 -101

No files matched your search

+41
View File
@@ -1,13 +1,17 @@
use codex_analytics::TurnProfile;
use codex_protocol::items::AgentMessageItem;
use codex_protocol::items::TurnItem;
use codex_protocol::models::ContentItem;
use codex_protocol::models::FunctionCallOutputPayload;
use codex_protocol::models::ResponseItem;
use pretty_assertions::assert_eq;
use std::time::Duration;
use std::time::Instant;
use std::time::SystemTime;
use std::time::UNIX_EPOCH;
use super::TurnProfilePhase;
use super::TurnProfileState;
use super::TurnTimingState;
use super::response_item_records_turn_ttft;
use crate::ResponseEvent;
@@ -146,3 +150,40 @@ fn response_item_records_turn_ttft_ignores_empty_non_output_items() {
}
));
}
#[test]
fn turn_profile_breaks_down_sampling_blocking_and_retry_overhead() {
let started_at = Instant::now();
let mut state = TurnProfileState::default();
state.start(started_at);
let _ = state.begin_sampling(started_at + Duration::from_millis(100));
state.end_phase(
started_at + Duration::from_millis(600),
TurnProfilePhase::Sampling,
);
let _ = state.begin_tool_blocking(started_at + Duration::from_millis(600));
state.end_phase(
started_at + Duration::from_millis(900),
TurnProfilePhase::ToolBlocking,
);
state.record_sampling_retry();
let _ = state.begin_sampling(started_at + Duration::from_millis(1_000));
state.end_phase(
started_at + Duration::from_millis(1_200),
TurnProfilePhase::Sampling,
);
assert_eq!(
state.complete(started_at + Duration::from_millis(1_300)),
TurnProfile {
before_first_sampling_ms: 100,
sampling_ms: 700,
between_sampling_overhead_ms: 100,
tool_blocking_ms: 300,
after_last_sampling_ms: 100,
sampling_request_count: 2,
sampling_retry_count: 1,
}
);
}