Follow-up to the event-time proxy snapshot: every upstream attempt now
carries attribution, the direct/unknown contract is enforced everywhere,
and the queued upstream_errors array has hard bounds again.
Attribution coverage
- compact fallback retry helper and the streaming compact failover event
now stamp proxy_id/proxy_name; the non-streaming SSE compact retry
previously dropped its first attempt on `continue` and now records it
- raw chat-completions stream truncation events (new on main) attributed
- Anthropic/Bedrock transport errors get attribution inside the shared
handleUpstreamTransportError helper, so all five callers inherit it
- OpenAI first-output timeout takes attribution from the caller: HTTP
callers pass the HTTP rule, the WS passthrough relay passes the WS rule
(no managed proxy => unknown, never direct, because http.DefaultClient
may honor environment proxies)
Contract hardening
- single opsUpstreamProxyAttribution returns (id, name) from one decision;
a hydrated Proxy without a durable ID is unknown, not id=null+name
- both normalizers (struct at append time, JSON on detail reads) enforce
proxy_id==null <=> sentinel name; blank managed name => "proxy"
- detail-read normalization edits only the attribution keys via sjson, so
keys from older struct versions and stored key order are preserved
Queue bounds
- keep at most 256 attempts and ~512 KiB serialized per entry, newest
first; the oldest retained event records dropped_earlier_attempts
- attempts outside the 16-event body window trim url/message to 512 bytes
- restore the empty-event drop rule, judged before the body window clears
detail so older real attempts survive
Tests cover each site above (managed proxy, direct, WS unknown), the
normalizer invariants, key/order preservation, and the count/byte bounds.
Long-running LLM inferences (such as high-thinking Gemini models) may experience
silent TCP disconnections across intermediate proxies, NAT gateways, or firewalls
when no data frames are emitted for tens of seconds to minutes.
Go's default http2.Transport sets ReadIdleTimeout=0 (disabling active PING checks).
This commit introduces HTTPUpstreamProfileLongStream with:
- ReadIdleTimeout = 10s
- PingTimeout = 5s
When a pooled H2 connection stays idle for 10s during long streams, active PING frames
are dispatched. Dead connections failing to respond within 5s are proactively evicted,
preventing client generation requests from hanging on dead sockets or failing with EOF/502.
Signed-off-by: WUJI-Labs <zoujincool@gmail.com>
The streaming forward loops treated upstream EOF as the only normal exit
condition; response.completed/[DONE] only set a flag. With keep-alive /
HTTP/2 connection reuse to the upstream, the upstream may hold the
stream open for 8-46s after the terminal event, during which the
gateway could only emit downstream keepalives, inflating tail latency.
Finish the stream as soon as the terminal event frame has been fully
written downstream (usage is already parsed from the terminal event),
and close the upstream body to unblock the reader goroutine. Applied
to the guarded/async, sync, and passthrough streaming paths. The WS
HTTP bridge already behaved this way.
Adds a regression test with an upstream that hangs after
response.completed.
lyy0709