Follow-up to the event-time proxy snapshot: every upstream attempt now
carries attribution, the direct/unknown contract is enforced everywhere,
and the queued upstream_errors array has hard bounds again.
Attribution coverage
- compact fallback retry helper and the streaming compact failover event
now stamp proxy_id/proxy_name; the non-streaming SSE compact retry
previously dropped its first attempt on `continue` and now records it
- raw chat-completions stream truncation events (new on main) attributed
- Anthropic/Bedrock transport errors get attribution inside the shared
handleUpstreamTransportError helper, so all five callers inherit it
- OpenAI first-output timeout takes attribution from the caller: HTTP
callers pass the HTTP rule, the WS passthrough relay passes the WS rule
(no managed proxy => unknown, never direct, because http.DefaultClient
may honor environment proxies)
Contract hardening
- single opsUpstreamProxyAttribution returns (id, name) from one decision;
a hydrated Proxy without a durable ID is unknown, not id=null+name
- both normalizers (struct at append time, JSON on detail reads) enforce
proxy_id==null <=> sentinel name; blank managed name => "proxy"
- detail-read normalization edits only the attribution keys via sjson, so
keys from older struct versions and stored key order are preserved
Queue bounds
- keep at most 256 attempts and ~512 KiB serialized per entry, newest
first; the oldest retained event records dropped_earlier_attempts
- attempts outside the 16-event body window trim url/message to 512 bytes
- restore the empty-event drop rule, judged before the body window clears
detail so older real attempts survive
Tests cover each site above (managed proxy, direct, WS unknown), the
normalizer invariants, key/order preservation, and the count/byte bounds.
Long-running LLM inferences (such as high-thinking Gemini models) may experience
silent TCP disconnections across intermediate proxies, NAT gateways, or firewalls
when no data frames are emitted for tens of seconds to minutes.
Go's default http2.Transport sets ReadIdleTimeout=0 (disabling active PING checks).
This commit introduces HTTPUpstreamProfileLongStream with:
- ReadIdleTimeout = 10s
- PingTimeout = 5s
When a pooled H2 connection stays idle for 10s during long streams, active PING frames
are dispatched. Dead connections failing to respond within 5s are proactively evicted,
preventing client generation requests from hanging on dead sockets or failing with EOF/502.
Signed-off-by: WUJI-Labs <zoujincool@gmail.com>
The streaming forward loops treated upstream EOF as the only normal exit
condition; response.completed/[DONE] only set a flag. With keep-alive /
HTTP/2 connection reuse to the upstream, the upstream may hold the
stream open for 8-46s after the terminal event, during which the
gateway could only emit downstream keepalives, inflating tail latency.
Finish the stream as soon as the terminal event frame has been fully
written downstream (usage is already parsed from the terminal event),
and close the upstream body to unblock the reader goroutine. Applied
to the guarded/async, sync, and passthrough streaming paths. The WS
HTTP bridge already behaved this way.
Adds a regression test with an upstream that hangs after
response.completed.
Moonshot pay-as-you-go and Kimi For Coding both expose native
/v1/responses. Stop converting Kimi Responses traffic to Chat
Completions, and treat Kimi like DeepSeek for protocol selection while
keeping /v1/responses URL construction.
classifySelectionFailureError upgrades a selection failure to 429
rate_limit_error whenever the scheduler's compact reason string matches
model_rate_limited=(\d+) with a positive count. It does so unconditionally,
including when the fallback it was handed is the authoritative 404
model_not_found classification.
Selection failure reasons are additive, so both can appear at once, e.g.
"pool=9, filtered: model_not_supported=8 model_rate_limited=1": eight accounts
cannot serve the model at all and the ninth happens to sit in a per-model
cooldown. classifyNoAccountError only produces ModelNotFound through
DiagnoseModelAvailabilityForPlatform, a dedicated database query over
persistent eligibility (active + schedulable + model_mapping), so it has
already established that no account in the group can serve the model. One
transient cooldown does not make "All available accounts are currently
rate-limited" true.
The mislabel is not cosmetic:
- Retrying can never succeed, yet 429 explicitly invites a retry.
- Clients that treat 429 as a rate limit retry hard and swallow the body;
Codex surfaces only "exceeded retry limit, last status: 429", discarding the
one message that names the real problem. Operators then look for capacity
and rate limits instead of a model whitelist.
- Ops attribution becomes self-contradictory: classifyNoAccountErrorFromGin
has already marked the request business-limited for local model
configuration, while call sites gate markOpsRoutingCapacityLimitedIfNoAvailable
on ModelNotFound, which the override clears, so the same request is also
counted as routing capacity limited.
Return the fallback untouched when it is ModelNotFound. The 429 upgrade is
preserved for every other fallback, which is the case it was written for:
accounts that can serve the model exist but are all cooling down, where the
fallback is 503 and retrying is meaningful.
Claude-Session: https://claude.ai/code/session_01LwvyJVV2sdqiu41Z9JDNUq
filterSchedulerExtra is an allowlist and never carried openai_passthrough /
openai_oauth_passthrough, while credentials.model_mapping is preserved in full.
Candidate filtering (ListSchedulableAccounts) reads that trimmed projection
(sched:meta:<id>), so there a passthrough account looks exactly like a
whitelist-mode one: Account.IsModelSupported falls back to the model_mapping
whitelist and the account is dropped as model_not_supported. Forwarding, which
hydrates the full account from sched:acc:<id>, still honors passthrough - hence
"testing the account directly succeeds, but the gateway reports no available
accounts supporting model X".
The practical impact is that a stale leftover whitelist silently shrinks the
candidate pool of accounts that were explicitly configured to let the upstream
decide. When no account in the group lists the requested model, the group is
fully dark for it even though every account can serve it.
This is not a regression of #4936: that fix corrected the predicate
(Account.IsModelSupported short-circuits passthrough before the mapping check),
this one restores the input the predicate is fed. The projection allowlist
predates #4936 by three and a half months and has never carried the flag, so
the fix simply never took effect on the snapshot path.
Add both keys to the allowlist, plus a regression test that round-trips the
projection through JSON (the real sched:meta write/read path) and asserts the
model gate still lets a non-whitelisted model through for a passthrough account.
Claude-Session: https://claude.ai/code/session_01LwvyJVV2sdqiu41Z9JDNUq
Clients now send body.fallbacks; upstream rejects it without
anthropic-beta: server-side-fallback-2026-07-01. Mirror the
context_management sanitize so OAuth mimic / default API-key
beta paths no longer 400 with "fallbacks: Extra inputs are not permitted".
OpenAI/Composite groups can now reject requests that exceed the max
reasoning effort instead of always rewriting them down to the ceiling.
Existing groups keep the previous auto-downgrade behavior.
wucm667