Add an opt-in first semantic output budget for native HTTP Responses, including response-header wait. Keep preamble and keepalive bytes non-semantic so a stalled account can fail over once without replaying its response IDs. Defaults remain disabled.
Related to #4201, #4185, and #4248. Complements the HTTP/2 dead-connection fix in #4207.
Follow-up fixes to #3569 based on code audit:
- Expose server_timezone / server_utc_offset in public settings (and the
__APP_CONFIG__ injection payload) and label every peak-window display
with the server UTC offset, so users don't misread the billing window
as browser-local time
- Unify CreateGroup/UpdateGroup peak-config sanitization via a single
NormalizePeakRateConfig chokepoint: non-subscription groups always get
peak fields cleared; unparseable window strings and negative
multipliers are scrubbed when peak is disabled
- Replace hot-path time.Parse in PeakMultiplierAt with a manual HH:MM
parser (accept set verified byte-for-byte identical to
time.Parse("15:04") by exhaustive fuzzing) and reuse it in validation
- Revert the zero-behavior CalculateCost indirection churn in
billing_service/gateway_service introduced by #3569
- Remove dead GetGroupPlatformMap and the duplicate deref helper in the
admin handler package
- Share frontend peak formatting via utils/peak-rate.ts, unify the ×N
label format, and move hardcoded Chinese tooltips to i18n keys
Recent Claude Code CLI versions no longer emit the cch=... signature field in
their x-anthropic-billing-header system block (issue #3358). sub2api still
injected cch=00000 when mimicking Claude Code for OAuth accounts and optionally
signed it, so mimicked requests now diverge from real CLI traffic — the opposite
of what the mimicry is for.
- buildBillingAttributionText emits the block without the cch=00000 segment;
cc_version + cc_entrypoint=cli are kept (detection and Anthropic's first-party
signal rely on the block, not on cch).
- Retire signing: remove the two enableCCH signBillingHeaderCCH call sites in
buildUpstreamRequest / buildCountTokensRequest and delete the now-dead
signBillingHeaderCCH, cchPlaceholderRe, cchSeed, xxHash64Seeded helpers.
- enable_cch_signing is now a documented no-op (kept for backward compat).
- Drop the obsolete signing tests (TestSignBillingHeaderCCH, TestXXHash64Seeded,
TestSanitizeMustBeBeforeCCHSigning_HashConsistency) and update the prompt test
to assert the injected block no longer carries cch=.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Vertex AI's Anthropic endpoint rejects unknown anthropic-beta tokens with
HTTP 400. buildUpstreamRequestAnthropicVertex forwarded the client header
verbatim via the allowedHeaders whitelist, so recent Claude Code CLIs that
send advisor-tool-2026-03-01, prompt-caching-scope-2026-01-05,
redact-thinking-2026-02-12 and thinking-token-count-2026-05-13 broke every
Vertex service_account request, even though plain account-test requests passed.
This is the only upstream builder that bypassed beta filtering: the
OAuth/API-key path uses computeFinalAnthropicBeta and the Bedrock path uses
filterBedrockBetaTokens. Close the gap with a Vertex-specific whitelist
(vertexSupportedBetaTokens) mirroring bedrockSupportedBetaTokens, plus the
existing BetaPolicy block check:
- evaluateBetaPolicy block check (symmetric to resolveBedrockBetaTokensForRequest)
- filterVertexBetaTokens strips policy-filtered + non-whitelisted tokens
- body context_management sanitize now keys on the final beta, not the raw client value
- overwrite the anthropic-beta header after the whitelist copy loop with the final value
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Adds a use-it-or-lose-it scheduling strategy: prefer accounts whose
session window resets soonest, so near-reset accounts get drained first
instead of accounts whose reset is still far away.
Both schedulers, opt-in, default behavior unchanged:
- Anthropic (gateway_service.go): new GatewaySchedulingConfig
.PreferSoonestReset flag. When on, the layered load-aware selection
inserts a filterBySoonestReset stage (priority -> soonest-reset ->
load -> LRU). Accounts with no active SessionWindowEnd are treated as
lowest priority; ties fall through to LRU.
- OpenAI/Codex (openai_account_scheduler.go): new "reset" score weight
in GatewayOpenAIWSSchedulerScoreWeights. Soonest-reset accounts score
higher; weight defaults to 0 (no effect).
SessionWindowEnd (upstream 5h/quota ResetsAt) is already carried in the
scheduler snapshot, so no snapshot changes are needed.
Documented in deploy/config.example.yaml. Adds unit tests for the
Anthropic filter and the OpenAI reset factor.
When an Anthropic upstream returned HTTP 200 but then emitted an SSE
`event: error` frame (overloaded_error / rate_limit_error / api_error /
etc.), Forward's stream branch matched on `err.Error() == "have error in
stream"` and returned `UpstreamFailoverError{StatusCode: 403}` with no
ResponseBody. That dropped three pieces of evidence:
- handleFailoverExhausted → ExtractUpstreamErrorMessage(nil) = "" →
ops_error_logs.upstream_error_message was empty.
- errorPassthroughService.MatchRule(_, 403, nil) could only match rules
without keywords, so keyword-based passthrough rules silently never
fired.
- upstream_errors carried no stream_error record, leaving ops looking at
a generic 403 with no clue whether the upstream was throttled,
overloaded, or rejecting the request. ping-during-slot-wait amplified
this by skipping failover (writerSizeBeforeForward guard), so 403s
ballooned in the ops view well past the upstream's actual rate.
Fix:
- Introduce *sseStreamErrorEventError that carries the SSE data line.
Error() still returns "have error in stream" so existing log searches
keep working.
- Forward extracts via errors.As, appends an OpsUpstreamErrorEvent
(kind="stream_error", with the sanitized message and a truncated raw
body honoring LogUpstreamErrorBody*), and returns
UpstreamFailoverError{StatusCode: 403, ResponseBody: rawJSON}.
StatusCode 403 is preserved verbatim: mapUpstreamError, failover
decisions (shouldFailoverUpstreamError(403)=true), client-visible message,
RetryableOnSameAccount, and rateLimitService side-effects (this path
already didn't invoke them) all match prior behavior. OAuth and API Key
accounts share this path; the API-Key passthrough branch is independent
and already forwards SSE error frames untouched, so it's unaffected.
Adds four unit tests: typed-error contract + RawData, empty data line,
event:error after partial stream output (streamStarted=true), and
non-JSON data line.
The gateway's thinking-block handling was designed for Anthropic's strict
semantics (drop blocks with missing/invalid signature), but third-party
Anthropic-compatible upstreams have INVERTED semantics:
* DeepSeek `/anthropic`, Kimi `/coding`, GLM, Moonshot, qwen-*-thinking
require ALL historical thinking blocks to round-trip verbatim.
* Stripping any of them produces:
400 "The content[].thinking in the thinking mode must be passed back
to the API"
Without this fix, every multi-turn request from a thinking-capable client
(Claude Code, pi, etc.) to such upstreams loses its thinking blocks and
fails. This becomes especially painful when an account's model_mapping
maps `claude-sonnet-4-6 → deepseek-v4-pro` — `reqModel` looks Anthropic
but the upstream contract is the opposite.
Approach
--------
Branch all thinking-block transforms by the *mapped* model id (after
account model_mapping is applied), classifying into three families:
* `anthropic-strict` claude-/opus-/sonnet-/haiku- → existing behaviour
* `passback-required` deepseek-/kimi-/moonshot-/glm-/ → preserve verbatim
qwen-*-thinking
* `unknown` other models → conservative
(preserve, no retry)
Affected entry points (all guarded):
* Pre-filter on outbound: `FilterThinkingBlocks`
Previously dropped blocks with missing/invalid signature; now skips
entirely for non-strict families. Pre-filter is needed because the
post-error retry path can run out of budget on long conversations
(maxRetryElapsed = 10 s).
* 400 retry rectifier: `FilterThinkingBlocksForRetry`
Disables top-level thinking and converts thinking → text. Now skips
for passback-required (those 400s aren't signature errors and any
transformation breaks the round-trip contract).
* 400 retry rectifier (tools): `FilterSignatureSensitiveBlocksForRetry`
Same family-aware short-circuit.
* 400 detector: `shouldRectifySignatureError`
Returns false for passback-required, so the retry path doesn't even
fire.
Tests
-----
* `thinking_protocol_test.go` — classifier across all known vendor
prefixes plus edge cases (empty, case, qwen non-thinking).
* `thinking_protocol_filter_integration_test.go` — locks in that the
three filter entry points return the body byte-for-byte unchanged
when the model id is passback-required or unknown, and still strip
invalid blocks for anthropic-strict.
This PR supersedes #1350 (which only added the pre-filter without the
upstream-family awareness, and would have made third-party upstreams
worse). Once merged, please close#1350.
Reference issues:
- NousResearch/hermes-agent#16748 — DeepSeek /anthropic strip behaviour
- NousResearch/hermes-agent#15700 — DeepSeek thinking:disabled requirement
Co-Authored-By: Claude Opus 4.7 (1M context) <noreply@anthropic.com>
Move anthropic-beta header filtering from separate FilterBedrockBetaHeader
into ApplyBedrockCCCompat, so one function handles all CC compat processing
(body cleanup + header filtering). Change signature from ctx to *gin.Context
to access request headers. Remove the redundant separate call in handler.
Service layer writes a complete JSON error response then returns error.
Handler's ensureForwardErrorResponse couldn't distinguish this from
"no response written" and appended an SSE event, corrupting the body.
Use gin.Context flag: service marks MarkResponseCommitted(c) after
writing, ensureForwardErrorResponse checks IsResponseCommitted(c)
and skips. Zero function signature changes, zero error wrapping.
- bump impersonated CLI version 2.1.92 -> 2.1.161; derive User-Agent from
CLICurrentVersion so the two hardcoded copies can no longer drift apart
- fix x-stainless headers to real 2.1.161 values: package-version
0.70.0 -> 0.94.0, runtime-version v24.13.0 -> v24.3.0 (verified against
the installed Bun-compiled binary)
- expand the disguise-path system prompt from a 2-block identity skeleton
to a 3-block layout (billing + identity + tool-agnostic prose), matching
real CC's multi-block shape; cache breakpoint moved to the last static
block. Deliberately excludes # Doing tasks / # Using your tools /
# Executing actions to avoid polluting proxied-client behavior
- stabilize the synthesized metadata.user_id session_id across conversation
turns: derive it from (account + client discriminator + first user
message) instead of a per-turn content/body hash. Sticky-routing
GenerateSessionHash is intentionally left untouched; remove now-dead
hashBodyForSessionSeed
Tests: update the 3-block system assertions in gateway_prompt_test and
gateway_anthropic_apikey_passthrough_test; add a session_id cross-turn
stability test in gateway_oauth_metadata_test.
When channel pricing is configured, image output token price now
correctly overrides all other sources (LiteLLM, group config).
Add ImageOutputPriceExplicit flag to distinguish intentional zero
price from missing price, preventing fallback to output price.
Affects Gemini, OpenAI inline image gen, and Images API paths.
Keep Anthropic request body rewrites attempt-local and synchronize the accepted wire body only after upstream success so failover and retry paths do not reuse stale parsed state.
Keep large gateway payloads as raw body ranges and bind OpenAI parsed-body caches to the body bytes so failover and mapping do not reuse stale mutable state.
Anthropic count_tokens rejects generation-only fields such as temperature, top_p, top_k, stream, and stop sequences. Passing the original messages payload through unchanged can turn otherwise valid requests into upstream 400 errors.
Sanitize only the count_tokens upstream payload after the gateway's existing request normalization, preserving fields that existing compatibility paths rely on while removing parameters the count_tokens endpoint does not accept.
Fixes#2764
Co-authored-by: Cursor <cursoragent@cursor.com>
王鹏