Files
sub2api/backend/internal/service
HuntercodeT 60756c0ca4 fix(openai): emit pre-output SSE keepalive on the /v1/responses passthrough path
The passthrough streaming loop buffers leading events in `pendingLines` until
the first visible output, so a client receives **zero bytes** — not even the
HTTP response headers, since gin's ResponseWriter only commits them on the
first write — for as long as the upstream takes to produce its first visible
event.

The Forward path already recognises this exact hazard and solves it with a
downstream keepalive:

    // Track downstream writes separately from upstream reads: pre-output
    // failover can buffer response.created / response.in_progress, so
    // keepalive must be based on downstream idle time.

The passthrough path has the same buffering but never starts a keepalive.
Reasoning models routinely think for several minutes before emitting a visible
event, so intermediate proxies close the connection on idle timeout.

Production evidence from one deployment (12h window, gpt-5.6 family):

  * 44 `/v1/responses` requests produced their first visible output only after
    600–900 s, each carrying 30k–50k output tokens — the upstream had finished
    the work.
  * Every one of them was killed by an intermediate nginx `proxy_read_timeout`
    (600 s) and returned 504. The user received nothing.
  * Cutting early does not even save resources: the code deliberately keeps
    draining the upstream for usage after the client disconnects.

Fix: start the existing keepalive helper for the passthrough stream as well.

* `StartOpenAICompactSSEKeepalive` keeps its compact-marker gate; the gate-free
  body is extracted as `startOpenAISSEKeepalive` for callers that already know
  they are in an SSE context.
* The passthrough loop starts it when `gateway.stream_keepalive_interval > 0`
  and stops it immediately before its first real write, so the ResponseWriter
  is never shared.

Pre-output failover is unaffected: keepalive bytes are already excluded by
`OpenAICompactKeepaliveAdjustedWrittenSize`, which `openAIStreamClientOutputStarted`
on this very path already consults (#3887). No pendingLines and no
account-specific headers are ever emitted by a keepalive.

Behaviour is unchanged when `gateway.stream_keepalive_interval` is 0.
2026-08-28 10:19:43 +08:00
..
2026-07-08 15:26:54 +08:00
2026-07-04 05:30:50 +08:00
2026-07-04 05:30:50 +08:00
2026-03-13 17:00:16 +08:00
2026-03-13 19:15:27 +08:00
2026-07-01 11:32:22 +08:00
2026-01-05 17:07:29 +08:00
2026-01-15 15:14:44 +08:00
2026-01-15 15:14:44 +08:00
2026-01-15 15:14:44 +08:00
2026-02-02 22:13:50 +08:00
2026-07-28 10:08:39 +08:00
2026-07-06 12:22:04 +08:00