Files
sub2api/backend/internal
cxx-cod 277aa1411d fix(openai): end streaming at terminal event instead of waiting for upstream EOF
The streaming forward loops treated upstream EOF as the only normal exit
condition; response.completed/[DONE] only set a flag. With keep-alive /
HTTP/2 connection reuse to the upstream, the upstream may hold the
stream open for 8-46s after the terminal event, during which the
gateway could only emit downstream keepalives, inflating tail latency.

Finish the stream as soon as the terminal event frame has been fully
written downstream (usage is already parsed from the terminal event),
and close the upstream body to unblock the reader goroutine. Applied
to the guarded/async, sync, and passthrough streaming paths. The WS
HTTP bridge already behaved this way.

Adds a regression test with an upstream that hangs after
response.completed.
2026-09-01 19:32:24 +00:00
..