diff --git a/.github/sowerd.service b/.github/sowerd.service index fee4a61..93e9500 100644 --- a/.github/sowerd.service +++ b/.github/sowerd.service @@ -7,6 +7,9 @@ Type=simple ExecStart=/usr/local/bin/sowerd Environment="FAKE_SITE=127.0.0.1:8080" Environment="PASSWORD=I_am_Passw0rd" +# Soft heap ceiling: a runaway heap degrades the service instead of +# OOM-killing the host (docs/security.md, Connection Resource Bounds). +Environment="GOMEMLIMIT=512MiB" Restart=on-failure RestartSec=1min diff --git a/.github/workflows/master.yml b/.github/workflows/master.yml index 9793480..d912d8b 100644 --- a/.github/workflows/master.yml +++ b/.github/workflows/master.yml @@ -11,7 +11,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-go@v7 with: - go-version: ^1.26 + go-version: ^1.27 - uses: oven-sh/setup-bun@v2 - name: test and build diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 41712ac..4c22207 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -92,7 +92,7 @@ jobs: - uses: actions/checkout@v7 - uses: actions/setup-go@v7 with: - go-version: ^1.26 + go-version: ^1.27 - uses: actions/download-artifact@v8 with: name: web-dist diff --git a/docs/plans/2026-10-12-sowerd-connection-bounds.md b/docs/plans/2026-10-12-sowerd-connection-bounds.md index 1c70de6..0008566 100644 --- a/docs/plans/2026-10-12-sowerd-connection-bounds.md +++ b/docs/plans/2026-10-12-sowerd-connection-bounds.md @@ -1,6 +1,20 @@ # Plan: sowerd 连接资源上界(2026-10-12 OOM 复盘) -状态:实施中(2026-10-12)。本文为非权威记录;定稿后的长期规则回写 [security.md](../security.md)。 +状态:已定稿并回写(2026-10-12)。本文为非权威历史记录,权威规则见 [security.md](../security.md) 的 +"Connection Resource Bounds"。 + +实施结果摘要: + +- 落地 commit `28c4f9e`:`newRedirectServer` 补齐 :80 的四个超时;`serve443` 加 + `maxInboundConns = 2048` 并发上界;`writeStallListener`/`writeStallConn` 位于 TLS + 之下,为每次写入武装 `writeStallTimeout`;`dumpHeapOnSignal` 在 SIGUSR1 时写 heap profile。 +- kr 已部署 `v1.7.1-4-g28c4f9e`(sha256 `9ec44038190b...`,备份 + `/usr/local/bin/sowerd.bak-20261012175623`),并以 drop-in + `/etc/systemd/system/sowerd.service.d/10-memory.conf` 设 `GOMEMLIMIT=512MiB`。 +- 验证:`:80` 仍 308 跳转;`:443` 回退反代 200;SIGUSR1 产出 + `/var/cache/sower/sowerd-heap-*.pprof`;用 `transport/sower` 从本机经 kr 中继 + `www.baidu.com:80` 得到 HTTP/1.0 200(证明新增的写超时不误伤中继);r6c 客户端 + socks5 到 google 仍 204,其日志自 17:31 起无握手失败。 ## 事件 diff --git a/docs/security.md b/docs/security.md index 710a897..73efb71 100644 --- a/docs/security.md +++ b/docs/security.md @@ -207,6 +207,36 @@ sower clients should set `[remote.tls] client_hello = "chrome"` (uTLS) in deployment configs. This is a client configuration knob, not a protocol property, and has no server-side effect. +## Connection Resource Bounds + +Both listeners are internet-facing, so an inbound connection must have a +bounded cost and a bounded lifetime. These bounds are normative: + +- The plain-HTTP listener (`:80`) sets `ReadHeaderTimeout`, `ReadTimeout`, + `WriteTimeout` and `IdleTimeout`. It only ever serves a redirect or an ACME + challenge, so a connection that is slow, idle or half-sent has nothing to + wait for; without the timeouts a peer can pin a goroutine, a socket and its + buffers indefinitely, and nothing about it is logged. +- The TLS listener (`:443`) serves at most `maxInboundConns` connections at + once; a connection that finds no slot is closed on accept instead of queued. + Per-connection state (TLS record buffers, relay copy buffers, goroutine + stacks) is what turns an arrival-rate burst into resident memory. +- Every write on a `:443` connection carries a deadline (`writeStallTimeout`, + armed below the TLS listener, so handshake, reverse-proxied response and + hijacked tunnel are all covered). The relay library only arms read + deadlines, so a peer that stops reading would otherwise block its writer + forever, leaking the connection pair, its goroutines and its descriptors. + Idle tunnels issue no writes and are not affected. +- A heap profile is written per `SIGUSR1` (`systemctl kill -s USR1 sowerd`, + read with `go tool pprof`). It is the only way to attribute a grown heap to + an allocation site, and unlike an HTTP endpoint it adds no network surface. + +The service unit also carries a soft heap ceiling (`GOMEMLIMIT`), so an +unbounded heap degrades this service instead of OOM-killing the host: the +2026-10-12 OOM on `kr` (970 MiB anon-rss on a 1.6 GiB host) killed the machine, +not the service. Historical analysis and the still-unproven trigger live in `docs/plans/` +(2026-10-12, sowerd connection bounds; non-authoritative). + ## Rejected Directions Decisions recorded here so they are not re-litigated without new evidence: diff --git a/internal/install/service.go b/internal/install/service.go index 36b830e..1bb7c43 100644 --- a/internal/install/service.go +++ b/internal/install/service.go @@ -191,6 +191,9 @@ Wants=network-online.target [Service] Type=simple ExecStart="%s" -c "%s" +# Soft heap ceiling: a runaway heap degrades this service instead of +# OOM-killing the host (see docs/security.md, Connection Resource Bounds). +Environment=GOMEMLIMIT=512MiB Restart=always RestartSec=5s