Compare commits

...
3 Commits
Author SHA1 Message Date
wweir 0593934066 docs(plans): record the rollout that was actually verified
r6c now runs the v1.7.7 release artifact (sower-linux-arm64.tar.gz, sower
sha256 8cf0a869..., vcs.modified=false) rather than a local build, so the
verification line names that artifact.
2026-10-13 07:01:54 +08:00
wweir 93ffaec848 docs: fold the review findings into the plans
The sowerd bounds plan still described the recursion driver as unfixed and
carried a deploy log that the board receipt already holds; the client plan
duplicated deploy pitfalls that live in the skill. Both now state what was
verified, point at each other, and keep the regression lesson (the guard must
ask the upstreams, not the system resolver) without restating the mechanism
twice.
2026-10-13 06:58:40 +08:00
wweir 116d16b98b refactor(sower): guard the dial exits instead of the routing entry
Two findings from reviewing the self-target guard:

- the guard sat at the top of DialSmart, so a blocked target that also points
  at this client reported ErrSelfProxyTarget instead of ErrBlocked, and every
  name on a served port paid an upstream lookup before the rules were even
  consulted. It now sits in directDial, the only other dial exit, so blocked
  targets are decided without resolution and policy keeps precedence.
- selfProxyLookupTimeout was left behind by the switch to upstream
  resolution; drop it.
2026-10-13 06:58:40 +08:00
6 changed files with 64 additions and 93 deletions
@@ -1,6 +1,6 @@
# Plan: 拒绝自指代理目标(2026-10-12 r6c↔kr 递归)
状态:实施中(2026-10-12)。本文为非权威记录;长期规则回写 [ARCHITECTURE.md](../../ARCHITECTURE.md) 的 Design Decisions。
状态:已定稿并回写(2026-10-12,v1.7.6)。本文为非权威历史记录;权威规则见 [ARCHITECTURE.md](../../ARCHITECTURE.md) 的 Design Decisions。
## 问题
@@ -55,33 +55,24 @@ finding `f-4d7d5bd2-cee`):
## 验证
- `go vet ./...`、`go test ./...` 全绿;新增用例覆盖:IP 字面量自指、名字解析到自指、
非自指(含未服务端口不解析)、未指定地址绑定(`0.0.0.0:443` 匹配任意本地地址)、
解析失败放行、决策缓存(同名只解析一次);`cmd/sower` 侧钉住 `selfProxyAddrs` 的集合。
- 现场:r6c 升级后,从 tailnet 发 `curl -x http://100.64.1.1:80 http://sower.wweir.fun/`
应得 403、`--socks5-hostname` 应被规则拒绝,且两侧连接数不增长;正常代理(google 204、
baidu 200)与 admin 控制台不受影响。
- 单元测试:`go vet ./...`、`go test ./...` 全绿。守卫用例覆盖 IP 字面量自指、上游解析到自指、
上游解析到远端(正常被代理域名必须放行)、未服务端口零解析、未指定地址绑定(`0.0.0.0:443`
匹配任意本地地址)、解析失败/空答案放行、决策缓存;`cmd/sower` 侧钉住 `selfProxyAddrs`
的集合,以及“拒绝回 403 而不是静默关闭”。
- 现场(r6c,v1.7.7 release 资产 `sower-linux-arm64.tar.gz`,sha256 `8cf0a869…`、
`vcs.modified=false`):自指 `curl -x http://100.64.1.1:80 http://sower.wweir.fun/`
→ `403 Forbidden`;`--socks5-hostname` 自指 → SOCKS5 码 `2`;正常路径 `--socks5-hostname`
到 google `204`、`-x http://100.64.1.1:80` 到 baidu `200`、`/api/config` `401`;kr 上 sowerd
到 `100.64.1.1` 的连接数为 `0`(修复前峰值 601)。
## 现场验证与一次回归(v1.7.4 → v1.7.5)
### 一次必须记住的回归(v1.7.4 → v1.7.5)
首版守卫(`v1.7.4`,commit `2de08e6`)用系统解析器做自指判定,部署到 r6c 后现场验证立刻
暴露反例:
- `curl --socks5-hostname 100.64.1.1:1080 https://www.google.com/generate_204` 返回
SOCKS5 码 `2`(not allowed)而不是 204;r6c 的 `logread` 给出根因——
`target is this client's own listener: sower.wweir.fun:80 resolves to 100.64.1.1:80`。
- 同时 `nslookup www.google.com 127.0.0.1` 在 r6c 上返回 `100.64.1.1`:本机 DNS 对被代理
域名回答的就是自己的 serve IP。于是“解析结果是我的地址”对**所有正常被代理域名**都成立,
守卫把整条正常代理路径拒了。这是判定依据错误,不是实现细节错误。
处理:立即把 r6c 回滚到上一个可用二进制(`/usr/sbin/sower.bak-20261012195538`,sha256
`76459b95…`,procd 重启后 pid 变化确认),随后改成向上游解析器求证(`v1.7.5`)并重新部署。
部署坑(已记入 deploy skill):跨机传二进制用 `cat > /tmp/sower.new` 得到的是 `0644`,
`cp -a` 会原样保留该权限,procd 因为不可执行而根本起不来(`/usr/sbin/sower: Permission
Denied`),表现为服务静默 down、LAN DNS 连锁失效。替换后必须 `chmod 755` 并用
`sha256sum` + `pidof` 验证;同时 r6c 的 SSH 在 sower down 时可能连 `home` 跳板的域名解析
都失败,此时改用 IP(`wweir@192.168.1.2` → `root@192.168.1.1`)。
首版守卫(`v1.7.4`,commit `2de08e6`)用系统解析器判定,现场立刻被推翻:`--socks5-hostname`
到 google 返回 SOCKS5 码 `2`,而 `nslookup www.google.com 127.0.0.1` 在 r6c 上返回
`100.64.1.1` —— 本机 DNS 按设计对每个被代理域名都回答自己的 serve IP。于是“解析结果是我的
地址”对**所有正常被代理域名**都成立,守卫把整条正常代理路径拒了:**这是判定依据错,不是
实现细节错**。处理是把 r6c 回滚到上一版可用二进制,改成向上游解析器求证后以 `v1.7.5` 重发
(部署期间的权限位与 LAN DNS 连锁坑见 skill `sower-deploy` 的 Pitfalls)。
## 未决(不在本次范围)
@@ -10,18 +10,13 @@
TLS 之下,为每次写入武装 `writeStallTimeout`;`dumpHeapOnSignal`(`heapdump_unix.go`,
其他平台 no-op)在 SIGUSR1 时写 heap profile;`GOMEMLIMIT=512MiB` 进入安装器 unit、
打包 unit 与 kr drop-in。
- 发布:`v1.7.2` 的 Release run 在 windows-amd64/arm64 上失败(`syscall.SIGUSR1` 在
Windows 未定义),Attach release assets 被跳过、没有发布任何资产;`06c357c` 把信号
处理挪到 `//go:build unix` 文件后以 `v1.7.3` 重新发布(12 个平台构建 + Attach assets 全绿)。
- kr 运行的是 **v1.7.3 release 资产**:tarball `sower-linux-amd64.tar.gz` 经
`SHA256SUMS` 校验,`sowerd` sha256 `74b34fd9119d45d0d8de2013656a292ed2ffa08ec7e6a2dff6cac9262fd6d428`,
`go version -m` 显示 `mod v1.7.3`、`vcs.revision=06c357c`、`vcs.modified=false`;
备份 `/usr/local/bin/sowerd.bak-20261012191818`(上一版为本地构建 v1.7.1-4-g28c4f9e,
备份 `sowerd.bak-20261012175623`)。
- 验证:`:80` 仍 308 跳转;`:443` 回退反代 200;SIGUSR1 产出
`/var/cache/sower/sowerd-heap-*.pprof`;用 `transport/sower` 从本机经 kr 中继
`www.baidu.com:80` 得到 HTTP/1.0 200(证明新增的写超时不误伤中继);r6c 客户端
socks5 到 google 仍 204,其日志自 17:31 起无握手失败;`GOMEMLIMIT=512MiB` 在进程中可见。
- 已发布 `v1.7.3`(`v1.7.2` 因 Windows 上 `syscall.SIGUSR1` 未定义而未发布);kr 运行的是
该 release 资产的 `sowerd`。
- OOM 的**驱动**随后被定位为自指代理递归并在客户端修复(`v1.7.5`/`v1.7.6`,见
[2026-10-12-self-proxy-target-guard.md](2026-10-12-self-proxy-target-guard.md));本文件的
连接上界是下限保护,不是根因修复。
- 验证:`:80` 308 跳转;`:443` 回退反代 200;SIGUSR1 产出 heap profile;`transport/sower`
经 kr 中继 `www.baidu.com:80` 得到 HTTP/1.0 200(新增写超时不误伤中继)。
## 事件
@@ -78,23 +73,18 @@ kr 的 zram swap 因 `vm.swappiness=0` 从未被使用。
(`systemctl kill -s USR1 sowerd`,`go tool pprof -http :0 <file>`),不新增网络面。
4. 部署侧:sowerd 单元加 `Environment=GOMEMLIMIT=512MiB`,让失控堆退化服务而不是
杀死宿主机。
5. 发布工具链:v1.7.1 是唯一用 go1.26.8 构建的制品(现场其余二进制为
go1.27.1/1.27.2),而 go1.26 的 Green Tea GC 有公开的内存回归报告
(golang/go#77567、#77361)。CI 的 `go-version: ^1.26` 收敛到 `^1.27`。
5. 发布工具链:CI 的 `go-version: ^1.26` 收敛到 `^1.27`,与本地/现场构建一致。当时曾怀疑
go1.26.8 的 GC 回归(golang/go#77567、#77361)是原因,后证明是自指递归;保留这个改动只是
让制品构建环境不再漂移。
## 未决
- **驱动已由另一会话定位(2026-10-12 18:44–18:50,board 消息 `m-77507977-64f`)**:任何
指名 `sower.wweir.fun` 的代理请求会形成自持链——该名字的公网 A 记录指向 r6c 的
tailscale 地址 `100.64.1.1`(r6c 上 sower 监听 80/443/53/1080),`cmd/sower` 无条件下
发给 kr,kr 解析同名后回dial 进 r6c:80,再转发,逐跳生成下一跳。实测 r6c 上
`100.64.1.1:80` 连接数 155→254→305→495→601(对端全是 kr),且触发它的 curl 退出后
仍在增长;kr 上 sowerd 同期 311 个 dial、649 个 fd。这与本次 OOM 的窗口、双向近似
对称流量、静默增长、以及“sowerd 一死链就断”全部吻合;18:50 重启 r6c 的 sower 后两侧
都回到 0 连接。sowerd 侧的 cap 与写超时只把它**限住**(链在 2048 处停止增长、内存有界),
真正的修法是客户端拒绝把请求 dial 回自己的监听地址(另一会话已开高优先级 finding)。
触发链是否就是 17:30 那次膨胀的唯一成因仍无直接证据(当时进程已死、无 pprof、sar 缺样),
因此上界与 SIGUSR1 dump 的价值是把下一次复现变成可定位事件,而不是宣称已修复根因。
- **驱动已定位并修复(见姊妹文件)**:任何指名 `sower.wweir.fun`(公网 A 记录 = r6c 的
tailscale 地址 `100.64.1.1`)的代理请求会形成自持链:实测 r6c 上 `100.64.1.1:80` 连接数
155→601、kr 上 sowerd 311 个 dial / 649 个 fd,且触发请求退出后仍增长;客户端守卫已在
`v1.7.5`/`v1.7.6` 拒绝这类自指拨号。本文件的连接上界因此是下限保护(链在 2048 处停止
增长、内存有界);该链是否就是 17:30 那次膨胀的唯一成因仍无直接证据(进程已死、无 pprof、
sar 缺样),SIGUSR1 dump 的价值是把下一次复现变成可定位事件。
- `relay.Relay` 的写超时缺陷属于 `sower-proxy/conns` 仓库,应在那里修(finding
`f-1ae16668-b92`)。sowerd 侧只做上界,不给隧道加全局空闲超时,避免误杀合法但长时间
空闲的隧道。
+4 -5
View File
@@ -231,11 +231,10 @@ bounded cost and a bounded lifetime. These bounds are normative:
read with `go tool pprof`). It is the only way to attribute a grown heap to
an allocation site, and unlike an HTTP endpoint it adds no network surface.
The service unit also carries a soft heap ceiling (`GOMEMLIMIT`), so an
unbounded heap degrades this service instead of OOM-killing the host: the
2026-10-12 OOM on `kr` (970 MiB anon-rss on a 1.6 GiB host) killed the machine,
not the service. Historical analysis and the still-unproven trigger live in `docs/plans/`
(2026-10-12, sowerd connection bounds; non-authoritative).
The service unit also carries a soft heap ceiling (`GOMEMLIMIT`): a heap that
grows without bound must degrade this service, not take the host down with it
(the 2026-10-12 OOM on `kr` reached 970 MiB anon-rss on a 1.6 GiB host).
Motivation and the historical analysis live in `docs/plans/` (non-authoritative).
## Rejected Directions
+14 -15
View File
@@ -222,15 +222,6 @@ func (r *Router) Dial(network, domain string, port uint16) (net.Conn, error) {
func (r *Router) DialSmart(network, domain string, port uint16) (net.Conn, error) {
ctx := context.Background()
addr := net.JoinHostPort(domain, strconv.FormatUint(uint64(port), 10))
// Guarded here as well as in DialProxyOnly: this path reaches directDial
// without passing it (rule-based and probe-based direct decisions), and
// dialing a listener this client itself forwards on re-enters the proxy
// either way — locally, in that case, since no tunnel is involved.
if ip, self := r.selfTarget(domain, port); self {
return nil, selfProxyError(domain, port, ip)
}
// 1. rule_based( block > direct > proxy )
// 2. detect_based( CN IP || access site )
@@ -243,14 +234,14 @@ func (r *Router) DialSmart(network, domain string, port uint16) (net.Conn, error
case r.DirectRule.Match(domain):
r.observe(RouteDirect, domain)
r.observeRuleHit(RouteDirect, domain)
return r.directDial(ctx, network, addr)
return r.directDial(ctx, network, domain, port)
case r.ProxyRule.Match(domain):
r.observeRuleHit(RouteProxy, domain)
return r.DialProxyOnly(network, domain, port)
case r.localSite(ctx, domain), r.isAccess(domain, port):
r.observe(RouteDirect, domain)
r.observeRuleMiss(domain)
return r.directDial(ctx, network, addr)
return r.directDial(ctx, network, domain, port)
default:
r.observeRuleMiss(domain)
return r.DialProxyOnly(network, domain, port)
@@ -261,9 +252,8 @@ func (r *Router) DialProxyOnly(network, domain string, port uint16) (net.Conn, e
if r.ProxyDial == nil {
return nil, fmt.Errorf("proxy dialer unavailable")
}
// Guards every path that reaches the tunnel, explicit proxy requests and
// auto-routed traffic alike: a target that resolves to a listener this
// client forwards on would be forwarded again, forever.
// A target that resolves to a listener this client forwards on would be
// forwarded again, forever: the tunnel is one of the two dial exits.
if ip, self := r.selfTarget(domain, port); self {
return nil, selfProxyError(domain, port, ip)
}
@@ -276,7 +266,16 @@ func (r *Router) DialProxyOnly(network, domain string, port uint16) (net.Conn, e
return rc, nil
}
func (r *Router) directDial(ctx context.Context, network, addr string) (net.Conn, error) {
// directDial is the other way into this client's own listeners: a rule or an
// access probe can classify a name that points at one of them as direct, and
// that connection re-enters the proxy just the same — locally, since no tunnel
// is involved. Both dial exits are therefore guarded, not just the tunnel.
func (r *Router) directDial(ctx context.Context, network, domain string, port uint16) (net.Conn, error) {
if ip, self := r.selfTarget(domain, port); self {
return nil, selfProxyError(domain, port, ip)
}
addr := net.JoinHostPort(domain, strconv.FormatUint(uint64(port), 10))
start := time.Now()
dialer := net.Dialer{Timeout: 5 * time.Second}
conn, err := dialer.DialContext(ctx, network, addr)
+1 -3
View File
@@ -16,9 +16,7 @@ const (
// self-target check. A proxied dial must not pay a DNS round trip per
// connection, and a name that points at this client keeps pointing at it
// for far longer than this window.
selfProxyLookupTTL = time.Minute
// selfProxyLookupTimeout matches localSite's resolution budget.
selfProxyLookupTimeout = 2 * time.Second
selfProxyLookupTTL = time.Minute
selfProxyCacheMaxNames = 1024
)
+12 -18
View File
@@ -100,27 +100,21 @@ func TestDialProxyOnlyRefusesUnspecifiedBind(t *testing.T) {
}
}
// A resolution failure says nothing about self-targeting: the dial proceeds so
// the relay reports its own resolution result.
func TestDialProxyOnlyAllowsTargetThatFailsToResolve(t *testing.T) {
// Neither a resolution failure nor an empty answer is evidence of
// self-targeting: the dial proceeds so the relay reports its own result, and
// the empty decision is cached so a failing lookup is not paid per connection.
func TestDialProxyOnlyAllowsTargetWithoutAnAnswer(t *testing.T) {
t.Parallel()
r := newSelfTargetRouter(t, []string{"100.64.1.1:80"}, func(string) ([]net.IP, error) {
return nil, errors.New("no such host")
})
if _, err := r.DialProxyOnly("tcp", "missing.example.com", 80); err != nil {
t.Fatalf("dial after a resolution failure: %v", err)
resolvers := map[string]func(string) ([]net.IP, error){
"resolution failure": func(string) ([]net.IP, error) { return nil, errors.New("no such host") },
"empty answer": func(string) ([]net.IP, error) { return nil, nil },
}
}
func TestDialProxyOnlyAllowsTargetWithNoAnswer(t *testing.T) {
t.Parallel()
r := newSelfTargetRouter(t, []string{"100.64.1.1:80"}, func(string) ([]net.IP, error) {
return nil, nil
})
if _, err := r.DialProxyOnly("tcp", "nodata.example.com", 80); err != nil {
t.Fatalf("dial after an empty answer: %v", err)
for name, resolve := range resolvers {
r := newSelfTargetRouter(t, []string{"100.64.1.1:80"}, resolve)
if _, err := r.DialProxyOnly("tcp", "missing.example.com", 80); err != nil {
t.Errorf("%s: dial = %v, want it to proceed", name, err)
}
}
}