mirror of
https://github.com/wweir/sower.git
synced 2026-10-19 16:14:33 +00:00
Compare commits
| Author | SHA1 | Date | |
|---|---|---|---|
|
|
0593934066 | ||
|
|
93ffaec848 | ||
|
|
116d16b98b |
@@ -1,6 +1,6 @@
|
||||
# Plan: 拒绝自指代理目标(2026-10-12 r6c↔kr 递归)
|
||||
|
||||
状态:实施中(2026-10-12)。本文为非权威记录;长期规则回写 [ARCHITECTURE.md](../../ARCHITECTURE.md) 的 Design Decisions。
|
||||
状态:已定稿并回写(2026-10-12,v1.7.6)。本文为非权威历史记录;权威规则见 [ARCHITECTURE.md](../../ARCHITECTURE.md) 的 Design Decisions。
|
||||
|
||||
## 问题
|
||||
|
||||
@@ -55,33 +55,24 @@ finding `f-4d7d5bd2-cee`):
|
||||
|
||||
## 验证
|
||||
|
||||
- `go vet ./...`、`go test ./...` 全绿;新增用例覆盖:IP 字面量自指、名字解析到自指、
|
||||
非自指(含未服务端口不解析)、未指定地址绑定(`0.0.0.0:443` 匹配任意本地地址)、
|
||||
解析失败放行、决策缓存(同名只解析一次);`cmd/sower` 侧钉住 `selfProxyAddrs` 的集合。
|
||||
- 现场:r6c 升级后,从 tailnet 发 `curl -x http://100.64.1.1:80 http://sower.wweir.fun/`
|
||||
应得 403、`--socks5-hostname` 应被规则拒绝,且两侧连接数不增长;正常代理(google 204、
|
||||
baidu 200)与 admin 控制台不受影响。
|
||||
- 单元测试:`go vet ./...`、`go test ./...` 全绿。守卫用例覆盖 IP 字面量自指、上游解析到自指、
|
||||
上游解析到远端(正常被代理域名必须放行)、未服务端口零解析、未指定地址绑定(`0.0.0.0:443`
|
||||
匹配任意本地地址)、解析失败/空答案放行、决策缓存;`cmd/sower` 侧钉住 `selfProxyAddrs`
|
||||
的集合,以及“拒绝回 403 而不是静默关闭”。
|
||||
- 现场(r6c,v1.7.7 release 资产 `sower-linux-arm64.tar.gz`,sha256 `8cf0a869…`、
|
||||
`vcs.modified=false`):自指 `curl -x http://100.64.1.1:80 http://sower.wweir.fun/`
|
||||
→ `403 Forbidden`;`--socks5-hostname` 自指 → SOCKS5 码 `2`;正常路径 `--socks5-hostname`
|
||||
到 google `204`、`-x http://100.64.1.1:80` 到 baidu `200`、`/api/config` `401`;kr 上 sowerd
|
||||
到 `100.64.1.1` 的连接数为 `0`(修复前峰值 601)。
|
||||
|
||||
## 现场验证与一次回归(v1.7.4 → v1.7.5)
|
||||
### 一次必须记住的回归(v1.7.4 → v1.7.5)
|
||||
|
||||
首版守卫(`v1.7.4`,commit `2de08e6`)用系统解析器做自指判定,部署到 r6c 后现场验证立刻
|
||||
暴露反例:
|
||||
|
||||
- `curl --socks5-hostname 100.64.1.1:1080 https://www.google.com/generate_204` 返回
|
||||
SOCKS5 码 `2`(not allowed)而不是 204;r6c 的 `logread` 给出根因——
|
||||
`target is this client's own listener: sower.wweir.fun:80 resolves to 100.64.1.1:80`。
|
||||
- 同时 `nslookup www.google.com 127.0.0.1` 在 r6c 上返回 `100.64.1.1`:本机 DNS 对被代理
|
||||
域名回答的就是自己的 serve IP。于是“解析结果是我的地址”对**所有正常被代理域名**都成立,
|
||||
守卫把整条正常代理路径拒了。这是判定依据错误,不是实现细节错误。
|
||||
|
||||
处理:立即把 r6c 回滚到上一个可用二进制(`/usr/sbin/sower.bak-20261012195538`,sha256
|
||||
`76459b95…`,procd 重启后 pid 变化确认),随后改成向上游解析器求证(`v1.7.5`)并重新部署。
|
||||
|
||||
部署坑(已记入 deploy skill):跨机传二进制用 `cat > /tmp/sower.new` 得到的是 `0644`,
|
||||
`cp -a` 会原样保留该权限,procd 因为不可执行而根本起不来(`/usr/sbin/sower: Permission
|
||||
Denied`),表现为服务静默 down、LAN DNS 连锁失效。替换后必须 `chmod 755` 并用
|
||||
`sha256sum` + `pidof` 验证;同时 r6c 的 SSH 在 sower down 时可能连 `home` 跳板的域名解析
|
||||
都失败,此时改用 IP(`wweir@192.168.1.2` → `root@192.168.1.1`)。
|
||||
首版守卫(`v1.7.4`,commit `2de08e6`)用系统解析器判定,现场立刻被推翻:`--socks5-hostname`
|
||||
到 google 返回 SOCKS5 码 `2`,而 `nslookup www.google.com 127.0.0.1` 在 r6c 上返回
|
||||
`100.64.1.1` —— 本机 DNS 按设计对每个被代理域名都回答自己的 serve IP。于是“解析结果是我的
|
||||
地址”对**所有正常被代理域名**都成立,守卫把整条正常代理路径拒了:**这是判定依据错,不是
|
||||
实现细节错**。处理是把 r6c 回滚到上一版可用二进制,改成向上游解析器求证后以 `v1.7.5` 重发
|
||||
(部署期间的权限位与 LAN DNS 连锁坑见 skill `sower-deploy` 的 Pitfalls)。
|
||||
|
||||
## 未决(不在本次范围)
|
||||
|
||||
|
||||
@@ -10,18 +10,13 @@
|
||||
TLS 之下,为每次写入武装 `writeStallTimeout`;`dumpHeapOnSignal`(`heapdump_unix.go`,
|
||||
其他平台 no-op)在 SIGUSR1 时写 heap profile;`GOMEMLIMIT=512MiB` 进入安装器 unit、
|
||||
打包 unit 与 kr drop-in。
|
||||
- 发布:`v1.7.2` 的 Release run 在 windows-amd64/arm64 上失败(`syscall.SIGUSR1` 在
|
||||
Windows 未定义),Attach release assets 被跳过、没有发布任何资产;`06c357c` 把信号
|
||||
处理挪到 `//go:build unix` 文件后以 `v1.7.3` 重新发布(12 个平台构建 + Attach assets 全绿)。
|
||||
- kr 运行的是 **v1.7.3 release 资产**:tarball `sower-linux-amd64.tar.gz` 经
|
||||
`SHA256SUMS` 校验,`sowerd` sha256 `74b34fd9119d45d0d8de2013656a292ed2ffa08ec7e6a2dff6cac9262fd6d428`,
|
||||
`go version -m` 显示 `mod v1.7.3`、`vcs.revision=06c357c`、`vcs.modified=false`;
|
||||
备份 `/usr/local/bin/sowerd.bak-20261012191818`(上一版为本地构建 v1.7.1-4-g28c4f9e,
|
||||
备份 `sowerd.bak-20261012175623`)。
|
||||
- 验证:`:80` 仍 308 跳转;`:443` 回退反代 200;SIGUSR1 产出
|
||||
`/var/cache/sower/sowerd-heap-*.pprof`;用 `transport/sower` 从本机经 kr 中继
|
||||
`www.baidu.com:80` 得到 HTTP/1.0 200(证明新增的写超时不误伤中继);r6c 客户端
|
||||
socks5 到 google 仍 204,其日志自 17:31 起无握手失败;`GOMEMLIMIT=512MiB` 在进程中可见。
|
||||
- 已发布 `v1.7.3`(`v1.7.2` 因 Windows 上 `syscall.SIGUSR1` 未定义而未发布);kr 运行的是
|
||||
该 release 资产的 `sowerd`。
|
||||
- OOM 的**驱动**随后被定位为自指代理递归并在客户端修复(`v1.7.5`/`v1.7.6`,见
|
||||
[2026-10-12-self-proxy-target-guard.md](2026-10-12-self-proxy-target-guard.md));本文件的
|
||||
连接上界是下限保护,不是根因修复。
|
||||
- 验证:`:80` 308 跳转;`:443` 回退反代 200;SIGUSR1 产出 heap profile;`transport/sower`
|
||||
经 kr 中继 `www.baidu.com:80` 得到 HTTP/1.0 200(新增写超时不误伤中继)。
|
||||
|
||||
## 事件
|
||||
|
||||
@@ -78,23 +73,18 @@ kr 的 zram swap 因 `vm.swappiness=0` 从未被使用。
|
||||
(`systemctl kill -s USR1 sowerd`,`go tool pprof -http :0 <file>`),不新增网络面。
|
||||
4. 部署侧:sowerd 单元加 `Environment=GOMEMLIMIT=512MiB`,让失控堆退化服务而不是
|
||||
杀死宿主机。
|
||||
5. 发布工具链:v1.7.1 是唯一用 go1.26.8 构建的制品(现场其余二进制为
|
||||
go1.27.1/1.27.2),而 go1.26 的 Green Tea GC 有公开的内存回归报告
|
||||
(golang/go#77567、#77361)。CI 的 `go-version: ^1.26` 收敛到 `^1.27`。
|
||||
5. 发布工具链:CI 的 `go-version: ^1.26` 收敛到 `^1.27`,与本地/现场构建一致。当时曾怀疑
|
||||
go1.26.8 的 GC 回归(golang/go#77567、#77361)是原因,后证明是自指递归;保留这个改动只是
|
||||
让制品构建环境不再漂移。
|
||||
|
||||
## 未决
|
||||
|
||||
- **驱动已由另一会话定位(2026-10-12 18:44–18:50,board 消息 `m-77507977-64f`)**:任何
|
||||
指名 `sower.wweir.fun` 的代理请求会形成自持链——该名字的公网 A 记录指向 r6c 的
|
||||
tailscale 地址 `100.64.1.1`(r6c 上 sower 监听 80/443/53/1080),`cmd/sower` 无条件下
|
||||
发给 kr,kr 解析同名后回dial 进 r6c:80,再转发,逐跳生成下一跳。实测 r6c 上
|
||||
`100.64.1.1:80` 连接数 155→254→305→495→601(对端全是 kr),且触发它的 curl 退出后
|
||||
仍在增长;kr 上 sowerd 同期 311 个 dial、649 个 fd。这与本次 OOM 的窗口、双向近似
|
||||
对称流量、静默增长、以及“sowerd 一死链就断”全部吻合;18:50 重启 r6c 的 sower 后两侧
|
||||
都回到 0 连接。sowerd 侧的 cap 与写超时只把它**限住**(链在 2048 处停止增长、内存有界),
|
||||
真正的修法是客户端拒绝把请求 dial 回自己的监听地址(另一会话已开高优先级 finding)。
|
||||
触发链是否就是 17:30 那次膨胀的唯一成因仍无直接证据(当时进程已死、无 pprof、sar 缺样),
|
||||
因此上界与 SIGUSR1 dump 的价值是把下一次复现变成可定位事件,而不是宣称已修复根因。
|
||||
- **驱动已定位并修复(见姊妹文件)**:任何指名 `sower.wweir.fun`(公网 A 记录 = r6c 的
|
||||
tailscale 地址 `100.64.1.1`)的代理请求会形成自持链:实测 r6c 上 `100.64.1.1:80` 连接数
|
||||
155→601、kr 上 sowerd 311 个 dial / 649 个 fd,且触发请求退出后仍增长;客户端守卫已在
|
||||
`v1.7.5`/`v1.7.6` 拒绝这类自指拨号。本文件的连接上界因此是下限保护(链在 2048 处停止
|
||||
增长、内存有界);该链是否就是 17:30 那次膨胀的唯一成因仍无直接证据(进程已死、无 pprof、
|
||||
sar 缺样),SIGUSR1 dump 的价值是把下一次复现变成可定位事件。
|
||||
- `relay.Relay` 的写超时缺陷属于 `sower-proxy/conns` 仓库,应在那里修(finding
|
||||
`f-1ae16668-b92`)。sowerd 侧只做上界,不给隧道加全局空闲超时,避免误杀合法但长时间
|
||||
空闲的隧道。
|
||||
|
||||
+4
-5
@@ -231,11 +231,10 @@ bounded cost and a bounded lifetime. These bounds are normative:
|
||||
read with `go tool pprof`). It is the only way to attribute a grown heap to
|
||||
an allocation site, and unlike an HTTP endpoint it adds no network surface.
|
||||
|
||||
The service unit also carries a soft heap ceiling (`GOMEMLIMIT`), so an
|
||||
unbounded heap degrades this service instead of OOM-killing the host: the
|
||||
2026-10-12 OOM on `kr` (970 MiB anon-rss on a 1.6 GiB host) killed the machine,
|
||||
not the service. Historical analysis and the still-unproven trigger live in `docs/plans/`
|
||||
(2026-10-12, sowerd connection bounds; non-authoritative).
|
||||
The service unit also carries a soft heap ceiling (`GOMEMLIMIT`): a heap that
|
||||
grows without bound must degrade this service, not take the host down with it
|
||||
(the 2026-10-12 OOM on `kr` reached 970 MiB anon-rss on a 1.6 GiB host).
|
||||
Motivation and the historical analysis live in `docs/plans/` (non-authoritative).
|
||||
|
||||
## Rejected Directions
|
||||
|
||||
|
||||
+14
-15
@@ -222,15 +222,6 @@ func (r *Router) Dial(network, domain string, port uint16) (net.Conn, error) {
|
||||
|
||||
func (r *Router) DialSmart(network, domain string, port uint16) (net.Conn, error) {
|
||||
ctx := context.Background()
|
||||
addr := net.JoinHostPort(domain, strconv.FormatUint(uint64(port), 10))
|
||||
|
||||
// Guarded here as well as in DialProxyOnly: this path reaches directDial
|
||||
// without passing it (rule-based and probe-based direct decisions), and
|
||||
// dialing a listener this client itself forwards on re-enters the proxy
|
||||
// either way — locally, in that case, since no tunnel is involved.
|
||||
if ip, self := r.selfTarget(domain, port); self {
|
||||
return nil, selfProxyError(domain, port, ip)
|
||||
}
|
||||
|
||||
// 1. rule_based( block > direct > proxy )
|
||||
// 2. detect_based( CN IP || access site )
|
||||
@@ -243,14 +234,14 @@ func (r *Router) DialSmart(network, domain string, port uint16) (net.Conn, error
|
||||
case r.DirectRule.Match(domain):
|
||||
r.observe(RouteDirect, domain)
|
||||
r.observeRuleHit(RouteDirect, domain)
|
||||
return r.directDial(ctx, network, addr)
|
||||
return r.directDial(ctx, network, domain, port)
|
||||
case r.ProxyRule.Match(domain):
|
||||
r.observeRuleHit(RouteProxy, domain)
|
||||
return r.DialProxyOnly(network, domain, port)
|
||||
case r.localSite(ctx, domain), r.isAccess(domain, port):
|
||||
r.observe(RouteDirect, domain)
|
||||
r.observeRuleMiss(domain)
|
||||
return r.directDial(ctx, network, addr)
|
||||
return r.directDial(ctx, network, domain, port)
|
||||
default:
|
||||
r.observeRuleMiss(domain)
|
||||
return r.DialProxyOnly(network, domain, port)
|
||||
@@ -261,9 +252,8 @@ func (r *Router) DialProxyOnly(network, domain string, port uint16) (net.Conn, e
|
||||
if r.ProxyDial == nil {
|
||||
return nil, fmt.Errorf("proxy dialer unavailable")
|
||||
}
|
||||
// Guards every path that reaches the tunnel, explicit proxy requests and
|
||||
// auto-routed traffic alike: a target that resolves to a listener this
|
||||
// client forwards on would be forwarded again, forever.
|
||||
// A target that resolves to a listener this client forwards on would be
|
||||
// forwarded again, forever: the tunnel is one of the two dial exits.
|
||||
if ip, self := r.selfTarget(domain, port); self {
|
||||
return nil, selfProxyError(domain, port, ip)
|
||||
}
|
||||
@@ -276,7 +266,16 @@ func (r *Router) DialProxyOnly(network, domain string, port uint16) (net.Conn, e
|
||||
return rc, nil
|
||||
}
|
||||
|
||||
func (r *Router) directDial(ctx context.Context, network, addr string) (net.Conn, error) {
|
||||
// directDial is the other way into this client's own listeners: a rule or an
|
||||
// access probe can classify a name that points at one of them as direct, and
|
||||
// that connection re-enters the proxy just the same — locally, since no tunnel
|
||||
// is involved. Both dial exits are therefore guarded, not just the tunnel.
|
||||
func (r *Router) directDial(ctx context.Context, network, domain string, port uint16) (net.Conn, error) {
|
||||
if ip, self := r.selfTarget(domain, port); self {
|
||||
return nil, selfProxyError(domain, port, ip)
|
||||
}
|
||||
|
||||
addr := net.JoinHostPort(domain, strconv.FormatUint(uint64(port), 10))
|
||||
start := time.Now()
|
||||
dialer := net.Dialer{Timeout: 5 * time.Second}
|
||||
conn, err := dialer.DialContext(ctx, network, addr)
|
||||
|
||||
+1
-3
@@ -16,9 +16,7 @@ const (
|
||||
// self-target check. A proxied dial must not pay a DNS round trip per
|
||||
// connection, and a name that points at this client keeps pointing at it
|
||||
// for far longer than this window.
|
||||
selfProxyLookupTTL = time.Minute
|
||||
// selfProxyLookupTimeout matches localSite's resolution budget.
|
||||
selfProxyLookupTimeout = 2 * time.Second
|
||||
selfProxyLookupTTL = time.Minute
|
||||
selfProxyCacheMaxNames = 1024
|
||||
)
|
||||
|
||||
|
||||
+12
-18
@@ -100,27 +100,21 @@ func TestDialProxyOnlyRefusesUnspecifiedBind(t *testing.T) {
|
||||
}
|
||||
}
|
||||
|
||||
// A resolution failure says nothing about self-targeting: the dial proceeds so
|
||||
// the relay reports its own resolution result.
|
||||
func TestDialProxyOnlyAllowsTargetThatFailsToResolve(t *testing.T) {
|
||||
// Neither a resolution failure nor an empty answer is evidence of
|
||||
// self-targeting: the dial proceeds so the relay reports its own result, and
|
||||
// the empty decision is cached so a failing lookup is not paid per connection.
|
||||
func TestDialProxyOnlyAllowsTargetWithoutAnAnswer(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
r := newSelfTargetRouter(t, []string{"100.64.1.1:80"}, func(string) ([]net.IP, error) {
|
||||
return nil, errors.New("no such host")
|
||||
})
|
||||
if _, err := r.DialProxyOnly("tcp", "missing.example.com", 80); err != nil {
|
||||
t.Fatalf("dial after a resolution failure: %v", err)
|
||||
resolvers := map[string]func(string) ([]net.IP, error){
|
||||
"resolution failure": func(string) ([]net.IP, error) { return nil, errors.New("no such host") },
|
||||
"empty answer": func(string) ([]net.IP, error) { return nil, nil },
|
||||
}
|
||||
}
|
||||
|
||||
func TestDialProxyOnlyAllowsTargetWithNoAnswer(t *testing.T) {
|
||||
t.Parallel()
|
||||
|
||||
r := newSelfTargetRouter(t, []string{"100.64.1.1:80"}, func(string) ([]net.IP, error) {
|
||||
return nil, nil
|
||||
})
|
||||
if _, err := r.DialProxyOnly("tcp", "nodata.example.com", 80); err != nil {
|
||||
t.Fatalf("dial after an empty answer: %v", err)
|
||||
for name, resolve := range resolvers {
|
||||
r := newSelfTargetRouter(t, []string{"100.64.1.1:80"}, resolve)
|
||||
if _, err := r.DialProxyOnly("tcp", "missing.example.com", 80); err != nil {
|
||||
t.Errorf("%s: dial = %v, want it to proceed", name, err)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
Reference in New Issue
Block a user