A tun client registers its device's addresses with the far side's tun
server by sending the handshake frame. That frame is what tells the server
which addresses this device holds, and it is the only thing that can: a
server routes by destination address, so without it the link is one-way —
traffic leaves and nothing comes back.
keepalive() has always sent it unconditionally and returned early when no
period was configured, so the period only ever decided whether it
repeated. The call site did not: it was gated on the udp network or a
configured period, which on a p2p link ("ip") meant no period, no
handshake, no registration at all. That is the shape a p2p hub needs
least — its route table has no TTL to refresh, so the repeat buys nothing
there and the one-shot is the whole of it.
The gate is gone; the handshake always runs. The period keeps its meaning
where it has one: a server reached over UDP is connectionless, so silence
is the only signal that a peer has left, and the repeat is what lets its
route expire.
Found by a user asking why the keepalive switch was needed at all. It was
not, on a p2p hub — and turning it off silently produced the one-way link
the entrypoint's own comment warns about.
gost#911: two concurrent UDP source sockets over one RUDP reverse tunnel
over relay+TLS corrupted echo responses and made the host tear the tunnel
down and rebind it every second.
The tunnel is a single stream shared by every endpoint multiplexed over it.
A datagram frame is not written to that stream atomically —
UDPDatagram.WriteTo issues three separate Write calls (RSV/FRAG, SOCKS5
address, payload). Each Write is concurrency-safe on its own, but with
several endpoints writing, frames interleave between those calls. A torn
frame also leaves half a header in the stream, which the far end reads as
io.ErrUnexpectedEOF and answers by rebuilding the whole tunnel.
Hold a mutex for the whole frame instead of just the writes the datagram
happens to make. Same defect in both udpTunConn copies.
Reproduced with the reporter's script: relay+tls concurrent phase went from
44% loss with corrupted payloads and 1.0s gaps to 0.02% loss, no corruption,
0.003s gaps. Plain relay likewise. Single-endpoint high-rate loss (~35% at
14k pps) is unaffected UDP backpressure, not this bug.
p2p.go was a complete engine with nothing driving it: no handler.Handler,
so no GOST service could reach the hub and a spoke's registration frame had
no path to one at all.
NewP2PHandler takes the tun device conn and an auther, builds the peerTable
(no TTL — a stream close says a peer is gone outright, where a socket peer
has only silence to guess from), the peerRouter and the p2pHub, and starts
the device reader. Handle installs one accepted stream under its remote
address — the peer key the p2p host stamped on it, replacing whatever a
reconnect left behind — and feeds its datagrams to the hub until the read
fails. Teardown is peerGone and nothing else: it reclaims a peer's routes
only while that stream still holds its key, so a stale stream ending cannot
cut off the live connection its peer already replaced it with.
ownNets comes off the device conn's context, the two steps server.go's Handle
already uses, because a nil answer silently disables the self-route guard.
Handle returns the read error unwrapped. io.EOF, net.ErrClosed and
context.Canceled are what isServerError calls an ordinary end, so a peer
reconnecting would otherwise land in someone's error log.
The p2p half of the compat suite now runs against the real handler and the
real hub rather than reporting itself unavailable: the same six assertions,
the same two cases skipped without an auther. A second p2p case covers the
reconnect, which is the only way the suite would notice a peerGone that
dropped routes unconditionally — a hub that did so passes every shared case.
The socket server and the p2p hub are two answers to the same question, and
the design rests on a spoke being unable to tell which one it reached: the
same frame, the same magic, the same passphrase, the same registered
addresses, the same reply it refreshes its read deadline from. A spoke
configured for one is supposed to work against the other.
That claim is only worth something if the *same* assertions run against
both. Two separately-written suites would be allowed to drift, and nobody
would notice until someone deployed one against the other — which is a
field failure with nothing in any log to explain it. So the assertions are
written once here, against a compatHub that names no transport, and each
case is run once per implementation. The duplication that is left is
structural: how to build a hub and how it delivers, which is the part that
genuinely differs.
The socket half drives server.go's own transportServer over a real UDP link
rather than a copy of its loops; a compat suite that asserted against a
reimplementation would only prove the reimplementation agreed with itself.
The p2p half skips, and says why: nothing yet turns an inbound gRPC Tunnel
stream into a peerStream and feeds it to p2pHub.fromSpoke, so a spoke's
registration frame has no path to that hub. The skip is visible and the
six shared cases are named in it, because an absent assertion is not a
skipped one.
Three deliberate divergences turn the suite red, each from a real behavior
rather than a stub: registering a route for a frame the auther refused,
delivering to whichever peer spoke last, and echoing nothing after a
registration that took.
The route table answers a destination with a name, and the socket server
turns that name into a UDP address. A p2p hub needs the other answer: the
name is a peer key, and the thing it resolves to is a stream, not an
address. That is the whole of the second delivery — resolveUDPAddr is
unreachable here by construction, and putting it back on this path would be
a design error rather than a bug.
The shape is forced by the device, not chosen. tunDevice shares one read
buffer and one write buffer across every call and takes no lock: two
readers would have the second overwrite the first's target mid-call, and two
writers would copy through the same write buffer at once. It is safe today
only because the socket server happens to run one goroutine per side. A p2p
hub runs one per peer, so it has to reach the same guarantee a different
way: run() is the only goroutine that reads, and wmu is the only way to
write. This is the socket server's shape arrived at from the other side —
not a bridge per stream. Which goroutine read a packet carries no meaning,
because the route is what says where it goes.
Two locks, for two different hazards. The device's wmu guards the shared
device buffer. peerStream's own mu guards one peer's stream, whose conn
frames every write onto a byte stream and serializes only its read buffer:
two writers would put one frame's length prefix against the other's
payload, and every frame length would then be read from the wrong offset.
The -race run is what proves both are doing their job.
withdraw removes a stream only while it still holds its key. A peer that
reconnected already installed a replacement under that name, so reclaiming
routes for it on the old stream's teardown would cut off the live
connection — which is what makes a flapping peer unreachable rather than
merely briefly absent.
The keepalive is answered on the stream it arrived on, because the table
registers but never writes. Drop the echo and every keepalive:true spoke
misses the reply its read deadline is refreshed from and redials on its own
timeout, with nothing in any log: client.go's registration gate tests for
"udp" and a p2p link is "ip", so a spoke that never registers is already
indistinguishable from silence. Hence the warning naming the destination.
The concurrency tests are mutation-checked: with the device lock removed the
first fails on overlapping writes in round 0, and with the stream lock
removed the second fails on a spliced frame. Both were caught by the
assertions, not only by the race detector — the first harness was a
datagram pipe, which is atomic per message and so passed with the lock gone
until it was replaced by a byte stream, which is what the p2p conn really
is.
Its keepalive handling, route lookups and both delivery loops move onto
peerTable unchanged: the same magic-header reply, the same gateway
fallback, the same discard for an unrouted packet. Only the route's
value is now a name, resolved at the point of delivery. A test that
needed updating to pass would mean the socket path moved when it must
not have, so the line-by-line diff is the evidence here — no test in
this package exercises server.go.
The route table's reverse index is gone. It was a second copy of a fact
already in the route's name, and two maps cannot be made consistent:
concurrent set calls could leave the index naming one peer while the
route belonged to another, and dropPeer would then delete the winner's
route on the loser's teardown. dropPeer now scans the routes themselves
and deletes only entries still owned by the peer it is dropping.
A lock would have been the wrong fix: lookup runs per delivered packet
and dropPeer once per stream close, so the cost would have landed on the
hot path to settle a race that only matters at reclamation.
The keepalive protocol, the auther check, the TTL and destination lookup
are the same whichever way datagrams arrive, so they belong together and
know nothing about transport. A route's value becomes a string: a UDP
address for the socket server, a peer key for the p2p hub. net.Addr fit
neither — it is an endpoint, and one of these is not.
dropPeer is the p2p half of reclamation. The socket server expires a
route by TTL, guessing from silence that a peer is gone; a stream close
says so exactly, so the hub drops a peer's routes when its stream ends
and keeps no timer.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
streamConn embeds the session's net.Conn and overrides Read/Write/Close,
but not the deadline methods, so those reached the connection every
stream shares. Any deadline set by one stream then outlived it and
decided when the others could read.
net/http makes this reachable: tearing down a connection parks a
pending read with SetReadDeadline(time.Unix(1, 0)) and clears it only
when a read was actually in flight. Without a read in flight the shared
conn is left with a deadline in 1970, and the next Accept on the
session fails immediately with an i/o timeout — reported as a listener
error even though the session was healthy.
Delegate the three deadline methods to the stream, which has always had
its own. The test drives one request end to end over a stream, applies
that teardown deadline, and asserts the session still hands out the
next stream.
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
A remote listener (rtcp/runix/rudp) wraps an accept failure in core's
AcceptError — which always answers Temporary() — to say "rebuild the transport
and try again", and a closed pipe under it is one of the failures that recovers
that way, so the retry stays exactly as it was. What was wrong is what it said: a
stop arrives down the same path, and a warning that promises a retry reads like a
hiccup in the log of a service that is simply being stopped. The retry is logged
at debug when the error means a closed listener, pipe or socket; every other
failure still warns, and the tail still logs real failures at error.
No behaviour change: the retry, its backoff and the state transitions are
untouched — the classification decides only the log level.
Stopping a service closes its listener, and a closed listener answers Accept
with core's listener.ErrClosed — which the guard never matched (it knew only
net.ErrClosed), so every stop reached the log at error level and read like a
crash. The tun handler logged the same shape a second time: its client wrapped a
closed device with %s, throwing the cause away, so the caller could not tell a
device closed under the reader from a real read failure. %w keeps the chain and
the handler logs the closed case at debug.
A configured output path without a rotation block is a plain file that nothing
rotates: it grows until the disk does something about it. The default logger —
the one an unconfigured program gets — always carries a rotation, so this only
happens to a hand-written config, and the parser is the only place that sees
both the path and the rotation that is missing. Reported on stderr, once, at
parse time; nothing about the behaviour changes.
ParseLogger reported the failure through logger.Default(), which is nil until a
caller installs one — and this call is usually what installs it. An unwritable
path therefore dereferenced a nil logger and panicked; inside a c-shared library
that aborts the process, which on Android took an app down a second or two after
start with nothing readable left behind (an app's stderr goes nowhere, and the
tombstone carries no abort text).
Measured with Output "file" — a path, not a mode — in an app whose working
directory is "/" and not writable.
The failure goes to stderr instead, which is always there, and the default
output stays in place. The regression test forks a child process, because a
fresh one is the only place the default logger is still unset: TestMain skips
installing its logger for that child.
An unprivileged Android app cannot open /dev/net/tun, so the device can only
come from a VpnService: the Java side creates and configures it (addresses,
routes, MTU, DNS via VpnService.Builder) and hands its fd over as the "fd"
metadata, either as a fixed descriptor or — for an embedder, which is what the
platform's own VPN is — as a provider resolved when each device is built, so a
device rebuilt after its session ended never copies a closed or recycled
descriptor.
The device is created with tun.CreateUnmonitoredTUNFromFD (no netlink listener,
no setMTU — the builder already set it), which makes the Linux variant, with
its netlink and resolvectl calls, inapplicable: it is now linux && !android.
The interface lookup in the accept loop was fatal on any error; it is
informational, and on Android the device may not be listed in this process.
Bump github.com/go-gost/plugin to v0.8.1, which reports transport stats
over the p2p Status RPC (direct/derp peer gauges plus punch/stream
counters), and pick up the matching go.sum entries.
SupportedDialer: also accept wss, grpc, http2, h2, ssh, sshd and mwss as
inner protocols on top of a p2p tunnel. The list stays fail-closed; each
entry is a dialer verified to dial one plain conn and use it as its base.
Node auth on the HTTP forwarder path read the origin credential
(Authorization) and challenged with 401/WWW-Authenticate. When the
forwarded backend also used Basic auth, the node consumed the credential
meant for the backend, so the two could not coexist.
Read the proxy credential (Proxy-Authorization) instead and challenge
with 407/Proxy-Authenticate. Authorization is left untouched so it
reaches the backend, and Proxy-Authorization is stripped before the
upstream write since it is hop-by-hop (RFC 9110 11.7.2).
Three fixes found while wiring the p2p hub topology:
- The client keepalive was gated on network == "udp", but on a p2p link
raddr is the peer's base64 key (no port), so SplitHostPort fails and
network is "ip": the keepalive never ran, a tun server never learned the
client's route, and the read deadline was never armed. Gate on the udp
network OR a configured keepalive period; the udp branch keeps the one-shot
registration handshake it always sent.
- Liveness was refreshed only by the keepalive echo; a transient channel
hiccup can drop that single packet and redial a healthy tunnel after
3xperiod. Refresh the read deadline after any successful read instead.
- Routes were never removed: a departed client's IP -> addr stayed forever
and the device kept blackholing to a dead address. Expire a route once its
last keepalive is older than 3x the configured period, checked lazily on
the write path (no sweeper). The TTL is opt-in: without a server-side
keepalive/ttl, routes keep their current never-expiring behavior.
The read deadline set for ClientHello sniffing was reset via defer, so it
remained on the client connection until Handle returned, i.e. for the whole
session. xnet.Pipe is called without a read timeout for TLS/HTTP, so nothing
refreshed it and any connection was torn down after readTimeout (15s default)
regardless of traffic. Clear the deadline immediately after sniffing, matching
the relay/router/tunnel/socks4 handlers.
Regression test asserts the deadline is zero when the upstream dial fires.
Fixes#904
The test fed all 715 chunks before the first Read; with the cap-1 pump
handoff and a 64-slot fake stream the feeder blocks long before the
reader starts — a deterministic deadlock the earlier "passing" runs
masked (a `go test | tail` pipeline hid the exit code). Feed from a
goroutine.
readDatagram kept appending into one backing array; an idle conn held
onto the largest frame's array indefinitely (bounded, but anchored until
a later append crossed capacity). Drain it to nil once a frame boundary
empties it, and assert the release plus the received-bytes-only bound in
the tests.
TunnelProvider becomes OpenTunnelStream: OpenTunnel authorizes the tunnel
and issues a single-use id, the Tunnel stream carries the data. The new
x/p2p/streamconn conn wraps the stream as a net.Conn with real
Set*Deadline support (bounded Recv pump, idempotent Close, framed mode
for udp). The tunnel base dialer collapses to the stream; its failure
path closes the conn nil-safely instead of leaking.
udp.Relay.Run ignored ctx cancellation, so an idle MASQUE association (or a
disconnecting client) leaked its target UDP socket and both copy goroutines,
and service shutdown blocked on the stuck handlers. quic-go's
ReceiveDatagram only unblocks on ctx cancellation, never on stream close, so
closing the datagram conn could not release the read either.
Make Relay.Run cancellation-aware and close both conns on exit, add
WithReadTimeout for an idle read deadline, give DatagramConn a cancelable
context that Close cancels, and apply the parsed idleTimeout to the UDP
relay.
Ref: https://github.com/go-gost/x/issues/128
Design and implementation notes for moving the tun-to-tun link out of the
p2p host: GOST owns the device, the host supplies a datagram tunnel. Records
how each review finding was closed and what the e2e run covered.
OpenTunnel carries a network: "udp" asks the plugin for a datagram endpoint
(one datagram per write) instead of a byte stream, which is what a tun
handler needs — a stream endpoint coalesces and splits packets. tcp stays
the default, so existing deployments are untouched.
The tun handler could not reach client mode without a service forwarder, so
a chain with no forwarder now means client mode with an empty dial address:
the link is transparent and the peer comes from the chain node (a p2p node's
addr is its peer key). That combination used to be dead config, so it is
logged. A forwarder still wins when present, and server mode is unchanged.
Alongside: the udp dialer no longer hard-asserts *net.UDPConn (a wrapped
base dialer yields a plain conn), udp joins the p2p dialer whitelist, and a
udp tunnel announces its client with an empty datagram right after dialing —
the endpoint learns where to send peer packets only from a packet it
receives, so without it a link whose peer speaks first would never come up.
Requires plugin v0.7.0 for the new field.
Add Command []string to PluginConfig and a new x/plugin package that
spawns plugin subprocesses (deduplicated by argv) and terminates them
on Shutdown. Linux uses Pdeathsig, Windows a kill-on-close job object,
so children follow an abrupt parent exit.
Wire Spawn into every config/parsing plugin constructor.
cipherConn.ReadFrom returned decrypt errors straight to quic-go, which
treats any packet-socket error as fatal and closes the shared QUIC
transport. A single malformed datagram (short, wrong key, tampered tag)
thus killed the whole listener while the process kept running.
Discard unauthenticated/invalid peer datagrams and keep reading; only
surface real socket errors and bad local AES keys. Also fix WriteTo to
return the caller-supplied length instead of the encrypted nonce+tag
length, per the net.PacketConn contract.
The tunnel is now opened inside the base dialer on first use: mux
inners (mtcp/mtls/mws) hit their session cache without touching the
base, so a cached session spends no OpenTunnel and leaks no tunnel
per dial. SupportedDialer grows to tcp/tls/ws/mtcp/mtls/mws (each
verified end to end; tls/ws handshakes trigger via the forwarded
Handshaker, mux inners reuse one tunnel as a session). Error paths
release the tunnel exactly once: base-dial failure resets the base
state, inner failure after a successful base dial is closed by the
wrapper, tunnelConn close clears the base state.
Plugin bumped to v0.6.1; QUIC evolution note (transparency principle,
no automatic inner substitution) recorded in the stub plan.
A session Dialed but not yet Handshaken has session.session == nil; a
concurrent Dial treated it as dead (IsClosed() is true for nil sessions),
deleted it and closed the base conn under the first caller's Handshake.
Re-initializing on the nil-session case self-heals instead of failing
the identity check.
Any dialer can now declare its base transport to run through a tunnel
opened by a named p2p plugin component: a top-level p2ps: section
registers TunnelProviders (gRPC plugin), and chain node metadata
"p2p: <name>" wraps the node's dialer so it dials the plugin's
OpenTunnel endpoint instead of the network directly. The inner dialer
keeps its protocol (tls/ws/mux) and Handshake, which the wrapper
forwards so it still triggers at Transport.Handshake.
- x/p2p: tunnelDialer (fail-closed, per-dial tunnel with idempotent
CloseTunnel on conn close), SupportedDialer whitelist (tcp only;
fail closed for unverified inner dialers), gRPC TunnelProvider
- config: P2PConfig, parsing, loader registration (reload-safe via
unregister+Close), mergeConfig wiring
- dialer wrapper registered before Init so the chain inits once
- pin github.com/go-gost/plugin v0.6.0 (p2p proto); dependency bumps
are a consequence of the new import
- inner dialer whitelist becomes prerequisite 3 (fail-closed, tcp only)
- record dead-session tunnel-recall fix and the known IsClosed race
- loopback default bind declared as the security boundary for the
unauthenticated control channel (active-dial SSRF surface)
- contract scope narrowed to TCP-semantic byte pipes; UDP/datagram
engines (hole punching) are an explicit future collision point
Dial() deleted a dead muxSession without closing session.conn, leaking
the base TCP conn on every dead session (and, once a p2p base conn is
injected, leaking the plugin-side tunnel). session.Close() is a no-op
when session.session is nil, so the base conn is closed explicitly.
The tun/tungo listeners never tore down the active device on Close, so
reloading (API config change or SIGHUP) with the same device name hit
"device or resource busy": the kernel kept the old interface because its
fd was still open, and the listen loop leaked devices created around
shutdown, spamming errors and blocking SIGTERM.
Track the accepted conn on the listener and close it (idempotently via
sync.Once folding cancel + fd close) from both the loop's shutdown branch
and Close(), with a send-race guard so a device created just as Close runs
is still released. Closing the fd removes the interface, so rebind with the
same name now succeeds and the process exits cleanly.
gopkg.in/yaml.v3 is archived/unmaintained upstream. Its only use in x is
Config.Write()'s YAML encoding, which is replaced with the already-present
goccy/go-yaml (compiled in via gin) — no new dependency. goccy output is
wire-equivalent (same keys/structure/omitempty); viper reads it back
identically, confirmed by the config round-trip tests. yaml.v3 drops to an
indirect dep (retained only by the inspector module in the workspace).
Drop the classic github.com/shadowsocks/go-shadowsocks2 dependency. tap and
internal/util/ss were its only remaining users after the ss/ssu handlers
moved to the go-gost fork. The fork's shadowaead is byte-identical on wire
(classic AEAD: same md5-KDF, HKDF-SHA1 subkey, salt sizes), so tap now uses
a small stateless ss.AEADPacketConn built on fork Pack/Unpack, and
ss.ShadowCipher becomes a fork builder (dummy/none -> existing none.Cipher).
Removes the undocumented tap handler key metadata and the dead ShadowConn
helper in the process.
Replace the sole govalidator.IsDNSName call in normalizeRequest with a local
isDNSName helper that mirrors the library's exact behavior (empty/size guard,
net.ParseIP exclusion, and the same label regex), removing the
asaskevich/govalidator dependency from x.
Replace the vulcand/predicate dependency in the routing rules parser with a
direct go/parser/go/ast implementation covering the same grammar (matcher
calls, parentheses, &&/||/!). This removes the predicate -> gravitational/
trace -> logrus logging chain from x, since trace was only used to build
errors. Public API (NewParser, Parser, TreeBuilder, Parse, CheckRule,
ParseMatchers) is unchanged and the existing test suite passes as-is.
Domain-only whitelist bypass never matched the bare destination IP, so the
pre-sniffing bypass check (fe394a8) rejected connections before SNI sniffing
could match the sniffed host (gost#899).
- skip the pre-sniffing check when bypass IsWhitelist(); the sniffer applies
the whitelist against the sniffed host
- restore the post-sniffing bypass check on dstAddr (removed by fe394a8) so
whitelist mode still rejects non-matching hosts on the sniffingFallback
path and when sniffing is disabled
- bypassGroup.IsWhitelist() now reports true only for all-whitelist groups
instead of always false (sole external consumer is the redirect handler)
- rewrite handler tests to drive a real TLS ClientHello through Init()'d
metadata: whitelisted SNI is forwarded, non-whitelisted SNI gets ErrBypass
Cacheable() now refuses X-Turbo-Short-Circuit responses (partial <meta>
payloads meant to merge client-side, would serve blank pages on later
hits). StoreKey() lets callers whose request identity is rewritten before
store (e.g. mirror Host routing) pin the key so Lookup and Store agree.
Bump github.com/go-gost/core to v0.6.1.
Implement the header counterpart of the existing body rewrite:
- rewriteHeaderBlock applies rewrite rules to an http.Header in place.
Regex mode matches header names case-insensitively (per HTTP spec) and
rewrites each value independently; an empty replacement drops the value,
and a fully-empty header is deleted. Plugin mode serializes the header
block (http.Header.Write + trailing CRLF, since ReadMIMEHeader needs the
blank line), sends it to a core/rewriter.Rewriter plugin, and parses the
result back with clear+copy (h is a map; reassignment would not escape).
- rewrite_metadata.go consolidates the hand-rolled plugin metadata maps
(baseMetadata + three SSE inline maps) into rewriteMeta() with common
sid/direction/uri fields and a new kind (body|header), additive for
existing plugins.
- sniffer_http.go wires rewriteReqHeader before the request is written
upstream (and refreshes the recorder header clone) and rewriteRespHeader
after static response-header overrides, before the recorder clone.
- hostPattern: derive req.Host from the request path via regex submatch
expansion (Host = $1-style template), for Host-routed mirror origins.
- parse: parseHeaderRewrites gates on the config string (regexp.Compile("")
yields a non-nil match-all, so gating on the compiled regexp would turn
an empty name into a silent no-op rule).
Fixes found by e2e: reqHeaderRewrites was assigned after its guard, so the
request header rewrite never ran; tests cover the header round-trip,
plugin gating, metadata kind, and hostPattern expansion.
ginuerzh