Design and implementation notes for moving the tun-to-tun link out of the
p2p host: GOST owns the device, the host supplies a datagram tunnel. Records
how each review finding was closed and what the e2e run covered.
OpenTunnel carries a network: "udp" asks the plugin for a datagram endpoint
(one datagram per write) instead of a byte stream, which is what a tun
handler needs — a stream endpoint coalesces and splits packets. tcp stays
the default, so existing deployments are untouched.
The tun handler could not reach client mode without a service forwarder, so
a chain with no forwarder now means client mode with an empty dial address:
the link is transparent and the peer comes from the chain node (a p2p node's
addr is its peer key). That combination used to be dead config, so it is
logged. A forwarder still wins when present, and server mode is unchanged.
Alongside: the udp dialer no longer hard-asserts *net.UDPConn (a wrapped
base dialer yields a plain conn), udp joins the p2p dialer whitelist, and a
udp tunnel announces its client with an empty datagram right after dialing —
the endpoint learns where to send peer packets only from a packet it
receives, so without it a link whose peer speaks first would never come up.
Requires plugin v0.7.0 for the new field.
Add Command []string to PluginConfig and a new x/plugin package that
spawns plugin subprocesses (deduplicated by argv) and terminates them
on Shutdown. Linux uses Pdeathsig, Windows a kill-on-close job object,
so children follow an abrupt parent exit.
Wire Spawn into every config/parsing plugin constructor.
cipherConn.ReadFrom returned decrypt errors straight to quic-go, which
treats any packet-socket error as fatal and closes the shared QUIC
transport. A single malformed datagram (short, wrong key, tampered tag)
thus killed the whole listener while the process kept running.
Discard unauthenticated/invalid peer datagrams and keep reading; only
surface real socket errors and bad local AES keys. Also fix WriteTo to
return the caller-supplied length instead of the encrypted nonce+tag
length, per the net.PacketConn contract.
The tunnel is now opened inside the base dialer on first use: mux
inners (mtcp/mtls/mws) hit their session cache without touching the
base, so a cached session spends no OpenTunnel and leaks no tunnel
per dial. SupportedDialer grows to tcp/tls/ws/mtcp/mtls/mws (each
verified end to end; tls/ws handshakes trigger via the forwarded
Handshaker, mux inners reuse one tunnel as a session). Error paths
release the tunnel exactly once: base-dial failure resets the base
state, inner failure after a successful base dial is closed by the
wrapper, tunnelConn close clears the base state.
Plugin bumped to v0.6.1; QUIC evolution note (transparency principle,
no automatic inner substitution) recorded in the stub plan.
A session Dialed but not yet Handshaken has session.session == nil; a
concurrent Dial treated it as dead (IsClosed() is true for nil sessions),
deleted it and closed the base conn under the first caller's Handshake.
Re-initializing on the nil-session case self-heals instead of failing
the identity check.
Any dialer can now declare its base transport to run through a tunnel
opened by a named p2p plugin component: a top-level p2ps: section
registers TunnelProviders (gRPC plugin), and chain node metadata
"p2p: <name>" wraps the node's dialer so it dials the plugin's
OpenTunnel endpoint instead of the network directly. The inner dialer
keeps its protocol (tls/ws/mux) and Handshake, which the wrapper
forwards so it still triggers at Transport.Handshake.
- x/p2p: tunnelDialer (fail-closed, per-dial tunnel with idempotent
CloseTunnel on conn close), SupportedDialer whitelist (tcp only;
fail closed for unverified inner dialers), gRPC TunnelProvider
- config: P2PConfig, parsing, loader registration (reload-safe via
unregister+Close), mergeConfig wiring
- dialer wrapper registered before Init so the chain inits once
- pin github.com/go-gost/plugin v0.6.0 (p2p proto); dependency bumps
are a consequence of the new import
- inner dialer whitelist becomes prerequisite 3 (fail-closed, tcp only)
- record dead-session tunnel-recall fix and the known IsClosed race
- loopback default bind declared as the security boundary for the
unauthenticated control channel (active-dial SSRF surface)
- contract scope narrowed to TCP-semantic byte pipes; UDP/datagram
engines (hole punching) are an explicit future collision point
Dial() deleted a dead muxSession without closing session.conn, leaking
the base TCP conn on every dead session (and, once a p2p base conn is
injected, leaking the plugin-side tunnel). session.Close() is a no-op
when session.session is nil, so the base conn is closed explicitly.
The tun/tungo listeners never tore down the active device on Close, so
reloading (API config change or SIGHUP) with the same device name hit
"device or resource busy": the kernel kept the old interface because its
fd was still open, and the listen loop leaked devices created around
shutdown, spamming errors and blocking SIGTERM.
Track the accepted conn on the listener and close it (idempotently via
sync.Once folding cancel + fd close) from both the loop's shutdown branch
and Close(), with a send-race guard so a device created just as Close runs
is still released. Closing the fd removes the interface, so rebind with the
same name now succeeds and the process exits cleanly.
gopkg.in/yaml.v3 is archived/unmaintained upstream. Its only use in x is
Config.Write()'s YAML encoding, which is replaced with the already-present
goccy/go-yaml (compiled in via gin) — no new dependency. goccy output is
wire-equivalent (same keys/structure/omitempty); viper reads it back
identically, confirmed by the config round-trip tests. yaml.v3 drops to an
indirect dep (retained only by the inspector module in the workspace).
Drop the classic github.com/shadowsocks/go-shadowsocks2 dependency. tap and
internal/util/ss were its only remaining users after the ss/ssu handlers
moved to the go-gost fork. The fork's shadowaead is byte-identical on wire
(classic AEAD: same md5-KDF, HKDF-SHA1 subkey, salt sizes), so tap now uses
a small stateless ss.AEADPacketConn built on fork Pack/Unpack, and
ss.ShadowCipher becomes a fork builder (dummy/none -> existing none.Cipher).
Removes the undocumented tap handler key metadata and the dead ShadowConn
helper in the process.
Replace the sole govalidator.IsDNSName call in normalizeRequest with a local
isDNSName helper that mirrors the library's exact behavior (empty/size guard,
net.ParseIP exclusion, and the same label regex), removing the
asaskevich/govalidator dependency from x.
Replace the vulcand/predicate dependency in the routing rules parser with a
direct go/parser/go/ast implementation covering the same grammar (matcher
calls, parentheses, &&/||/!). This removes the predicate -> gravitational/
trace -> logrus logging chain from x, since trace was only used to build
errors. Public API (NewParser, Parser, TreeBuilder, Parse, CheckRule,
ParseMatchers) is unchanged and the existing test suite passes as-is.
Domain-only whitelist bypass never matched the bare destination IP, so the
pre-sniffing bypass check (fe394a8) rejected connections before SNI sniffing
could match the sniffed host (gost#899).
- skip the pre-sniffing check when bypass IsWhitelist(); the sniffer applies
the whitelist against the sniffed host
- restore the post-sniffing bypass check on dstAddr (removed by fe394a8) so
whitelist mode still rejects non-matching hosts on the sniffingFallback
path and when sniffing is disabled
- bypassGroup.IsWhitelist() now reports true only for all-whitelist groups
instead of always false (sole external consumer is the redirect handler)
- rewrite handler tests to drive a real TLS ClientHello through Init()'d
metadata: whitelisted SNI is forwarded, non-whitelisted SNI gets ErrBypass
Cacheable() now refuses X-Turbo-Short-Circuit responses (partial <meta>
payloads meant to merge client-side, would serve blank pages on later
hits). StoreKey() lets callers whose request identity is rewritten before
store (e.g. mirror Host routing) pin the key so Lookup and Store agree.
Bump github.com/go-gost/core to v0.6.1.
Implement the header counterpart of the existing body rewrite:
- rewriteHeaderBlock applies rewrite rules to an http.Header in place.
Regex mode matches header names case-insensitively (per HTTP spec) and
rewrites each value independently; an empty replacement drops the value,
and a fully-empty header is deleted. Plugin mode serializes the header
block (http.Header.Write + trailing CRLF, since ReadMIMEHeader needs the
blank line), sends it to a core/rewriter.Rewriter plugin, and parses the
result back with clear+copy (h is a map; reassignment would not escape).
- rewrite_metadata.go consolidates the hand-rolled plugin metadata maps
(baseMetadata + three SSE inline maps) into rewriteMeta() with common
sid/direction/uri fields and a new kind (body|header), additive for
existing plugins.
- sniffer_http.go wires rewriteReqHeader before the request is written
upstream (and refreshes the recorder header clone) and rewriteRespHeader
after static response-header overrides, before the recorder clone.
- hostPattern: derive req.Host from the request path via regex submatch
expansion (Host = $1-style template), for Host-routed mirror origins.
- parse: parseHeaderRewrites gates on the config string (regexp.Compile("")
yields a non-nil match-all, so gating on the compiled regexp would turn
an empty name into a silent no-op rule).
Fixes found by e2e: reqHeaderRewrites was assigned after its guard, so the
request header rewrite never ran; tests cover the header round-trip,
plugin gating, metadata kind, and hostPattern expansion.
A valid Gost-Target / X-Gost-Target header replaced only req.Host while
leaving req.URL.Host untouched. Destination-policy checks (bypass) evaluated
the header-derived authority, but http.Transport dialed the unchanged
absolute-URL authority. A client could label a forbidden URL with an allowed
authority and reach the blocked destination (403 -> 200 with backend body).
setRequestHost now applies the decoded target to both req.Host and req.URL.Host
so policy evaluation and transport dialing always resolve to the same authority.
Origin-form requests ("GET / HTTP/1.1" with only a Host header, as
sent by nginx proxy_pass to an upstream proxy) failed RoundTrip with
"http: no Host in request URL": normalizeRequest infers URL.Scheme
but leaves URL.Host empty, and http.Transport requires a non-empty
URL.Host. Also affected every keep-alive request after the first,
which bypass normalizeRequest entirely.
Fill scheme+Host from req.Host in the shared proxyRoundTrip path,
covering both initial and subsequent requests.
e2e-verified: raw origin-form request proxied by Host header
(gost/tests/e2e TestHTTPProxyOriginForm).
filterToMatcherRule produced `Host(x) || Host(.x) && Proto(...) && ...`,
but the matcher DSL (vulcand/predicate) binds && tighter than ||, so the
apex Host(x) branch bypassed the protocol/path conjuncts — a request to
the apex with a non-matching protocol/path matched where the legacy
filter (AND of host/protocol/path) rejected it.
Also normalize `*.example.com` to `.example.com` before the leading-dot
branch: legacy ParseNode did this so both forms matched apex+subdomains;
without it, `*.example.com` reached Host() as a wildcard matching only
subdomains, dropping apex coverage that `.example.com` still had.
- Parenthesize the host OR when protocol/path conjuncts follow.
- Strip leading `*` + ensure leading dot, mirroring legacy normalization.
- Update TestFilterToMatcherRule expectations; add dot/star combined cases.
- Add TestFilterToMatcherRule_Precedence: runtime guard that apex+wrong
proto/path is rejected and apex/subdomain+valid is accepted, for both
`.example.com` and `*.example.com` forms.
The tunnel bind conns (bindConn/bindUDPConn) now expose their metadata (the
tunneled hostname) via the xctx.Context interface, so the service accept loop
forwards it into the handler context. handleRawForwarding selects a hop node by
that hostname when a hop is configured, so a node with a matching filter.host is
picked; otherwise it falls back to dialing the hostname directly. Fixes
reverse-proxy-tunnel TCP routing (gost#898) where the docs example failed with
"node not available" because the mux path never set the host metadata.
Fold the deprecated per-node filter (host/protocol/path) into an equivalent
matcher DSL rule during config parsing, so hop.Select keeps a single
filtering path (routing.Matcher). Legacy filter configs stay accepted.
- filterToMatcherRule() maps host->Host(), protocol->Proto(), path->PathPrefix();
a leading-dot host emits `Host(x) || Host(.x)` to preserve the legacy
apex+subdomain coverage. Converted rules get priority=-1 to keep the
always-through-selector behavior of the legacy path.
- Remove isEligible/checkHost/checkProtocol/checkPath from x/hop.
core/ chain.NodeFilterSettings is left untouched (dead but harmless).
- Migrate hop tests to matchers; add TestFilterToMatcherRule.
Since 486a7766 HandleHTTP records on an internal clone of the passed
recorder object, TestHandleHTTP_BasicProxy's assertions on the original
ro were always empty, failing every run. Assert SrcAddr/DstAddr on the
captured recorder payload instead, and drain the response body before
closing the net.Pipe so HandleHTTP completes the write and exits cleanly.
Add a json-: match prefix to http.rewriteRequestBody/rewriteResponseBody
that deletes the matched JSON field (sjson.DeleteBytes) instead of
replacing it, keeping the existing value-regex conditional semantics.
Replace the 5 duplicated switch statements that dispatch sniffed
connections (forward/local, forward/remote, http CONNECT, sni, relay)
with per-package registries keyed by proto string. Protocol handlers
self-register via init(), following the house registration pattern, so
adding a protocol no longer requires touching every dispatch site.
Unify the two HandleOptions types (forwarder lacked dialTLS; sniffing
lacked hop/node/httpKeepalive) into one exported struct in sniffing;
forwarder aliases it. Unknown protocols fall through to raw forwarding
exactly as before; SSH remains detection-only. redirect/tcp's special
fallback-replay dispatch is intentionally left untouched.
Adds Dispatch helpers and 8 registry tests (register/get/duplicate/
unknown/known) in both packages. Net -151 lines.
POST /config without an explicit path= always wrote gost.yaml/gost.json to
the process CWD, silently leaving the config loaded via -C untouched. Track
the last file read by ReadFile and default the save target to it, deriving
the output format from the file extension so a JSON config is saved back as
JSON. Falls back to the CWD default when no config file was loaded.
WrapConn stores the target and only writes it on first I/O, but the
connector encoded it into a bufpool buffer that was returned to the pool
when Connect returned. A later Connect could reuse the backing array and
overwrite the bytes, misrouting the first connection. Allocate the target
per connection instead of sharing a pooled buffer.
Regression test drives two Connect calls and asserts the first conn's
wire target survives a second Connect reusing the pool.
Failed nodes (http.failCodes marker) were still selected when the
priority short-circuit fired or when FailFilter saw a single-node tier,
so failover never happened for a high-priority matcher node returning
429. Check IsFailed first in the candidate-pool scan, and when every
candidate is failed return the first node matching the request gates as
a best-effort last resort instead of nil.
Refactors the fail-code check into shared selector.IsFailed (cee264e)
wired through FailFilterSettingsOption (64ea518).
Extract the per-item failed-state decision (failure count >= maxFails,
last failure within failTimeout, metadata overrides) from FailFilter
into an exported IsFailed helper, and reuse it in the hop's
pre-selection dead-node filter instead of a local duplicate.
http.failCodes (and probe) marking had no effect when the failing node
had a strictly higher matcher priority: the priority short-circuit
returned the top node directly, and FailFilter no-ops on a single-node
tier (len(vs) <= 1 guard), so the marked node was always re-selected
and failover never happened.
Pre-filter nodes marked failed (same Count >= maxFails && within
failTimeout criteria as FailFilter) out of the candidate pool before
priority selection. Single-node hops keep their lone node (safety net),
and an all-failed pool returns nil as before.
Wired maxFails/failTimeout from the hop selector config into the hop
via FailFilterSettingsOption.
Pulls in the classic-AEAD UDP [addr][payload] response-framing fix that
restores shadowsocks UDP interop with third-party servers/clients
(go-shadowsocks2, shadowsocks-libev). Fixesgo-gost/gost#894.
slog.Logger.With does not deduplicate keys, so chained WithFields calls
(e.g. service -> listener/handler, each setting "kind") emitted duplicate
JSON keys. slogLogger now rebuilds from a pristine base logger with
accumulated attrs deduplicated by key, last write wins.
The MASQUE handler's CONNECT-UDP path was broken in two ways, so no
standards-compliant client (e.g. Surge) could establish UDP tunnels:
1. Missing Capsule-Protocol response header. HTTPStream() flushes the
response internally and commits the headers, but it was called before
Header().Set("Capsule-Protocol", "?1"), so the mandatory RFC 9298
header was silently dropped and clients rejected the tunnel.
2. WriteTo on a pre-connected socket. A direct UDP dial via the router
returns a *net.UDPConn that satisfies net.PacketConn but is
pre-connected; the relay called WriteTo on it, which fails with "use
of WriteTo with pre-connected connection", so no packet reached the
target.
Set the Capsule-Protocol header before hijacking the stream (dialing the
target first, matching handleConnectTCP), and always wrap the
router-returned conn so writes go through Conn.Write. Verified with a
standards-compliant HTTP/3 MASQUE client: CONNECT-UDP now returns
Capsule-Protocol: ?1 and datagrams relay end to end; CONNECT-TCP
unaffected.
Commit b3b5986 crammed the URL path into the listen/connect address and
disabled forward mode for path-based protocols, breaking the
`unix://a.sock/b.sock` (listen on a, forward to b) port-forward form and
producing an invalid `a.sockb.sock` address.
Distinguish the two URL forms by host presence:
- three-slash (unix:///abs.sock, empty host) → path is the absolute socket
address, no forward target;
- two-slash (unix://a.sock/b.sock) → host is the socket and the path is a
forward target (service) or ignored (node).
Add regression tests for both forms.
Co-Authored-By: Claude (claude-opus-4-8[1m])
Path-based address selection used the dialer/listener type after registry
fallback, so unix remapped to tcp skipped EscapedPath and dropped
addrs like /var/run/docker.sock from three-slash URLs.
Decide path-based protocols from the original scheme components.
Signed-off-by: Dean Chen <862469039@qq.com>
websocketConn.Read called Conn.ReadMessage, which is NextReader plus
io.ReadAll. io.ReadAll starts at 512 bytes and grows by doubling, so every
frame allocated a fresh slice and a chain of intermediate ones: a stream of
64KB frames costs about 2.5 bytes of garbage per byte carried, even though
the caller already supplied a buffer to copy into.
On a relay+wss tunnel moving 750MB this was 2526KB of garbage per MB of
payload and 757 GC cycles in 15 seconds against a 1.5MB live heap. An
allocation profile attributed 99.92% of all process garbage to io.ReadAll
under ReadMessage; a CPU profile spent 15% in memclrNoHeapPointers and ~21%
in the collector, against 8% in AES-GCM.
Use NextReader and copy straight into the caller's buffer. The reader is
kept across calls and dropped at end of message, so a message larger than
the caller's buffer is still delivered across successive reads.
This also fixes a zero-length message surfacing as a (0, nil) read, which
io.Copy counts as no progress and rejects after 100 in a row.
Benchmark on 64KB frames:
before 39267 ns/op 1668.97 MB/s 138444 B/op 19 allocs/op
after 19920 ns/op 3289.98 MB/s 80 B/op 3 allocs/op
ParseAuthorizedKeysFile looped over ssh.ParseAuthorizedKey, but x/crypto's
parser splits on '\n' and returns rest='\n' for a trailing newline. The next
iteration called ParseAuthorizedKey('\n'), which skips whitespace and returns
'ssh: no key found', aborting the whole parse. Any normal authorized_keys file
(ends with newline, or has comments/blank lines) failed despite being valid.
Break the loop once rest is exhausted so all keys are parsed.
Disable Nagle on the underlying TCP socket for relay+ws, relay+wss, and
relay+tls listeners/dialers. Small latency-sensitive datagrams (e.g.
WireGuard UDP over a relay tunnel) were being coalesced by Nagle until a
peer ACK returned, adding up to one RTT per datagram and inflating
round-trip latency (go-gost/gost#858). wstunnel sets TCP_NODELAY on its
tunnel socket by default and keeps latency flat; gost did not.
Add ApplyNoDelay helper and WrapNoDelayListener (mirrors ApplyKeepalive),
applied on the raw *net.TCPConn before the proxyproto/tls wrapper chain.
Add RESP protocol detection to the sniffing framework so plaintext
Redis traffic on a port-reuse listener can be identified and routed by
protocol. Sniff() now peeks 1 byte first to match RESP type bytes
(*+-:$) without blocking, then falls through to the existing 5-byte
TLS/HTTP/SSH detection.
ParseRedisMetadata consumes exactly one RESP message via io.TeeReader
and records command/key in HandlerRecorderObject.Redis. HandleRedis in
both the sniffing (injected-dial) and forwarder (hop-selected) packages
replays the captured first message and pipes the rest, so pipelined
commands beyond the first are forwarded transparently.
Wired into all 5 dispatch sites: sni, forward/local, forward/remote,
http CONNECT, and relay tunnel handlers. forwarder paths route via
resolveRedisNode using ProtocolSelectOption(ProtoRedis), enabling
`protocol: redis` hop selection.
Part of two-phase plan (2026-07-29-sniffing-redis-postgres.md);
PostgreSQL deferred to phase 2.
Extract PopulateQUICClientHello/PopulateQUICServerHello into
internal/util/sniffing/quic.go to deduplicate the QUIC ClientHello/ServerHello
population logic that was copy-pasted across forward/local (Handle, handleRawDatagram)
and redirect/udp handlers (3 sites → 2 helpers).
- handler/redirect/udp: add QUIC ClientHello + ServerHello sniff with
sniffing/sniffing.timeout metadata, populating ro.Proto/ro.TLS for recording.
- handler/forward/local: replace inline QUIC blocks with shared helpers;
use h.md.bufferSize instead of hardcoded 8192 for sniff reads.
- internal/util/sniffing: add ProtoQUIC constant.
Bump quic-dissector dependency to v0.1.0.
ginuerzh
