Compare commits

...
112 Commits
Author SHA1 Message Date
ginuerzh c962805818 fix(handler/tun): end a p2p dial when the echoes stop
A p2p dial's stream survives the peer session under it being rebuilt,
so the link can go silently dead: the far side re-creates its session
while this side's stream becomes a zombie — writes succeed into the
void and nothing reports an error. firstInboundTimeout was a one-shot,
cleared by the first inbound byte and never re-armed, so such a dial
stayed "running" forever: on 2026-10-17 an Android spoke sat in a
two-way blackhole for 17 minutes after the hub killed and rebuilt its
relay session alone (derp peer-gone kill), while both ends reported
the path up; only a manual entrypoint restart healed it.

The registration refresh is echoed by the hub while the route to this
device is real, so silence is measurable. A p2p dial now re-arms the
read deadline to p2pInboundTimeout on every inbound byte; when it
expires the dial ends and the caller redials, which re-registers and
heals a hub-side rebuild — the same recovery a manual restart performs
(observed as "peer relay session up … client=true"). Socket dials
keep the one-shot semantics: their transport reports the far side gone
by itself.
2026-10-17 11:09:25 +00:00
ginuerzh cd1aabd1cc fix(handler/tun): name each unrouted window after the miss that opened it
The no-route report named the destination of the first unrouted packet
the process ever saw: the field was set while the window was empty and
never rewritten. A hub's own MLD noise (ff02::16) arrives early and
usually wins that race, so every later window blamed it -- including the
windows made of a peer's downlink being discarded, which is how a lost
registration stayed invisible in the log.

Set the window's destination when the window opens, and clear it with the
window, so each report names the miss it describes.
2026-10-16 06:29:07 +00:00
ginuerzh 88a68f51c8 fix(handler/tun): refresh the p2p registration on an interval
A spoke registers the addresses it holds by writing a handshake frame to
the hub, and the hub routes return traffic by that mapping. The handshake
ran once per dial, which was safe while a stream ending was what told a
spoke its peer was gone: the dial failed, the loop dialed again, and the
registration was re-sent.

A p2p dial no longer works that way. Its stream is kept across a peer
session rebuild, so the hub can drop the route while the dial still reads
as alive -- and then every return packet is discarded for want of a
route, silently at both ends. The link still reports itself "direct" and
healthy, the uplink keeps working, and only a manual restart of the
entrypoint brings the downlink back.

Re-send the handshake on a fixed interval when the dial target is a p2p
peer (network "ip"). A socket dial keeps the one-shot: its transport
reports the far side gone by itself.
2026-10-16 06:29:07 +00:00
ginuerzh 22eb86d08e test(p2p): sync SupportedDialer test with the widened whitelist
5b85e90 widened the inner-dialer whitelist to wss/grpc/http2/h2/ssh/sshd/mwss but left the test asserting fail-closed for grpc and http2, so TestSupportedDialer failed on every machine. Update the test list to match the implementation (kcp/quic stay fail-closed) and refresh the stale doc comment that still claimed http2 probes and destroys tunnels.
2026-10-14 01:56:32 +00:00
ginuerzh a44625fcb7 test(metrics/wrapper): enable metrics where wrapping is asserted
Since WrapListener returns the listener unchanged when metrics are disabled (3f01456, kept for the splice(2) optimization), the accept-time wrapping test never saw a wrapped conn and failed on the type assertion. Enable metrics for the test with a helper that restores the previous state, and add a companion test that pins the disabled passthrough behavior.
2026-10-14 01:56:32 +00:00
ginuerzh 92272e0bb4 fix(handler/dns): stop dropping nameservers with a port
systemNameservers validated the whole host:port string with net.ParseIP, which rejects anything carrying a port, so a legal resolv.conf entry such as "nameserver 8.8.8.8:5353" was silently skipped and the following SplitHostPort branch (missing port defaults to 53) could never run. Split first, validate the host only, and reassemble with JoinHostPort, which also brackets IPv6 addresses.

The Init fallback chain (hop/metadata DNS -> resolv.conf "system-N" -> hardcoded default) was also untestable: the resolv.conf rung, added in 684a0dd, depends on the host's file, so TestInit_DefaultExchanger failed on any machine that has one. Introduce a resolvConfPath indirection (a no-op var on Windows, where systemNameservers never reads a file) and add TestInit_SystemNameservers, which exposed the parser bug above.
2026-10-14 01:56:16 +00:00
ginuerzh 993419d845 fix(handler/tun): guard every packet classifier against an empty buffer
waterutil.IsIPv4 and IsIPv6 index packet[0] with no length check, and a
tun device can legally return zero bytes with no error — an Android
VpnService fd does exactly that. routing such a read into a classifier
panics with "index out of range [0] with length 0", and in the c-shared
library the phone loads the runtime cannot unwind past the cgo boundary,
so the panic took the app: the spoke died on 2026-10-12 with three such
lines in crash.log, no stack, and a 17-minute gap before Android
restarted it.

The knowledge was already in the tree — sourceOf and destinationOf in
handler/tun/p2p.go carry the guard, with a comment naming this exact
panic — but the eight reads in the client and server loops and the
association classifier in handler/router did not. The asymmetry was the
bug.

The guard now lives once, in internal/util/tun, as IPv4Packet and
IPv6Packet, and every classifier goes through it. Tested at the real
entry point: a fake device that returns (0, nil) before handing over a
real IPv4 packet, asserted to be survived and forwarded.
2026-10-13 14:12:50 +00:00
ginuerzh 8d253dcd88 fix(logger): rotate a file log that names no rotation
A log config can name an output without naming a rotation, and that
combination used to receive a plain *os.File — nothing rotates it, ever.
One spoke on Android ran debug-level logging into exactly that shape and
grew the file to 111MB with no backups, which on a phone is not untidy so
much as fatal.

Default the rotation instead. The values match what an unconfigured
logger has always carried, so naming an output is no longer a way to ask
for an unbounded file, and a configured rotation still wins as written.
outputWriter is split out of ParseLogger so the decision — the part that
chooses whether a file log stops growing — is testable without a logger.

Signed-off-by: Step 5 Preview Free <noreply@stepfun.com>
2026-10-13 02:40:19 +00:00
ginuerzh 49c998ee45 fix(service): keep accepting after a conn limiter rejection
One connection over the climiter quota killed the whole service. The conn
limiter wrapper signals an over-limit connection by returning a plain
error from Accept, which the accept loop does not recognise as either a
listener.AcceptError or a temporary net.Error, so it fell through to the
fatal branch and returned. program.go discards Serve's return value, so
the process stayed up with no further logging while nothing accepted
connections any more. Every later connection sat in the listen backlog,
where the kernel completes the handshake but no service ever reads it, so
clients saw a connection that opens and then never negotiates. Only a
restart recovered the service.

The counter itself was never the problem: llimiter.Allow rolls the count
back when it refuses, and a test now pins that. A test also pins that
the accept loop survives a rejection and still serves the connection
that follows it.

Export ErrConnLimit so the refusal can be told apart from a listener
that has genuinely failed, and continue the accept loop on it. The
plain-http-tunnel listener refused over-limit connections with an
identical error and is fixed the same way.

Closes #914
2026-10-11 10:26:15 +00:00
ginuerzh 0954e57dcb fix(redirect/tcp): forward protocols the sniffer does not handle
Since 3.3.0 a transparent red service with sniffing enabled closed every
connection whose protocol Sniff does not classify as HTTP or TLS. SSH,
Redis, and any raw TCP — plus server-speaks-first protocols such as SMTP,
where Sniff returns "" after sniffing.timeout — took no matching branch
of the switch on proto, so sniffErr stayed nil and the handler returned
nil, closing the connection via the deferred conn.Close(). No dial was
attempted, so the upstream node never saw the connection, and nothing was
logged above info.

This traces to c23a2f1, which introduced the error-returning switch in
order to replay sniffed bytes for the empty-SNI case (#891). Before it,
the switch had no error variable and simply fell out into raw
forwarding, so every unclassified protocol was proxied as-is.

sniffing.fallback is not a usable workaround: it also changes dial() so
that a failure to reach the sniffed host re-dials the original IP, where
domain-based hop/node bypass rules no longer match. That can route a
domain through a node its bypass excludes.

Mark the two handled protocols explicitly and let everything else fall
through to raw forwarding unconditionally, restoring the 3.2.x
behaviour. sniffing.fallback keeps its intended meaning: it governs only
what happens after HTTP/TLS handling itself fails (#891).

Closes #913
2026-10-11 10:15:11 +00:00
ginuerzhandClaude Opus 4.8 1d0b6de411 feat(handler/tun): log head bytes of dropped non-IP frames
The hub drops non-IP frames instead of writing them to the tun device,
where the kernel rejects them with EINVAL and takes the service down.
That guard was correct but silent about what it was dropping: a 4-byte
frame was seen in the field and the bytes are the only way to tell a
tagged-stream leak from anything else.

channelTag "P2PU" is exactly 4 bytes, so a tag leak is the obvious
suspect, but the sequence that preceded the sighting -- an untagged
inbound delivery after a 2s peek timeout -- was later reproduced 23
times, including forced mobile-data switches on the phone, without
producing another one. The source stays unconfirmed either way, so the
guard keeps logging up to 8 head bytes to make the next occurrence
identifiable from the log alone.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-11 08:14:30 +00:00
ginuerzh 4f143aa694 fix(tun): drop non-IP frames at the hub instead of writing them to the device
A frame that is not a parseable IPv4 or IPv6 packet is exactly what the
kernel refuses. Measured on a real up tun device: the version nibble is
the only thing it inspects, and anything but 4 or 6 comes back EINVAL at
any length -- 1421, 1500 and 2000-byte frames with a valid header all go
through untouched. That error left fromSpoke, unwound Handle, and took a
54-minute peer stream down with it.

The spoke has always refused these frames before its device write, which
is why a phone logs "unknown packet, discarded" all day and never dies
while the hub had no equivalent. destinationOf returning !ok is exactly
the kernel's reject condition, so that branch now drops and counts the
frame instead of handing it to the device.

This stops the malformed frame from killing the service; it does not fix
what put one on the wire. The candidate cause is concurrent serveEdge
writers racing on one local conn with no send lock on streamConn.Write,
which is still unproven. Silently accepted sub-MTU spliced frames also
remain -- the kernel does not validate TotalLen either.
2026-10-11 02:30:15 +00:00
ginuerzh a654aee7af feat(stats): the hub's LAN routing counters
LanRouted, LanDenied and LanWithdrawn sit beside the probe counters on the
same struct: a hub's peer-LAN routing is counted where every other service
counter lives, and where the API and the stats view already read from.
2026-10-09 13:31:47 +00:00
ginuerzh c2c78f3279 feat(tun): install prefix routes into the p2p handler
A hub that decides which peer LANs exist is handed this handler as a
handler.Handler, so the table has to be reachable by shape: the route type
is exported and the p2p handler takes the same call the socket one does,
installing into the peerTable its packet path already reads.
2026-10-09 11:01:04 +00:00
ginuerzh 9e3ce7cb68 fix(tun): gate prefix routes on the authenticated stream peer 2026-10-09 07:18:54 +00:00
ginuerzh bb682a90c9 fix(tun): resolve a prefix route's requester from the packet source 2026-10-09 06:57:23 +00:00
ginuerzh be86cfef74 feat(tun): route hub packets by prefix after exact match 2026-10-09 06:48:08 +00:00
ginuerzh 640e239280 feat(tun): relay spoke-to-spoke packets directly between peers
The p2p hub wrote every spoke packet to its tun device, relying on the
host kernel's same-interface forwarding to hairpin it back to the
destination peer. Hosts without that forwarding silently dropped
peer-to-peer traffic. fromSpoke now resolves the destination through
the route table first and delivers to the owning peer's stream;
unroutable, self-addressed, and hub-local packets keep the device path.

Tests cover direct delivery (v4/v6/large), self-send drops, unrouted
fallback, and hub-own-address fallback. The suite passes under -race
except TestRunDeviceProbeReportsSent, which races on a clean tree.
2026-10-07 11:26:09 +00:00
ginuerzh 98f64ea13d feat(tun): drop p2p stream on refused registration, bound first-inbound wait
A hub that refuses a spoke registration (bad passphrase, allowlist,
self-loop) used to stay silent and keep the stream: the spoke sat
'running' with zero traffic and nothing to look at. Return
ErrRegistrationRefused instead so the stream is torn down — the spoke
redials and re-registers, which also recovers on its own once the hub
is fixed. Malformed frames and bare echoes still return nil and leave
the stream up, so transport noise cannot turn into a reconnect storm.

Without a keepalive period the client reader had no deadline at all and
waited forever on a silent link. Bound it with a 30s first-inbound
one-shot (firstInboundTimeout, cleared by any inbound byte); the
keepalive>0 path is untouched.
2026-10-07 09:28:32 +00:00
ginuerzh 295a2c2296 docs: add tun device probe implementation plan 2026-10-07 05:07:13 +00:00
ginuerzh 1d42c2ec66 feat(tun): add device write-side self-probe
A running tun entrypoint cannot tell a dead device fd from an idle link: both look like frozen stats. The probe writes a self-addressed IPv4/UDP packet (spokeIP->spokeIP, magic+seq) into the device every 30s and reads it back on its own bound socket; sent/acked deltas go to the caller through the metadata probeReport callback. Enabled by metadata probe (default off, gost path unchanged); new probeSent/probeAcked counters (101/102) in x stats.
2026-10-07 05:07:13 +00:00
ginuerzh 3e533c61c7 Merge pull request #126: emit periodic session snapshots and byte deltas 2026-10-07 02:08:41 +00:00
ginuerzh 89188536d3 Merge pull request #133: let concurrent redials converge on one session 2026-10-07 01:47:38 +00:00
ginuerzh 239da867c3 Merge pull request #132: authorize every request against the bypass 2026-10-07 01:43:22 +00:00
ginuerzh 17909c4f95 fix(nodes): explicit reference check, trim lookup, deep-copy global definitions
IsReference no longer uses reflect.DeepEqual; ResolveConfig trims the name before registry lookup; cloneConfig deep-copies all nested fields so hop parsing never mutates shared globals. Also documents the intentional fail-fast vs warn-and-skip error semantics.
2026-10-07 01:36:45 +00:00
ginuerzh 8f6a725e7e Merge pull request #130: Global nodes registry for reusing nodes across hops 2026-10-07 01:30:30 +00:00
ginuerzh 312187d3d8 test(bypass): cover empty whitelist bypass-all after #129 2026-10-07 01:22:47 +00:00
ginuerzh bacaa9af89 Merge pull request #129 from iBug/bypass 2026-10-07 01:21:06 +00:00
ginuerzh a939758583 fix(tun): survive oversized packets without panic or stream teardown
A 20546-byte spoke datagram hit wbuf[:writeOffset+len(p)] (cap 16400)
and panicked the whole hub process. Three parts:

- tunDevice.Write rejects packets bigger than its buffers with the new
  exported ErrPacketTooLarge instead of reslicing past the end. The
  kernel would have refused the write anyway (tun MTU 1420), so nothing
  deliverable is lost.
- Buffers follow the device MTU (max 16KB): newTunDevice(ifce, mtu),
  so raising tun.mtu for legitimate jumbo UDP automatically makes such
  packets writable. The relay/KCP/stream path already passes 64KB.
- fromSpoke drops an oversized datagram with a windowed count and
  keeps the peer stream up; only a real device failure still tears
  the stream down.

Tests: tun write guard, normal passthrough, large-MTU accept, and
stream survival with rate-limited warnings; all written first and
watched fail.
2026-10-07 01:19:06 +00:00
ginuerzh 9ec8bdbfeb fix(tun): summarize unrouted-packet floods; log peer stream lifetimes
An unrouted destination under load produced one warn line per packet,
burying the event that caused the routing gap. Count misses and report
the total once per window instead; the first miss is still logged at
once.

Also log how long a peer stream lived and how much arrived on it when
it ends, so a dying peer's story (failed to connect vs failed mid-
transfer) is visible without correlating route drops.

Also ignore .memory/.
2026-10-07 01:18:38 +00:00
ginuerzh 4bbf81c9f4 feat(p2p): let a chain node ask the tunnel for a network
The tun link needs the tunnel network "ip" rather than the "udp" its dialer
would otherwise derive: "ip" is the datagram link, marked session-scoped on
the host so it ends on a lost peer session and the consumer re-dials (see
the p2p companion change). Add WithTunnelNetwork to override the derived
network and MDKeyP2PNetwork ("p2p.network") for a chain node to set it, and
have streamconn frame "ip" like "udp" so the GOST-side conn owns the 2-byte
framing on both carriers. An empty network keeps the derived value, so every
existing node is unchanged.

Gate: go build ./... and go vet ./... clean.
2026-10-06 00:36:49 +08:00
lueyoungandClaude Opus 5.5 6911ecd304 fix(mtls,mtcp,mws): let concurrent redials converge on one session
When no live session exists, each request dials a conn of its own and
registers it in Dial, then builds the session on it in Handshake, each
step under sessionMutex. A Dial that finds a registration still waiting
for its handshake takes it for dead (IsClosed reports true while
session is nil) and replaces it, so the earlier request's Handshake no
longer finds its own conn and fails with "unrecognized connection".
Since c210b74 the replacing Dial also closes that conn.

Once requests queue for the lock, another request's Dial can sit
between every Dial and its Handshake, and no session is completed for
as long as requests keep arriving. Seen on a three-hop relay chain
whose last hop is mtls: after the client host woke from sleep, nearly
every request on a chain with ~28 concurrent clients failed this way
for 80 minutes, until the load happened to drop.

Dial now closes the base conn only of a session that was established
and died; a session still waiting for its handshake belongs to the
request that dialed it, whose Handshake either builds the session on
it or closes it. Handshake no longer fails on a mismatch: if the
registered session is live it uses that and closes its own conn,
otherwise it builds the session on its own conn and registers it, and
the other request's Handshake then finds a live session to use.

The tests dial twice before either handshake, in both orders, against
the package's own listener. On master they fail with "unrecognized
connection", and with "use of closed network connection" when only
Handshake is changed.

Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
2026-10-04 08:50:41 -04:00
ginuerzh 429c6a6199 fix(tun): drop keepalive frames from the client read loop
A registration handshake or keepalive echo is protocol, not a packet,
and must never reach the device. The client read loop only special-cased
the 20-byte bare echo, so a 20+N*16-byte registration frame from the
peer was misread as IPv4 ('G' is 0x47) and injected into the kernel
device as junk. The hub (p2pHub.fromSpoke) and the socket server already
filter this shape; the client now does too.

Covered by TestTransportClientDropsKeepaliveFrames, which fails against
the old shape (the device receives the frame verbatim).
2026-10-04 01:51:14 +00:00
ginuerzhandClaude Opus 4.8 1e17b38512 docs(tun): correct a stale comment on the p2p hub's authorizer
The comment above newPeerTable claimed "the authorizer is an option rather
than an argument", written when NewP2PHandler did not take one. It now does,
so the comment contradicted the signature directly above it, and it still
referred to an "auther argument" that no longer exists.

Both facts are true but at different levels, which is what made the old
wording hard to read: the authorizer is a variadic option on newPeerTable
(whose other callers all mean "no address authorization"), while
NewP2PHandler takes it as an argument so a hub's owner has one place to
pass it. The auther is nil throughout, since the p2p peer allowlist has
already decided who may connect.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-03 23:53:59 +00:00
ginuerzh d2678b8f5f feat(tun): the p2p hub authorizes addresses instead of credentials 2026-10-03 22:57:58 +00:00
ginuerzh 3d49aef879 feat(tun): let a hub decide which addresses a peer may claim 2026-10-03 17:10:21 +00:00
Mike-7777777 dc9c7444b9 sniffing,forwarder: authorize every request against the bypass
The bypass was evaluated once per connection, which authorizes the first
request and nothing after it. A keep-alive request naming another host
re-dialed without a check, a request served from the HTTP cache needed no
dial and so passed no check, a request arriving after a reload was still
judged by the policy that had been in force when the connection opened,
and an HTTP/2 :authority was never checked at all. With MITM enabled the
same holes applied to HTTPS, since the decrypted stream is handed to the
HTTP path.

State the invariant instead of patching the sites one by one: every
request forwarded to an upstream is authorized against the bypass in
effect at the time of that request. CheckBypass is that one check, called
from the four request-handling paths in the two sniffer copies. In the
HTTP loop it replaces the pre-loop check rather than adding to it, so the
file ends up with one check site instead of two.

A reload now reaches established connections at their next request. That
is a behaviour change, and it is what closes the cache and reload paths.
Tunnels stay authorized at establishment: a TLS connection without MITM,
a websocket after upgrade and a CONNECT tunnel have no later request to
check.

CheckBypass is nil-receiver safe because h2Handler can be built without
handle options, and a data path should not panic over that.

Measured cost against a 269-pattern bypass: 645 ns for a host that
matches nothing (full wildcard scan), 149 ns for an exact hit.
2026-10-03 18:46:11 +02:00
ginuerzh f1cd7c945a tun: the registration handshake is not a keepalive option
A tun client registers its device's addresses with the far side's tun
server by sending the handshake frame. That frame is what tells the server
which addresses this device holds, and it is the only thing that can: a
server routes by destination address, so without it the link is one-way —
traffic leaves and nothing comes back.

keepalive() has always sent it unconditionally and returned early when no
period was configured, so the period only ever decided whether it
repeated. The call site did not: it was gated on the udp network or a
configured period, which on a p2p link ("ip") meant no period, no
handshake, no registration at all. That is the shape a p2p hub needs
least — its route table has no TTL to refresh, so the repeat buys nothing
there and the one-shot is the whole of it.

The gate is gone; the handshake always runs. The period keeps its meaning
where it has one: a server reached over UDP is connectionless, so silence
is the only signal that a peer has left, and the repeat is what lets its
route expire.

Found by a user asking why the keepalive switch was needed at all. It was
not, on a p2p hub — and turning it off silently produced the one-way link
the entrypoint's own comment warns about.
2026-10-03 22:45:29 +08:00
ginuerzh bc137fbd83 relay,socks: one reverse-tunnel datagram frame per write
gost#911: two concurrent UDP source sockets over one RUDP reverse tunnel
over relay+TLS corrupted echo responses and made the host tear the tunnel
down and rebind it every second.

The tunnel is a single stream shared by every endpoint multiplexed over it.
A datagram frame is not written to that stream atomically —
UDPDatagram.WriteTo issues three separate Write calls (RSV/FRAG, SOCKS5
address, payload). Each Write is concurrency-safe on its own, but with
several endpoints writing, frames interleave between those calls. A torn
frame also leaves half a header in the stream, which the far end reads as
io.ErrUnexpectedEOF and answers by rebuilding the whole tunnel.

Hold a mutex for the whole frame instead of just the writes the datagram
happens to make. Same defect in both udpTunConn copies.

Reproduced with the reporter's script: relay+tls concurrent phase went from
44% loss with corrupted payloads and 1.0s gaps to 0.02% loss, no corruption,
0.003s gaps. Plain relay likewise. Single-endpoint high-rate loss (~35% at
14k pps) is unaffected UDP backpressure, not this bug.
2026-10-03 15:32:38 +08:00
iBug c67b2ec1d2 feat: Global nodes registry for reusing nodes across hops 2026-10-02 22:23:34 +08:00
ginuerzh 8787aff108 tun: a handler that owns a p2p hub and bridges streams into it
p2p.go was a complete engine with nothing driving it: no handler.Handler,
so no GOST service could reach the hub and a spoke's registration frame had
no path to one at all.

NewP2PHandler takes the tun device conn and an auther, builds the peerTable
(no TTL — a stream close says a peer is gone outright, where a socket peer
has only silence to guess from), the peerRouter and the p2pHub, and starts
the device reader. Handle installs one accepted stream under its remote
address — the peer key the p2p host stamped on it, replacing whatever a
reconnect left behind — and feeds its datagrams to the hub until the read
fails. Teardown is peerGone and nothing else: it reclaims a peer's routes
only while that stream still holds its key, so a stale stream ending cannot
cut off the live connection its peer already replaced it with.

ownNets comes off the device conn's context, the two steps server.go's Handle
already uses, because a nil answer silently disables the self-route guard.

Handle returns the read error unwrapped. io.EOF, net.ErrClosed and
context.Canceled are what isServerError calls an ordinary end, so a peer
reconnecting would otherwise land in someone's error log.

The p2p half of the compat suite now runs against the real handler and the
real hub rather than reporting itself unavailable: the same six assertions,
the same two cases skipped without an auther. A second p2p case covers the
reconnect, which is the only way the suite would notice a peerGone that
dropped routes unconditionally — a hub that did so passes every shared case.
2026-10-02 20:05:07 +08:00
ginuerzh 6e84d375c1 tun: one assertion set against both hubs, so a spoke cannot tell them apart
The socket server and the p2p hub are two answers to the same question, and
the design rests on a spoke being unable to tell which one it reached: the
same frame, the same magic, the same passphrase, the same registered
addresses, the same reply it refreshes its read deadline from. A spoke
configured for one is supposed to work against the other.

That claim is only worth something if the *same* assertions run against
both. Two separately-written suites would be allowed to drift, and nobody
would notice until someone deployed one against the other — which is a
field failure with nothing in any log to explain it. So the assertions are
written once here, against a compatHub that names no transport, and each
case is run once per implementation. The duplication that is left is
structural: how to build a hub and how it delivers, which is the part that
genuinely differs.

The socket half drives server.go's own transportServer over a real UDP link
rather than a copy of its loops; a compat suite that asserted against a
reimplementation would only prove the reimplementation agreed with itself.

The p2p half skips, and says why: nothing yet turns an inbound gRPC Tunnel
stream into a peerStream and feeds it to p2pHub.fromSpoke, so a spoke's
registration frame has no path to that hub. The skip is visible and the
six shared cases are named in it, because an absent assertion is not a
skipped one.

Three deliberate divergences turn the suite red, each from a real behavior
rather than a stub: registering a route for a frame the auther refused,
delivering to whichever peer spoke last, and echoing nothing after a
registration that took.
2026-10-02 19:14:20 +08:00
ginuerzh e95b830670 tun: a p2p hub reads the device once and locks every write to it
The route table answers a destination with a name, and the socket server
turns that name into a UDP address. A p2p hub needs the other answer: the
name is a peer key, and the thing it resolves to is a stream, not an
address. That is the whole of the second delivery — resolveUDPAddr is
unreachable here by construction, and putting it back on this path would be
a design error rather than a bug.

The shape is forced by the device, not chosen. tunDevice shares one read
buffer and one write buffer across every call and takes no lock: two
readers would have the second overwrite the first's target mid-call, and two
writers would copy through the same write buffer at once. It is safe today
only because the socket server happens to run one goroutine per side. A p2p
hub runs one per peer, so it has to reach the same guarantee a different
way: run() is the only goroutine that reads, and wmu is the only way to
write. This is the socket server's shape arrived at from the other side —
not a bridge per stream. Which goroutine read a packet carries no meaning,
because the route is what says where it goes.

Two locks, for two different hazards. The device's wmu guards the shared
device buffer. peerStream's own mu guards one peer's stream, whose conn
frames every write onto a byte stream and serializes only its read buffer:
two writers would put one frame's length prefix against the other's
payload, and every frame length would then be read from the wrong offset.
The -race run is what proves both are doing their job.

withdraw removes a stream only while it still holds its key. A peer that
reconnected already installed a replacement under that name, so reclaiming
routes for it on the old stream's teardown would cut off the live
connection — which is what makes a flapping peer unreachable rather than
merely briefly absent.

The keepalive is answered on the stream it arrived on, because the table
registers but never writes. Drop the echo and every keepalive:true spoke
misses the reply its read deadline is refreshed from and redials on its own
timeout, with nothing in any log: client.go's registration gate tests for
"udp" and a p2p link is "ip", so a spoke that never registers is already
indistinguishable from silence. Hence the warning naming the destination.

The concurrency tests are mutation-checked: with the device lock removed the
first fails on overlapping writes in round 0, and with the stream lock
removed the second fails on a spliced frame. Both were caught by the
assertions, not only by the race detector — the first harness was a
datagram pipe, which is atomic per message and so passed with the lock gone
until it was replaced by a byte stream, which is what the p2p conn really
is.
2026-10-02 17:56:32 +08:00
ginuerzh 64b14930b3 tun: the socket server keeps its behavior, on the shared router
Its keepalive handling, route lookups and both delivery loops move onto
peerTable unchanged: the same magic-header reply, the same gateway
fallback, the same discard for an unrouted packet. Only the route's
value is now a name, resolved at the point of delivery. A test that
needed updating to pass would mean the socket path moved when it must
not have, so the line-by-line diff is the evidence here — no test in
this package exercises server.go.

The route table's reverse index is gone. It was a second copy of a fact
already in the route's name, and two maps cannot be made consistent:
concurrent set calls could leave the index naming one peer while the
route belonged to another, and dropPeer would then delete the winner's
route on the loser's teardown. dropPeer now scans the routes themselves
and deletes only entries still owned by the peer it is dropping.

A lock would have been the wrong fix: lookup runs per delivered packet
and dropPeer once per stream close, so the cost would have landed on the
hot path to settle a race that only matters at reclamation.
2026-10-02 16:56:30 +08:00
ginuerzhandClaude Opus 4.8 70b52694eb tun: the route table holds a name, not an address
The keepalive protocol, the auther check, the TTL and destination lookup
are the same whichever way datagrams arrive, so they belong together and
know nothing about transport. A route's value becomes a string: a UDP
address for the socket server, a peer key for the p2p hub. net.Addr fit
neither — it is an endpoint, and one of these is not.

dropPeer is the p2p half of reclamation. The socket server expires a
route by TTL, guessing from silence that a peer is gone; a stream close
says so exactly, so the hub drops a peer's routes when its stream ends
and keeps no timer.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-02 16:13:25 +08:00
ginuerzhandClaude Opus 4.8 8683da6a65 mux: a stream's deadline must not reach the shared session conn
streamConn embeds the session's net.Conn and overrides Read/Write/Close,
but not the deadline methods, so those reached the connection every
stream shares. Any deadline set by one stream then outlived it and
decided when the others could read.

net/http makes this reachable: tearing down a connection parks a
pending read with SetReadDeadline(time.Unix(1, 0)) and clears it only
when a read was actually in flight. Without a read in flight the shared
conn is left with a deadline in 1970, and the next Accept on the
session fails immediately with an i/o timeout — reported as a listener
error even though the session was healthy.

Delegate the three deadline methods to the stream, which has always had
its own. The test drives one request end to end over a stream, applies
that teardown deadline, and asserts the session still hands out the
next stream.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-10-02 12:44:14 +08:00
ginuerzh 2306f45bf9 service: a closed listener is not a failure to warn about
A remote listener (rtcp/runix/rudp) wraps an accept failure in core's
AcceptError — which always answers Temporary() — to say "rebuild the transport
and try again", and a closed pipe under it is one of the failures that recovers
that way, so the retry stays exactly as it was. What was wrong is what it said: a
stop arrives down the same path, and a warning that promises a retry reads like a
hiccup in the log of a service that is simply being stopped. The retry is logged
at debug when the error means a closed listener, pipe or socket; every other
failure still warns, and the tail still logs real failures at error.

No behaviour change: the retry, its backoff and the state transitions are
untouched — the classification decides only the log level.
2026-10-01 14:28:33 +08:00
ginuerzh 85c257ad65 service,handler/tun: a deliberate stop is not an error
Stopping a service closes its listener, and a closed listener answers Accept
with core's listener.ErrClosed — which the guard never matched (it knew only
net.ErrClosed), so every stop reached the log at error level and read like a
crash. The tun handler logged the same shape a second time: its client wrapped a
closed device with %s, throwing the cause away, so the caller could not tell a
device closed under the reader from a real read failure. %w keeps the chain and
the handler logs the closed case at debug.
2026-10-01 13:28:40 +08:00
ginuerzh ec4800293d config/parsing/logger: say when a log file has no rotation
A configured output path without a rotation block is a plain file that nothing
rotates: it grows until the disk does something about it. The default logger —
the one an unconfigured program gets — always carries a rotation, so this only
happens to a hand-written config, and the parser is the only place that sees
both the path and the rotation that is missing. Reported on stderr, once, at
parse time; nothing about the behaviour changes.
2026-09-29 16:18:39 +08:00
ginuerzh 1fadb8fe38 config/parsing/logger: a log file that cannot be opened is not fatal
ParseLogger reported the failure through logger.Default(), which is nil until a
caller installs one — and this call is usually what installs it. An unwritable
path therefore dereferenced a nil logger and panicked; inside a c-shared library
that aborts the process, which on Android took an app down a second or two after
start with nothing readable left behind (an app's stderr goes nowhere, and the
tombstone carries no abort text).

Measured with Output "file" — a path, not a mode — in an app whose working
directory is "/" and not writable.

The failure goes to stderr instead, which is always there, and the default
output stays in place. The regression test forks a child process, because a
fresh one is the only place the default logger is still unset: TestMain skips
installing its logger for that child.
2026-09-29 16:03:21 +08:00
iBug 660af7ce59 feat(bypass): Make network respect whitelist 2026-09-26 22:00:37 +08:00
ginuerzh ce290f9143 feat(listener/tun): adopt a host-provided device fd on Android
An unprivileged Android app cannot open /dev/net/tun, so the device can only
come from a VpnService: the Java side creates and configures it (addresses,
routes, MTU, DNS via VpnService.Builder) and hands its fd over as the "fd"
metadata, either as a fixed descriptor or — for an embedder, which is what the
platform's own VPN is — as a provider resolved when each device is built, so a
device rebuilt after its session ended never copies a closed or recycled
descriptor.

The device is created with tun.CreateUnmonitoredTUNFromFD (no netlink listener,
no setMTU — the builder already set it), which makes the Linux variant, with
its netlink and resolvectl calls, inapplicable: it is now linux && !android.
The interface lookup in the accept loop was fatal on any error; it is
informational, and on Android the device may not be listed in this process.
2026-09-24 15:56:46 +08:00
ginuerzh 1776e1a4ef p2p: rename TunnelProvider to Tunnel and OpenTunnelStream to Dial 2026-09-22 15:17:10 +08:00
ginuerzh a9cc544470 docs: widen the rename plan to TunnelProvider -> Tunnel 2026-09-22 15:09:37 +08:00
ginuerzh 73baffc689 docs: plan the TunnelProvider.Dial rename 2026-09-22 15:06:42 +08:00
ginuerzh 5b85e90db8 chore(x): bump plugin to v0.8.1 and widen the p2p inner dialer list
Bump github.com/go-gost/plugin to v0.8.1, which reports transport stats
over the p2p Status RPC (direct/derp peer gauges plus punch/stream
counters), and pick up the matching go.sum entries.

SupportedDialer: also accept wss, grpc, http2, h2, ssh, sshd and mwss as
inner protocols on top of a p2p tunnel. The list stays fail-closed; each
entry is a dialer verified to dial one plain conn and use it as its base.
2026-09-20 21:23:22 +08:00
ginuerzh 3627030098 Revert "fix(forwarder): authenticate HTTP node with Proxy-Authorization"
This reverts commit 34fbff3d3c.
2026-09-20 20:51:30 +08:00
ginuerzh 34fbff3d3c fix(forwarder): authenticate HTTP node with Proxy-Authorization
Node auth on the HTTP forwarder path read the origin credential
(Authorization) and challenged with 401/WWW-Authenticate. When the
forwarded backend also used Basic auth, the node consumed the credential
meant for the backend, so the two could not coexist.

Read the proxy credential (Proxy-Authorization) instead and challenge
with 407/Proxy-Authenticate. Authorization is left untouched so it
reaches the backend, and Proxy-Authorization is stripped before the
upstream write since it is hop-by-hop (RFC 9110 11.7.2).
2026-09-20 20:43:11 +08:00
Leonov Vladislav 1c42c638a6 recorder: emit periodic session snapshots and byte deltas
Handlers write one recorder entry when a session ends, so long-lived
SOCKS5 tunnels, streaming HTTP responses and UDP associations stay
invisible while they run and report their traffic as one late record.

With recorder.period set, a session now emits interim records on the
interval while it is active and one final record when it ends. Each
record carries the cumulative counters alongside the increments since the
previous one, so a consumer sums records instead of replacing them, plus
sessionID, recordIndex and phase. A session that ends inside one period
still writes a single record, as it does today.

One goroutine per handler samples every live session on the tick and
writes its record straight to the recorder. There is no queue and no
retry loop, and none is needed: a session moves its cursor past a record
only once the recorder has taken it, so a refused write costs no bytes.
They stay in the delta and go out with the next record. Holding on to
what the sink cannot take right now is the sink's job.

Interim and Finish take a context and return an error, so a handler
learns what became of every record it asked for. Start writes nothing, so
it takes neither. Close stops the sampling and reports the sessions its
owner had not finished; nothing is buffered, so there is nothing to
drain.

With recorder.period unset or non-positive the previous behaviour is
kept: one record per session, written inline, in the old shape. An object
its handler never stamped, and one whose record mode is off, is dropped
in both modes.
2026-09-15 15:05:02 +03:00
ginuerzh 06197bd987 fix(handler/tun): run keepalive on p2p links and expire stale routes
Three fixes found while wiring the p2p hub topology:

- The client keepalive was gated on network == "udp", but on a p2p link
  raddr is the peer's base64 key (no port), so SplitHostPort fails and
  network is "ip": the keepalive never ran, a tun server never learned the
  client's route, and the read deadline was never armed. Gate on the udp
  network OR a configured keepalive period; the udp branch keeps the one-shot
  registration handshake it always sent.
- Liveness was refreshed only by the keepalive echo; a transient channel
  hiccup can drop that single packet and redial a healthy tunnel after
  3xperiod. Refresh the read deadline after any successful read instead.
- Routes were never removed: a departed client's IP -> addr stayed forever
  and the device kept blackholing to a dead address. Expire a route once its
  last keepalive is older than 3x the configured period, checked lazily on
  the write path (no sweeper). The TTL is opt-in: without a server-side
  keepalive/ttl, routes keep their current never-expiring behavior.
2026-09-13 14:50:29 +08:00
ginuerzh a5605b935c chore(x): bump kcp-go to v5.6.72 and smux to v1.5.57 2026-09-12 12:07:28 +08:00
ihipop 8c999f1f5b feat(socks5): add optional UDP device binding (#127) 2026-09-12 10:37:46 +08:00
ginuerzh 7a4db061c9 fix(handler/sni): clear sniffing read deadline before the data path
The read deadline set for ClientHello sniffing was reset via defer, so it
remained on the client connection until Handle returned, i.e. for the whole
session. xnet.Pipe is called without a read timeout for TLS/HTTP, so nothing
refreshed it and any connection was torn down after readTimeout (15s default)
regardless of traffic. Clear the deadline immediately after sniffing, matching
the relay/router/tunnel/socks4 handlers.

Regression test asserts the deadline is zero when the upstream dial fires.

Fixes #904
2026-09-12 10:23:53 +08:00
ginuerzh a3e82c086c chore(x): bump github.com/go-gost/plugin to v0.8.0 2026-09-12 09:39:40 +08:00
ginuerzh ad702b4cbe test(p2p): feed chunks concurrently in the reassembly test
The test fed all 715 chunks before the first Read; with the cap-1 pump
handoff and a 64-slot fake stream the feeder blocks long before the
reader starts — a deterministic deadlock the earlier "passing" runs
masked (a `go test | tail` pipeline hid the exit code). Feed from a
goroutine.
2026-09-12 09:39:40 +08:00
ginuerzh 2cb1365d46 fix(p2p): release the datagram assembly buffer at frame boundaries
readDatagram kept appending into one backing array; an idle conn held
onto the largest frame's array indefinitely (bounded, but anchored until
a later append crossed capacity). Drain it to nil once a frame boundary
empties it, and assert the release plus the received-bytes-only bound in
the tests.
2026-09-12 09:21:59 +08:00
ginuerzh 4f3261a9d9 feat(p2p): move tunnel data onto the gRPC Tunnel stream
TunnelProvider becomes OpenTunnelStream: OpenTunnel authorizes the tunnel
and issues a single-use id, the Tunnel stream carries the data. The new
x/p2p/streamconn conn wraps the stream as a net.Conn with real
Set*Deadline support (bounded Recv pump, idempotent Close, framed mode
for udp). The tunnel base dialer collapses to the stream; its failure
path closes the conn nil-safely instead of leaking.
2026-09-12 08:19:21 +08:00
ginuerzh 988c9afe28 fix(masque): reap idle CONNECT-UDP associations
udp.Relay.Run ignored ctx cancellation, so an idle MASQUE association (or a
disconnecting client) leaked its target UDP socket and both copy goroutines,
and service shutdown blocked on the stuck handlers. quic-go's
ReceiveDatagram only unblocks on ctx cancellation, never on stream close, so
closing the datagram conn could not release the read either.

Make Relay.Run cancellation-aware and close both conns on exit, add
WithReadTimeout for an idle read deadline, give DatagramConn a cancelable
context that Close cancels, and apply the parsed idleTimeout to the UDP
relay.

Ref: https://github.com/go-gost/x/issues/128
2026-09-11 21:40:12 +08:00
ginuerzh 30f5a0b2c8 docs: add the pending issue and plan write-ups
Sixteen issue analyses, fourteen feature plans, and three superpowers
specs/plans that were sitting untracked.
2026-09-11 20:51:35 +08:00
ginuerzh 5bf06ef9ac docs(plans): drop the plans moved to the p2p repo
These four p2p phase plans now live in the p2p repo's docs/, alongside the
work they describe.
2026-09-11 20:50:46 +08:00
ginuerzh 8c255d8ec4 docs(plans): add the p2p tun link plan
Design and implementation notes for moving the tun-to-tun link out of the
p2p host: GOST owns the device, the host supplies a datagram tunnel. Records
how each review finding was closed and what the e2e run covered.
2026-09-11 20:43:00 +08:00
ginuerzh 54ea0220a9 feat(p2p): datagram tunnel endpoints for tun-to-tun links
OpenTunnel carries a network: "udp" asks the plugin for a datagram endpoint
(one datagram per write) instead of a byte stream, which is what a tun
handler needs — a stream endpoint coalesces and splits packets. tcp stays
the default, so existing deployments are untouched.

The tun handler could not reach client mode without a service forwarder, so
a chain with no forwarder now means client mode with an empty dial address:
the link is transparent and the peer comes from the chain node (a p2p node's
addr is its peer key). That combination used to be dead config, so it is
logged. A forwarder still wins when present, and server mode is unchanged.

Alongside: the udp dialer no longer hard-asserts *net.UDPConn (a wrapped
base dialer yields a plain conn), udp joins the p2p dialer whitelist, and a
udp tunnel announces its client with an empty datagram right after dialing —
the endpoint learns where to send peer packets only from a packet it
receives, so without it a link whose peer speaks first would never come up.

Requires plugin v0.7.0 for the new field.
2026-09-11 20:42:29 +08:00
ginuerzh f70b114578 plugin: manage external plugin subprocess lifecycle
Add Command []string to PluginConfig and a new x/plugin package that
spawns plugin subprocesses (deduplicated by argv) and terminates them
on Shutdown. Linux uses Pdeathsig, Windows a kill-on-close job object,
so children follow an abrupt parent exit.

Wire Spawn into every config/parsing plugin constructor.
2026-09-07 17:40:20 +08:00
ginuerzh 4cd6cd5c30 add p2p M2 hole-punch plan (STUN + KCP + smux direct path) 2026-09-07 15:16:48 +08:00
ginuerzh dcd02f8d52 fix(quic): don't close shared transport on invalid datagram
cipherConn.ReadFrom returned decrypt errors straight to quic-go, which
treats any packet-socket error as fatal and closes the shared QUIC
transport. A single malformed datagram (short, wrong key, tampered tag)
thus killed the whole listener while the process kept running.

Discard unauthenticated/invalid peer datagrams and keep reading; only
surface real socket errors and bad local AES keys. Also fix WriteTo to
return the caller-supplied length instead of the encrypted nonce+tag
length, per the net.PacketConn contract.
2026-09-06 23:03:35 +08:00
ginuerzh d7f0b88086 docs(plans): add p2p mux + token plan (phase one) 2026-09-06 13:35:18 +08:00
ginuerzh 71c1566b00 docs(plans): add p2p DERP relay plan (phase two) 2026-09-06 13:34:24 +08:00
ginuerzh 8127c99153 feat(p2p): lazy tunnel open, whitelist the tls/ws/mux inner dialer family
The tunnel is now opened inside the base dialer on first use: mux
inners (mtcp/mtls/mws) hit their session cache without touching the
base, so a cached session spends no OpenTunnel and leaks no tunnel
per dial. SupportedDialer grows to tcp/tls/ws/mtcp/mtls/mws (each
verified end to end; tls/ws handshakes trigger via the forwarded
Handshaker, mux inners reuse one tunnel as a session). Error paths
release the tunnel exactly once: base-dial failure resets the base
state, inner failure after a successful base dial is closed by the
wrapper, tunnelConn close clears the base state.

Plugin bumped to v0.6.1; QUIC evolution note (transparency principle,
no automatic inner substitution) recorded in the stub plan.
2026-09-06 00:17:10 +08:00
ginuerzh 967053c690 fix(mtcp): re-init session in Handshake when it has no mux session yet
A session Dialed but not yet Handshaken has session.session == nil; a
concurrent Dial treated it as dead (IsClosed() is true for nil sessions),
deleted it and closed the base conn under the first caller's Handshake.
Re-initializing on the nil-session case self-heals instead of failing
the identity check.
2026-09-06 00:17:10 +08:00
ginuerzh fe9d9c9574 add p2p plugin-backed tunnel dialer for chain nodes
Any dialer can now declare its base transport to run through a tunnel
opened by a named p2p plugin component: a top-level p2ps: section
registers TunnelProviders (gRPC plugin), and chain node metadata
"p2p: <name>" wraps the node's dialer so it dials the plugin's
OpenTunnel endpoint instead of the network directly. The inner dialer
keeps its protocol (tls/ws/mux) and Handshake, which the wrapper
forwards so it still triggers at Transport.Handshake.

- x/p2p: tunnelDialer (fail-closed, per-dial tunnel with idempotent
  CloseTunnel on conn close), SupportedDialer whitelist (tcp only;
  fail closed for unverified inner dialers), gRPC TunnelProvider
- config: P2PConfig, parsing, loader registration (reload-safe via
  unregister+Close), mergeConfig wiring
- dialer wrapper registered before Init so the chain inits once
- pin github.com/go-gost/plugin v0.6.0 (p2p proto); dependency bumps
  are a consequence of the new import
2026-09-05 21:25:13 +08:00
ginuerzh dadbbdf5c7 docs(plans): harden p2p stub plan after review
- inner dialer whitelist becomes prerequisite 3 (fail-closed, tcp only)
- record dead-session tunnel-recall fix and the known IsClosed race
- loopback default bind declared as the security boundary for the
  unauthenticated control channel (active-dial SSRF surface)
- contract scope narrowed to TCP-semantic byte pipes; UDP/datagram
  engines (hole punching) are an explicit future collision point
2026-09-05 16:39:27 +08:00
ginuerzh c210b74106 fix(mtcp,mws,mtls): close dead session's base conn before dropping it
Dial() deleted a dead muxSession without closing session.conn, leaking
the base TCP conn on every dead session (and, once a p2p base conn is
injected, leaking the plugin-side tunnel). session.Close() is a no-op
when session.session is nil, so the base conn is closed explicitly.
2026-09-05 16:39:27 +08:00
ginuerzh dac42ba39c fix(tun,tungo): release tun/tap device on listener close (gost#901)
The tun/tungo listeners never tore down the active device on Close, so
reloading (API config change or SIGHUP) with the same device name hit
"device or resource busy": the kernel kept the old interface because its
fd was still open, and the listen loop leaked devices created around
shutdown, spamming errors and blocking SIGTERM.

Track the accepted conn on the listener and close it (idempotently via
sync.Once folding cancel + fd close) from both the loop's shutdown branch
and Close(), with a send-race guard so a device created just as Close runs
is still released. Closing the fd removes the interface, so rebind with the
same name now succeeds and the process exits cleanly.
2026-09-05 11:48:52 +08:00
ginuerzh 53269265e1 refactor(config): swap yaml.v3 encoder for goccy/go-yaml
gopkg.in/yaml.v3 is archived/unmaintained upstream. Its only use in x is
Config.Write()'s YAML encoding, which is replaced with the already-present
goccy/go-yaml (compiled in via gin) — no new dependency. goccy output is
wire-equivalent (same keys/structure/omitempty); viper reads it back
identically, confirmed by the config round-trip tests. yaml.v3 drops to an
indirect dep (retained only by the inspector module in the workspace).
2026-09-02 18:00:43 +08:00
ginuerzh cfccd22fc9 refactor(ss,tap): migrate tap off upstream go-shadowsocks2 onto the fork
Drop the classic github.com/shadowsocks/go-shadowsocks2 dependency. tap and
internal/util/ss were its only remaining users after the ss/ssu handlers
moved to the go-gost fork. The fork's shadowaead is byte-identical on wire
(classic AEAD: same md5-KDF, HKDF-SHA1 subkey, salt sizes), so tap now uses
a small stateless ss.AEADPacketConn built on fork Pack/Unpack, and
ss.ShadowCipher becomes a fork builder (dummy/none -> existing none.Cipher).
Removes the undocumented tap handler key metadata and the dead ShadowConn
helper in the process.
2026-09-02 17:42:09 +08:00
ginuerzh de301e8cfb refactor(handler/http): drop govalidator dep, inline DNS name check
Replace the sole govalidator.IsDNSName call in normalizeRequest with a local
isDNSName helper that mirrors the library's exact behavior (empty/size guard,
net.ParseIP exclusion, and the same label regex), removing the
asaskevich/govalidator dependency from x.
2026-09-02 17:01:58 +08:00
ginuerzh e119a973f2 refactor(routing): drop predicate dep, parse matcher rules via go/ast
Replace the vulcand/predicate dependency in the routing rules parser with a
direct go/parser/go/ast implementation covering the same grammar (matcher
calls, parentheses, &&/||/!). This removes the predicate -> gravitational/
trace -> logrus logging chain from x, since trace was only used to build
errors. Public API (NewParser, Parser, TreeBuilder, Parse, CheckRule,
ParseMatchers) is unchanged and the existing test suite passes as-is.
2026-09-02 16:58:35 +08:00
ginuerzh 66960e7dd6 fix(redirect): skip pre-sniffing IP bypass check in whitelist mode
Domain-only whitelist bypass never matched the bare destination IP, so the
pre-sniffing bypass check (fe394a8) rejected connections before SNI sniffing
could match the sniffed host (gost#899).

- skip the pre-sniffing check when bypass IsWhitelist(); the sniffer applies
  the whitelist against the sniffed host
- restore the post-sniffing bypass check on dstAddr (removed by fe394a8) so
  whitelist mode still rejects non-matching hosts on the sniffingFallback
  path and when sniffing is disabled
- bypassGroup.IsWhitelist() now reports true only for all-whitelist groups
  instead of always false (sole external consumer is the redirect handler)
- rewrite handler tests to drive a real TLS ClientHello through Init()'d
  metadata: whitelisted SNI is forwarded, non-whitelisted SNI gets ErrBypass
2026-08-31 21:45:28 +08:00
ginuerzh 14fed91c62 httpcache: guard Turbo short-circuit responses, add StoreKey
Cacheable() now refuses X-Turbo-Short-Circuit responses (partial <meta>
payloads meant to merge client-side, would serve blank pages on later
hits). StoreKey() lets callers whose request identity is rewritten before
store (e.g. mirror Host routing) pin the key so Lookup and Store agree.

Bump github.com/go-gost/core to v0.6.1.
2026-08-29 09:23:26 +08:00
ginuerzh 2299c43bb7 forwarder: HTTP header rewrite (regex + plugin) and hostPattern wiring
Implement the header counterpart of the existing body rewrite:

- rewriteHeaderBlock applies rewrite rules to an http.Header in place.
  Regex mode matches header names case-insensitively (per HTTP spec) and
  rewrites each value independently; an empty replacement drops the value,
  and a fully-empty header is deleted. Plugin mode serializes the header
  block (http.Header.Write + trailing CRLF, since ReadMIMEHeader needs the
  blank line), sends it to a core/rewriter.Rewriter plugin, and parses the
  result back with clear+copy (h is a map; reassignment would not escape).
- rewrite_metadata.go consolidates the hand-rolled plugin metadata maps
  (baseMetadata + three SSE inline maps) into rewriteMeta() with common
  sid/direction/uri fields and a new kind (body|header), additive for
  existing plugins.
- sniffer_http.go wires rewriteReqHeader before the request is written
  upstream (and refreshes the recorder header clone) and rewriteRespHeader
  after static response-header overrides, before the recorder clone.
- hostPattern: derive req.Host from the request path via regex submatch
  expansion (Host = $1-style template), for Host-routed mirror origins.
- parse: parseHeaderRewrites gates on the config string (regexp.Compile("")
  yields a non-nil match-all, so gating on the compiled regexp would turn
  an empty name into a silent no-op rule).

Fixes found by e2e: reqHeaderRewrites was assigned after its guard, so the
request header rewrite never ran; tests cover the header round-trip,
plugin gating, metadata kind, and hostPattern expansion.
2026-08-29 08:37:44 +08:00
ginuerzh dd65d46926 fix(handler/http): sync Gost-Target override into req.URL.Host to prevent policy bypass
A valid Gost-Target / X-Gost-Target header replaced only req.Host while
leaving req.URL.Host untouched. Destination-policy checks (bypass) evaluated
the header-derived authority, but http.Transport dialed the unchanged
absolute-URL authority. A client could label a forbidden URL with an allowed
authority and reach the blocked destination (403 -> 200 with backend body).

setRequestHost now applies the decoded target to both req.Host and req.URL.Host
so policy evaluation and transport dialing always resolve to the same authority.
2026-08-27 20:16:22 +08:00
ginuerzh 2e9befc39c fix(handler/http): resolve origin-form requests via Host header (#679)
Origin-form requests ("GET / HTTP/1.1" with only a Host header, as
sent by nginx proxy_pass to an upstream proxy) failed RoundTrip with
"http: no Host in request URL": normalizeRequest infers URL.Scheme
but leaves URL.Host empty, and http.Transport requires a non-empty
URL.Host. Also affected every keep-alive request after the first,
which bypass normalizeRequest entirely.

Fill scheme+Host from req.Host in the shared proxyRoundTrip path,
covering both initial and subsequent requests.

e2e-verified: raw origin-form request proxied by Host header
(gost/tests/e2e TestHTTPProxyOriginForm).
2026-08-27 00:02:13 +08:00
ginuerzh 93bd2ea760 fix(hop): parenthesize host OR and normalize wildcard in filter->matcher rule
filterToMatcherRule produced `Host(x) || Host(.x) && Proto(...) && ...`,
but the matcher DSL (vulcand/predicate) binds && tighter than ||, so the
apex Host(x) branch bypassed the protocol/path conjuncts — a request to
the apex with a non-matching protocol/path matched where the legacy
filter (AND of host/protocol/path) rejected it.

Also normalize `*.example.com` to `.example.com` before the leading-dot
branch: legacy ParseNode did this so both forms matched apex+subdomains;
without it, `*.example.com` reached Host() as a wildcard matching only
subdomains, dropping apex coverage that `.example.com` still had.

- Parenthesize the host OR when protocol/path conjuncts follow.
- Strip leading `*` + ensure leading dot, mirroring legacy normalization.
- Update TestFilterToMatcherRule expectations; add dot/star combined cases.
- Add TestFilterToMatcherRule_Precedence: runtime guard that apex+wrong
  proto/path is rejected and apex/subdomain+valid is accepted, for both
  `.example.com` and `*.example.com` forms.
2026-08-26 21:02:50 +08:00
ginuerzh 9292e0af3e fix(tunnel, forward): route tunneled hostname through hop filter.host
The tunnel bind conns (bindConn/bindUDPConn) now expose their metadata (the
tunneled hostname) via the xctx.Context interface, so the service accept loop
forwards it into the handler context. handleRawForwarding selects a hop node by
that hostname when a hop is configured, so a node with a matching filter.host is
picked; otherwise it falls back to dialing the hostname directly. Fixes
reverse-proxy-tunnel TCP routing (gost#898) where the docs example failed with
"node not available" because the mux path never set the host metadata.
2026-08-26 20:28:37 +08:00
ginuerzh 6ad7269644 refactor(hop): normalize deprecated node filter into matcher at parse time
Fold the deprecated per-node filter (host/protocol/path) into an equivalent
matcher DSL rule during config parsing, so hop.Select keeps a single
filtering path (routing.Matcher). Legacy filter configs stay accepted.

- filterToMatcherRule() maps host->Host(), protocol->Proto(), path->PathPrefix();
  a leading-dot host emits `Host(x) || Host(.x)` to preserve the legacy
  apex+subdomain coverage. Converted rules get priority=-1 to keep the
  always-through-selector behavior of the legacy path.
- Remove isEligible/checkHost/checkProtocol/checkPath from x/hop.
  core/ chain.NodeFilterSettings is left untouched (dead but harmless).
- Migrate hop tests to matchers; add TestFilterToMatcherRule.
2026-08-26 20:26:26 +08:00
ginuerzh 4ea4efaba9 fix(forwarder, sniffing): assert BasicProxy recording via captured output
Since 486a7766 HandleHTTP records on an internal clone of the passed
recorder object, TestHandleHTTP_BasicProxy's assertions on the original
ro were always empty, failing every run. Assert SrcAddr/DstAddr on the
captured recorder payload instead, and drain the response body before
closing the net.Pipe so HandleHTTP completes the write and exits cleanly.
2026-08-25 22:17:05 +08:00
ginuerzh faeede7881 feat(forwarder): delete JSON body fields via json-: rewrite match
Add a json-: match prefix to http.rewriteRequestBody/rewriteResponseBody
that deletes the matched JSON field (sjson.DeleteBytes) instead of
replacing it, keeping the existing value-regex conditional semantics.
2026-08-25 21:54:27 +08:00
ginuerzh a38a701dc4 refactor(sniffing): register protocol handlers via init() registries
Replace the 5 duplicated switch statements that dispatch sniffed
connections (forward/local, forward/remote, http CONNECT, sni, relay)
with per-package registries keyed by proto string. Protocol handlers
self-register via init(), following the house registration pattern, so
adding a protocol no longer requires touching every dispatch site.

Unify the two HandleOptions types (forwarder lacked dialTLS; sniffing
lacked hop/node/httpKeepalive) into one exported struct in sniffing;
forwarder aliases it. Unknown protocols fall through to raw forwarding
exactly as before; SSH remains detection-only. redirect/tcp's special
fallback-replay dispatch is intentionally left untouched.

Adds Dispatch helpers and 8 registry tests (register/get/duplicate/
unknown/known) in both packages. Net -151 lines.
2026-08-25 21:35:45 +08:00
ginuerzh 0132cefaec fix(api): save config back to the loaded file path (gost/x#123)
POST /config without an explicit path= always wrote gost.yaml/gost.json to
the process CWD, silently leaving the config loaded via -C untouched. Track
the last file read by ReadFile and default the save target to it, deriving
the output format from the file extension so a JSON config is saved back as
JSON. Falls back to the CWD default when no config file was loaded.
2026-08-21 21:43:17 +08:00
ginuerzh 25cd60d42d fix(connector/ss): own the target addr instead of aliasing the pooled buffer (gost#896)
WrapConn stores the target and only writes it on first I/O, but the
connector encoded it into a bufpool buffer that was returned to the pool
when Connect returned. A later Connect could reuse the backing array and
overwrite the bytes, misrouting the first connection. Allocate the target
per connection instead of sharing a pooled buffer.

Regression test drives two Connect calls and asserts the first conn's
wire target survives a second Connect reusing the pool.
2026-08-21 21:42:43 +08:00
ginuerzh f8ca4908f3 fix(hop): skip failed nodes before priority short-circuit, last-resort fallback
Failed nodes (http.failCodes marker) were still selected when the
priority short-circuit fired or when FailFilter saw a single-node tier,
so failover never happened for a high-priority matcher node returning
429. Check IsFailed first in the candidate-pool scan, and when every
candidate is failed return the first node matching the request gates as
a best-effort last resort instead of nil.

Refactors the fail-code check into shared selector.IsFailed (cee264e)
wired through FailFilterSettingsOption (64ea518).
2026-08-18 23:42:08 +08:00
ginuerzh cee264e184 refactor(selector): share IsFailed dead-node check with FailFilter
Extract the per-item failed-state decision (failure count >= maxFails,
last failure within failTimeout, metadata overrides) from FailFilter
into an exported IsFailed helper, and reuse it in the hop's
pre-selection dead-node filter instead of a local duplicate.
2026-08-18 23:21:34 +08:00
ginuerzh 64ea5184f8 fix(hop): drop marked-failed nodes before priority selection
http.failCodes (and probe) marking had no effect when the failing node
had a strictly higher matcher priority: the priority short-circuit
returned the top node directly, and FailFilter no-ops on a single-node
tier (len(vs) <= 1 guard), so the marked node was always re-selected
and failover never happened.

Pre-filter nodes marked failed (same Count >= maxFails && within
failTimeout criteria as FailFilter) out of the candidate pool before
priority selection. Single-node hops keep their lone node (safety net),
and an all-failed pool returns nil as before.

Wired maxFails/failTimeout from the hop selector config into the hop
via FailFilterSettingsOption.
2026-08-18 23:15:59 +08:00
ginuerzh fc5fc052e4 chore(deps): bump go-shadowsocks2 to v0.1.4
Pulls in the classic-AEAD UDP [addr][payload] response-framing fix that
restores shadowsocks UDP interop with third-party servers/clients
(go-shadowsocks2, shadowsocks-libev). Fixes go-gost/gost#894.
2026-08-18 22:17:20 +08:00
ginuerzh e6aeba8701 fix(listener/redirect/udp): set SO_REUSEADDR on tproxy listener socket (gost#413)
Match the dialUDP socket so per-packet tproxy sockets can bind a
destination port overlapping this listener without "address already in use".
2026-08-18 21:55:12 +08:00
ginuerzh cc02cff00c fix(logger): deduplicate keys across WithFields chains (gost#895)
slog.Logger.With does not deduplicate keys, so chained WithFields calls
(e.g. service -> listener/handler, each setting "kind") emitted duplicate
JSON keys. slogLogger now rebuilds from a pristine base logger with
accumulated attrs deduplicated by key, last write wins.
2026-08-18 21:49:11 +08:00
ginuerzh 18df26e9bc Merge branch 'master' of github.com:go-gost/x 2026-08-17 23:05:14 +08:00
Vincent Yang a9be7e6dce fix(masque): repair server CONNECT-UDP relay (#122)
The MASQUE handler's CONNECT-UDP path was broken in two ways, so no
standards-compliant client (e.g. Surge) could establish UDP tunnels:

1. Missing Capsule-Protocol response header. HTTPStream() flushes the
   response internally and commits the headers, but it was called before
   Header().Set("Capsule-Protocol", "?1"), so the mandatory RFC 9298
   header was silently dropped and clients rejected the tunnel.

2. WriteTo on a pre-connected socket. A direct UDP dial via the router
   returns a *net.UDPConn that satisfies net.PacketConn but is
   pre-connected; the relay called WriteTo on it, which fails with "use
   of WriteTo with pre-connected connection", so no packet reached the
   target.

Set the Capsule-Protocol header before hijacking the stream (dialing the
target first, matching handleConnectTCP), and always wrap the
router-returned conn so writes go through Conn.Write. Verified with a
standards-compliant HTTP/3 MASQUE client: CONNECT-UDP now returns
Capsule-Protocol: ?1 and datagrams relay end to end; CONNECT-TCP
unaffected.
2026-08-17 23:05:00 +08:00
ginuerzhandgithub 0e0eab0c52 fix(cmd): restore unix socket port-forward semantics
Commit b3b5986 crammed the URL path into the listen/connect address and
disabled forward mode for path-based protocols, breaking the
`unix://a.sock/b.sock` (listen on a, forward to b) port-forward form and
producing an invalid `a.sockb.sock` address.

Distinguish the two URL forms by host presence:
- three-slash (unix:///abs.sock, empty host) → path is the absolute socket
  address, no forward target;
- two-slash (unix://a.sock/b.sock) → host is the socket and the path is a
  forward target (service) or ignored (node).

Add regression tests for both forms.

Co-Authored-By: Claude (claude-opus-4-8[1m])
2026-08-17 22:51:00 +08:00
Dean Chen 98e9651361 cmd: keep absolute unix socket paths for path-based schemes (#121)
Path-based address selection used the dialer/listener type after registry
fallback, so unix remapped to tcp skipped EscapedPath and dropped
addrs like /var/run/docker.sock from three-slash URLs.

Decide path-based protocols from the original scheme components.

Signed-off-by: Dean Chen <862469039@qq.com>
2026-08-17 22:29:17 +08:00
PahaKush e4e36ca095 x: stream websocket reads instead of buffering each frame (#120)
websocketConn.Read called Conn.ReadMessage, which is NextReader plus
io.ReadAll. io.ReadAll starts at 512 bytes and grows by doubling, so every
frame allocated a fresh slice and a chain of intermediate ones: a stream of
64KB frames costs about 2.5 bytes of garbage per byte carried, even though
the caller already supplied a buffer to copy into.

On a relay+wss tunnel moving 750MB this was 2526KB of garbage per MB of
payload and 757 GC cycles in 15 seconds against a 1.5MB live heap. An
allocation profile attributed 99.92% of all process garbage to io.ReadAll
under ReadMessage; a CPU profile spent 15% in memclrNoHeapPointers and ~21%
in the collector, against 8% in AES-GCM.

Use NextReader and copy straight into the caller's buffer. The reader is
kept across calls and dropped at end of message, so a message larger than
the caller's buffer is still delivered across successive reads.

This also fixes a zero-length message surfacing as a (0, nil) read, which
io.Copy counts as no progress and rejects after 100 in a row.

Benchmark on 64KB frames:

    before  39267 ns/op  1668.97 MB/s  138444 B/op  19 allocs/op
    after   19920 ns/op  3289.98 MB/s      80 B/op   3 allocs/op
2026-08-17 22:28:48 +08:00
Niek van der Maas 746811eea5 fix: support SOCKS5 UDP associations over MASQUE (#119)
* fix: support SOCKS5 UDP over MASQUE

* fix: complete MASQUE SOCKS5 proxy support

* fix: isolate MASQUE UDP destinations
2026-08-17 22:28:20 +08:00
233 changed files with 26245 additions and 1718 deletions
+1
View File
@@ -15,6 +15,7 @@ package-lock.json
.codegraph/
.claude/
.remember/
.memory/
.build/
__pycache__/
@@ -0,0 +1,27 @@
# SDD ledger — plan: x/docs/superpowers/plans/2026-10-07-tun-device-probe.md
Ruling: sdd-workspace script absent in this skill install (only task-start/task-done exist) — workspace created manually at x/.superpowers/sdd/2026-10-07-tun-device-probe/. Cost if wrong: none, git-ignored scratch.
Pre-flight shared-interface rows:
- T1→T2: md.probe/probeReport + runDeviceProbe signature. Plan pins report func(sentDelta, ackedDelta uint64); runDeviceProbe assumes non-nil (nil guard lives in probeDevice). No conflict.
- T1+T2→T3: h.md.probe/probeReport, runDeviceProbe, handleClient iterCtx/conn/ips. handleClient shape to verify in code recon before wiring.
- T3→T4: only files + green suite; no interface.
Task 1: complete (no commit per plan Global Constraints; tests: go test -count=1 ./handler/tun/ ./observer/stats/ → both ok, tun 14.356s).
Ruling: metadata constructor alias is xmd (github.com/go-gost/x/metadata), not mdx as plan Step 1 guessed — plan told me to grep first, did. Cost if wrong: nil.
Task 2: complete (no commit per plan Global Constraints; tests: go test -count=1 ./handler/tun/ → ok 14.987s).
Ruling: probeInterval/probeRecvTimeout are vars not consts (plan pinned values 30s/5s kept; tests override). Pinned signature runDeviceProbe(ctx,dev,ip,report,log) unchanged. Cost if wrong: nil.
Ruling: wrote runDeviceProbe impl before its loop test, caught it, deleted impl, re-did RED (undefined) → GREEN. Cost if wrong: unproven test.
Ruling: loop tests use atomic counters + polling (net.Pipe Read returns before writer Write/report; plain vars would race). Plan sketch used plain vars. Cost if wrong: flaky test.
Task 3: complete (no commit per plan Global Constraints; tests: go test -count=1 ./handler/tun/ ./observer/stats/ → both ok).
Note: gofmt -l flags observer/stats/wrapper/io_test.go — pre-existing, untouched (not in git status). My files clean; go vet clean.
Ruling: probeDevice takes no log param (uses h.options.Logger, same as keepalive); conn at the wire site IS the tun device (transportClient(ctx, tun=conn, ...) proves it). Cost if wrong: probe writes into the tunnel stream.
Task 4: green to the release gate (build ./... + both suites PASS). STOPPING before tag+push: publish side effect, maintainer-manual per plan + repo no-push norm. Awaiting human: commit + tag v0.23.0.
Final: fresh review via subagent. C1 (UDP length 16→must be 24, acked could never fire on real device) — confirmed real, re-graded Critical stands.
Final: fixed C1 — TestBuildProbePacket UDP-length assertion RED→GREEN, suite handler/tun ok.
Final: fixed I1 (Reset forgot probe counters) — TestStatsProbeReset RED→GREEN, suite observer/stats ok.
Final: minor (deferred): M1 redial overlap — old probe writes ≤5s alongside new after iterCtx cancel; datagram-atomic, self-healing.
Final: minor (deferred): M2 log assumed non-nil in probe paths.
Final: minor (deferred): M3 gate goroutine spawn on probeReport != nil to avoid one throwaway goroutine per redial on gost path.
Final: reviewer declined go toolchain — overruled, tests ran under /root/.local/go/bin (see memory dev-container-toolchain); kernel trim-vs-drop of len-16 packet no longer matters post-fix; water-driver atomicity stands on datagram semantics + keepalive precedent.
Pre-release workspace check (GOWORK=/root/opencode/probework/go.work, x+wisper+gost): go build ./... all three OK; wisper ./tunnel/... PASS (14.6s/15.3s); x handler/tun + observer/stats PASS.
+29 -11
View File
@@ -5,6 +5,8 @@ import (
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"github.com/gin-gonic/gin"
"github.com/go-gost/core/observer/stats"
@@ -127,10 +129,11 @@ func getConfig(ctx *gin.Context) {
// swagger:parameters saveConfigRequest
type saveConfigRequest struct {
// output format, one of yaml|json, default is yaml.
// output format, one of yaml|json, default is derived from the file extension.
// in: query
Format string `form:"format" json:"format"`
// file path, default is gost.yaml|gost.json in current working directory.
// file path, default is the loaded config file, or gost.yaml|gost.json in
// the current working directory when no config file was loaded.
// in: query
Path string `form:"path" json:"path"`
}
@@ -155,16 +158,31 @@ func saveConfig(ctx *gin.Context) {
var req saveConfigRequest
ctx.ShouldBindQuery(&req)
file := "gost.yaml"
switch req.Format {
case "json":
file = "gost.json"
default:
req.Format = "yaml"
file := req.Path
if file == "" {
// Default to the config file that was loaded (via -C), falling back to
// the current working directory when the config came from elsewhere.
file = config.ConfigFile()
}
if req.Path != "" {
file = req.Path
if file == "" {
file = "gost.yaml"
if req.Format == "json" {
file = "gost.json"
}
}
switch strings.ToLower(req.Format) {
case "json":
req.Format = "json"
case "yaml":
req.Format = "yaml"
default:
// Derive the format from the file extension so a loaded JSON config
// is saved back as JSON (ReadFile forces format from extension).
if strings.ToLower(filepath.Ext(file)) == ".json" {
req.Format = "json"
} else {
req.Format = "yaml"
}
}
f, err := os.Create(file)
+34 -40
View File
@@ -44,13 +44,13 @@ var ErrBypass = errors.New("bypass")
type options struct {
// whitelist toggles between blacklist and whitelist mode.
// When false (default): matching addresses bypass the proxy.
// When true: only matching addresses bypass the proxy.
// When true: matching addresses use the proxy and all others bypass it.
whitelist bool
// network restricts this bypass to a specific network protocol.
// When set, the incoming network must match before address matchers
// are evaluated. Recognized values include "tcp" and "udp".
// An empty value disables the network check.
// When set, the incoming network must match as part of the base rule.
// Recognized values include "tcp" and "udp". An empty value
// disables the network check.
network string
// matchers holds static patterns provided at construction time
@@ -89,8 +89,8 @@ func WhitelistOption(whitelist bool) Option {
}
// NetworkOption restricts this bypass to a specific network protocol.
// When set, Contains returns false when the incoming network does not
// match, regardless of address matchers. Accepts values like "tcp" or "udp".
// Network and address predicates are combined before whitelist mode is
// applied to the resulting match. Accepts values like "tcp" or "udp".
func NetworkOption(network string) Option {
return func(opts *options) {
opts.network = network
@@ -427,43 +427,28 @@ func (p *localBypass) Contains(ctx context.Context, network, addr string, opts .
return decision == decisionBypass
}
// decide returns the bypass decision for the given address, applying the
// whitelist or blacklist mode to the pattern match result.
// decide returns the bypass decision for the given address. Network and
// address predicates form one base match, then whitelist mode is applied once
// to that combined result.
func (p *localBypass) decide(network string, addr string) bypassDecision {
p.mu.RLock()
defer p.mu.RUnlock()
if p.options.network != "" && p.options.network != network {
matched := p.options.network == "" || p.options.network == network
if p.patterns != nil {
matched = matched && p.patterns.matchAny(addr)
} else if p.options.network == "" {
// A bypass with no predicates has no base match.
matched = false
}
// Whitelist is a top-level negation of the normal blacklist decision:
// blacklist mode proxies a failed combined match, while whitelist mode
// proxies a successful combined match.
if matched == p.options.whitelist {
return decisionProxy
}
if p.patterns == nil {
if p.options.network != "" {
if p.options.whitelist {
return decisionProxy
}
return decisionBypass
}
return decisionProxy
}
matched := p.patterns.matchAny(addr)
if p.options.whitelist {
// Whitelist mode: the pattern set specifies addresses that MUST use the proxy.
// Matched addresses go through the proxy; unmatched addresses bypass.
if matched {
return decisionProxy
}
return decisionBypass
}
// Blacklist mode: the pattern set specifies addresses that should bypass the proxy.
// Matched addresses bypass; unmatched addresses go through the proxy.
if matched {
return decisionBypass
}
return decisionProxy
return decisionBypass
}
func (p *localBypass) IsWhitelist() bool {
@@ -552,8 +537,17 @@ func (p *bypassGroup) evaluate(ctx context.Context, network, addr string, opts .
return decisionProxy
}
// IsWhitelist always returns false for a group, since the group may contain
// a mix of whitelist and blacklist rules.
// IsWhitelist reports whether the group acts as a pure whitelist: every
// member is a whitelist rule. A mixed group is neither pure whitelist nor
// pure blacklist, so it reports false.
func (p *bypassGroup) IsWhitelist() bool {
return false
if len(p.bypasses) == 0 {
return false
}
for _, bp := range p.bypasses {
if !bp.IsWhitelist() {
return false
}
}
return true
}
+69 -12
View File
@@ -667,6 +667,20 @@ func TestBypassGroup_IsWhitelist(t *testing.T) {
assert.False(t, g.IsWhitelist())
}
// gost#899: in whitelist mode a domain-only allowlist must not reject the
// destination IP before sniffing runs (the sniffer checks the sniffed host).
func TestBypassGroup_WhitelistDomainNoMatch(t *testing.T) {
g := BypassGroup(
NewBypass(
WhitelistOption(true),
MatchersOption([]string{"example.com"}),
LoggerOption(xlogger.Nop()),
),
)
assert.True(t, g.Contains(context.Background(), "tcp", "93.184.216.34:443"))
assert.False(t, g.Contains(context.Background(), "tcp", "example.com:443"))
}
// --- NewBypass reload error in background ---
func TestNewBypass_ReloadError(t *testing.T) {
@@ -751,7 +765,40 @@ func TestContains_NetworkMatch_WhitelistBypass(t *testing.T) {
assert.True(t, b.Contains(context.Background(), "udp", "10.0.0.1"))
}
func TestContains_NetworkMismatch_ReturnsFalse(t *testing.T) {
func TestContains_WhitelistNegatesCombinedMatch(t *testing.T) {
tests := []struct {
name string
whitelist bool
network string
addr string
wantProxy bool
}{
{name: "blacklist both match", network: "tcp", addr: "192.168.1.1", wantProxy: false},
{name: "blacklist network fails", network: "udp", addr: "192.168.1.1", wantProxy: true},
{name: "blacklist matcher fails", network: "tcp", addr: "10.0.0.1", wantProxy: true},
{name: "blacklist both fail", network: "udp", addr: "10.0.0.1", wantProxy: true},
{name: "whitelist both match", whitelist: true, network: "tcp", addr: "192.168.1.1", wantProxy: true},
{name: "whitelist network fails", whitelist: true, network: "udp", addr: "192.168.1.1", wantProxy: false},
{name: "whitelist matcher fails", whitelist: true, network: "tcp", addr: "10.0.0.1", wantProxy: false},
{name: "whitelist both fail", whitelist: true, network: "udp", addr: "10.0.0.1", wantProxy: false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
b := newSyncedBypass(
WhitelistOption(tt.whitelist),
NetworkOption("tcp"),
MatchersOption([]string{"192.168.1.1"}),
)
defer b.Close()
gotProxy := !b.Contains(context.Background(), tt.network, tt.addr)
assert.Equal(t, tt.wantProxy, gotProxy)
})
}
}
func TestContains_NetworkMismatch_BlacklistReturnsFalse(t *testing.T) {
b := newSyncedBypass(
NetworkOption("tcp"),
MatchersOption([]string{"192.168.1.1"}),
@@ -775,7 +822,7 @@ func TestContains_NetworkMismatch_EmptyAddr(t *testing.T) {
assert.False(t, b.Contains(context.Background(), "tcp", ""))
}
func TestContains_NetworkMismatch_WhitelistReturnsFalse(t *testing.T) {
func TestContains_NetworkMismatch_WhitelistBypasses(t *testing.T) {
b := newSyncedBypass(
WhitelistOption(true),
NetworkOption("tcp"),
@@ -783,9 +830,9 @@ func TestContains_NetworkMismatch_WhitelistReturnsFalse(t *testing.T) {
)
defer b.Close()
// Network does not match → bypass returns false even in whitelist mode
assert.False(t, b.Contains(context.Background(), "udp", "10.0.0.1"))
assert.False(t, b.Contains(context.Background(), "udp", "192.168.1.1"))
// Network fails the combined predicate, then whitelist mode negates it.
assert.True(t, b.Contains(context.Background(), "udp", "10.0.0.1"))
assert.True(t, b.Contains(context.Background(), "udp", "192.168.1.1"))
}
func TestContains_PatternSetPassesThroughNetworkFilter(t *testing.T) {
@@ -832,15 +879,15 @@ func TestNetworkOnly_Whitelist_ProxiesMatchingNetwork(t *testing.T) {
assert.False(t, b.Contains(context.Background(), "tcp", "10.0.0.1"))
}
func TestNetworkOnly_Whitelist_NonMatchingNotAffected(t *testing.T) {
func TestNetworkOnly_Whitelist_NonMatchingBypasses(t *testing.T) {
b := newSyncedBypass(
WhitelistOption(true),
NetworkOption("tcp"),
)
defer b.Close()
// network: tcp 是作用域限定,非 TCP 流量不受影响
assert.False(t, b.Contains(context.Background(), "udp", "192.168.1.1"))
// The network predicate fails, then whitelist mode negates it.
assert.True(t, b.Contains(context.Background(), "udp", "192.168.1.1"))
}
func TestNetworkOnly_NoNetwork_BehavesAsBefore(t *testing.T) {
@@ -852,6 +899,16 @@ func TestNetworkOnly_NoNetwork_BehavesAsBefore(t *testing.T) {
assert.False(t, b.Contains(context.Background(), "udp", "192.168.1.1"))
}
func TestNetworkOnly_NoNetwork_WhitelistBypassesAll(t *testing.T) {
b := newSyncedBypass(WhitelistOption(true))
defer b.Close()
// No predicates → base match is false, whitelist negates it → bypass all.
// This flipped from proxy-all before #129; lock in the new behavior.
assert.True(t, b.Contains(context.Background(), "tcp", "192.168.1.1"))
assert.True(t, b.Contains(context.Background(), "udp", "192.168.1.1"))
}
func TestNetworkOnly_EmptyAddr_ReturnsFalse(t *testing.T) {
b := newSyncedBypass(NetworkOption("tcp"))
defer b.Close()
@@ -1022,9 +1079,9 @@ func TestPatternSet_MatchAny_Empty(t *testing.T) {
func TestPatternSet_MatchAny_MixedTypes(t *testing.T) {
ps := classifyPatterns([]string{
"192.168.1.1", // address
"10.0.0.0/8", // CIDR
"*.example.com", // wildcard
"192.168.1.1", // address
"10.0.0.0/8", // CIDR
"*.example.com", // wildcard
"172.16.0.1-172.16.0.255", // IP range
}, xlogger.Nop())
assert.True(t, ps.matchAny("192.168.1.1"))
@@ -1129,7 +1186,7 @@ func TestDecide_NilPatterns(t *testing.T) {
func TestEvaluate_AllBlacklistAllMatch(t *testing.T) {
g := BypassGroup(alwaysContains{}, alwaysContains{}).(*bypassGroup)
assert.Equal(t, decisionBypass, g.evaluate(context.Background(), "tcp", "any", ))
assert.Equal(t, decisionBypass, g.evaluate(context.Background(), "tcp", "any"))
}
func TestEvaluate_BlacklistNoneMatch(t *testing.T) {
+38 -21
View File
@@ -397,15 +397,24 @@ func buildServiceConfig(url *url.URL) ([]*config.ServiceConfig, error) {
listener = schemes[1]
}
// For path-based protocols (Unix socket listeners and serial), use path as address
// Decide path-based from the scheme before registry remaps listener to tcp.
isPathBasedProtocol := listener == "unix" || listener == "runix" || listener == "serial"
for _, s := range schemes {
if s == "unix" || s == "runix" || s == "serial" {
isPathBasedProtocol = true
break
}
}
// A path-based protocol with an empty host (three-slash URL like
// unix:///var/run/sock) uses the path as the absolute listen address and
// has no forward target. With a non-empty host (two-slash URL like
// unix://sock/other.sock), the host is the listen address and any path is a
// forward target (port-forward to another socket), handled below.
isPathAddr := isPathBasedProtocol && url.Host == ""
var addrs []string
if isPathBasedProtocol {
path := url.EscapedPath()
if url.Host != "" {
addrs = append(addrs, url.Host+strings.TrimPrefix(path, "/"))
} else if path != "" {
if isPathAddr {
if path := url.EscapedPath(); path != "" {
addrs = append(addrs, path)
} else {
addrs = append(addrs, url.Host)
@@ -436,8 +445,10 @@ func buildServiceConfig(url *url.URL) ([]*config.ServiceConfig, error) {
var nodes []*config.ForwardNodeConfig
// forward mode
// For path-based protocols, path is address not forward target
if remotes := strings.Trim(url.EscapedPath(), "/"); remotes != "" && !isPathBasedProtocol {
// A path-based protocol with an empty host (e.g. unix:///abs.sock) treats
// the path as the listen address, not a forward target. With a host
// (e.g. unix://a.sock/b.sock) the path is a forward target.
if remotes := strings.Trim(url.EscapedPath(), "/"); remotes != "" && !isPathAddr {
i := 0
for _, addr := range strings.Split(remotes, ",") {
addrs := xnet.AddrPortRange(addr).Addrs()
@@ -640,6 +651,18 @@ func buildNodeConfig(url *url.URL, m map[string]any) (*config.NodeConfig, error)
dialer = schemes[1]
}
// Path-based address logic must use the requested scheme, not the
// registry fallback (unix/serial often remap to http/tcp when only
// the CLI registry is partially populated, or in unit tests).
isPathBasedProtocol := dialer == "unix" || dialer == "serial" ||
connector == "unix" || connector == "serial"
for _, s := range schemes {
if s == "unix" || s == "serial" || s == "runix" {
isPathBasedProtocol = true
break
}
}
md := mdx.NewMetadata(m)
if c := registry.ConnectorRegistry().Get(connector); c == nil {
@@ -725,20 +748,14 @@ func buildNodeConfig(url *url.URL, m map[string]any) (*config.NodeConfig, error)
nodeMd[k] = v
}
// For path-based protocols (Unix socket dialers and serial), use path as address
// For path-based protocols (Unix socket dialers and serial), use path as address.
// Three-slash URLs (empty host) give the absolute socket path; two-slash URLs
// use the host as the socket. The path on a host form is not part of the node
// address (it is a forward target for services, ignored for nodes).
var nodeAddr string
isPathBasedProtocol := dialer == "unix" || dialer == "serial"
if isPathBasedProtocol {
path := url.EscapedPath()
if url.Host != "" {
// Two slashes: host + path = relative path
nodeAddr = url.Host + strings.TrimPrefix(path, "/")
} else if path != "" {
// Three slashes: path is absolute (keep leading slash)
nodeAddr = path
} else {
nodeAddr = url.Host
}
if isPathBasedProtocol && url.Host == "" {
// Three slashes: path is absolute (keep leading slash)
nodeAddr = url.EscapedPath()
} else {
nodeAddr = url.Host
}
+62 -6
View File
@@ -1765,10 +1765,10 @@ func TestBuildNodeConfig_TLSWithCert(t *testing.T) {
}
}
func TestBuildNodeConfig_UnixFallsBackToTCP(t *testing.T) {
func TestBuildNodeConfig_UnixAbsolutePath(t *testing.T) {
// http+unix scheme: connector=http, dialer=unix.
// Since unix is not in the test registry, dialer falls back to tcp,
// and the path-based address logic (dialer=="unix") is skipped.
// Dialer falls back to tcp when unix is not in the test registry, but
// the absolute socket path from a three-slash URL must still be kept.
rawURL := "http+unix:///var/run/socket"
u, _ := url.Parse(rawURL)
@@ -1780,12 +1780,68 @@ func TestBuildNodeConfig_UnixFallsBackToTCP(t *testing.T) {
if node.Dialer.Type != "tcp" {
t.Errorf("Dialer.Type = %q, want tcp (unix not in test registry)", node.Dialer.Type)
}
// Addr is url.Host ("") since no host component in three-slash unix URL
if node.Addr != "" {
t.Errorf("Addr = %q, want empty", node.Addr)
if node.Addr != "/var/run/socket" {
t.Errorf("Addr = %q, want /var/run/socket", node.Addr)
}
}
func TestBuildNodeConfig_UnixRelativeHost(t *testing.T) {
u, _ := url.Parse("unix://docker.sock")
node, err := buildNodeConfig(u, map[string]any{})
if err != nil {
t.Fatalf("buildNodeConfig: %v", err)
}
if node.Addr != "docker.sock" {
t.Errorf("Addr = %q, want docker.sock", node.Addr)
}
}
func TestBuildServiceConfig_UnixAbsolutePath(t *testing.T) {
// three-slash: path is the absolute socket listen address, no forward target
svcs, err := buildServiceConfig(mustURL("unix:///var/run/gost.sock"))
if err != nil {
t.Fatalf("buildServiceConfig: %v", err)
}
if len(svcs) != 1 {
t.Fatalf("len(svcs) = %d, want 1", len(svcs))
}
if svcs[0].Addr != "/var/run/gost.sock" {
t.Errorf("Addr = %q, want /var/run/gost.sock", svcs[0].Addr)
}
if svcs[0].Forwarder != nil {
t.Errorf("Forwarder = %+v, want nil (no forward target)", svcs[0].Forwarder)
}
}
func TestBuildServiceConfig_UnixPortForward(t *testing.T) {
// two-slash: host is the listen socket, path is a forward target socket
svcs, err := buildServiceConfig(mustURL("unix://gost.sock/gost2.sock"))
if err != nil {
t.Fatalf("buildServiceConfig: %v", err)
}
if len(svcs) != 1 {
t.Fatalf("len(svcs) = %d, want 1", len(svcs))
}
if svcs[0].Addr != "gost.sock" {
t.Errorf("Addr = %q, want gost.sock", svcs[0].Addr)
}
if svcs[0].Forwarder == nil || len(svcs[0].Forwarder.Nodes) == 0 {
t.Fatalf("Forwarder = %+v, want a forward target node", svcs[0].Forwarder)
}
if got := svcs[0].Forwarder.Nodes[0].Addr; got != "gost2.sock" {
t.Errorf("forward Addr = %q, want gost2.sock", got)
}
}
func mustURL(raw string) *url.URL {
u, err := url.Parse(raw)
if err != nil {
panic(err)
}
return u
}
func TestBuildServiceConfig_RelayTLSForward(t *testing.T) {
// relay+tls scheme: handler=relay, listener=tls.
// relay not in test registry → handler falls back to auto, listener falls back to tcp.
+46 -3
View File
@@ -7,11 +7,12 @@ import (
"reflect"
"strings"
"sync"
"sync/atomic"
"time"
yaml "github.com/goccy/go-yaml"
"github.com/mitchellh/mapstructure"
"github.com/spf13/viper"
"gopkg.in/yaml.v3"
)
var (
@@ -28,8 +29,21 @@ func init() {
var (
global = &Config{}
globalMux sync.RWMutex
// configFile remembers the last file loaded via ReadFile so the API can
// save back to the same location (issue go-gost/x#123).
configFile atomic.Value // string
)
// ConfigFile returns the path of the last config file loaded via ReadFile,
// or an empty string when the config came from stdin, a URL, or inline JSON.
func ConfigFile() string {
if s, ok := configFile.Load().(string); ok {
return s
}
return ""
}
func Global() *Config {
globalMux.RLock()
defer globalMux.RUnlock()
@@ -134,6 +148,7 @@ type TLSOptions struct {
type PluginConfig struct {
Type string `json:"type"`
Addr string `json:"addr"`
Command []string `yaml:"command,omitempty" json:"command,omitempty"`
TLS *TLSConfig `yaml:",omitempty" json:"tls,omitempty"`
Timeout time.Duration `yaml:",omitempty" json:"timeout,omitempty"`
Token string `yaml:",omitempty" json:"token,omitempty"`
@@ -287,6 +302,13 @@ type RecorderConfig struct {
Plugin *PluginConfig `yaml:",omitempty" json:"plugin,omitempty"`
}
// P2PConfig defines a named p2p tunnel provider, currently backed by an
// external gRPC plugin process.
type P2PConfig struct {
Name string `json:"name"`
Plugin *PluginConfig `yaml:",omitempty" json:"plugin,omitempty"`
}
type FileRecorder struct {
Path string `json:"path"`
Sep string `yaml:",omitempty" json:"sep,omitempty"`
@@ -450,6 +472,17 @@ type HTTPBodyRewriteConfig struct {
MaxChunkSize int `yaml:"maxChunkSize,omitempty" json:"maxChunkSize,omitempty"`
}
type HTTPHeaderRewriteConfig struct {
// Name is the header name regex, e.g. "(?i)^(location|set-cookie)$".
Name string
// Match is the header value regex.
Match string
// Replacement is the replacement string.
Replacement string
// Rewriter is the name of the rewriter plugin (via registry).
Rewriter string `yaml:",omitempty" json:"rewriter,omitempty"`
}
type NodeFilterConfig struct {
Host string `yaml:",omitempty" json:"host,omitempty"`
Protocol string `yaml:",omitempty" json:"protocol,omitempty"`
@@ -491,6 +524,9 @@ type ProbeConfig struct {
type HTTPNodeConfig struct {
// rewrite host header
Host string `yaml:",omitempty" json:"host,omitempty"`
// regex matched against the request path to derive Host; Host becomes a
// replacement template ($1, $2, ...) when set.
HostPattern string `yaml:"hostPattern,omitempty" json:"hostPattern,omitempty"`
// Deprecated: use requestHeader instead
Header map[string]string `yaml:",omitempty" json:"header,omitempty"`
// additional request header
@@ -508,6 +544,11 @@ type HTTPNodeConfig struct {
// rewrite response body
RewriteResponseBody []HTTPBodyRewriteConfig `yaml:"rewriteResponseBody,omitempty" json:"rewriteResponseBody,omitempty"`
// rewrite request header values
RewriteRequestHeader []HTTPHeaderRewriteConfig `yaml:"rewriteRequestHeader,omitempty" json:"rewriteRequestHeader,omitempty"`
// rewrite response header values
RewriteResponseHeader []HTTPHeaderRewriteConfig `yaml:"rewriteResponseHeader,omitempty" json:"rewriteResponseHeader,omitempty"`
// comma-separated response status codes marking the node failed, e.g. "429,5xx"
FailCodes string `yaml:"failCodes,omitempty" json:"failCodes,omitempty"`
@@ -733,6 +774,7 @@ type Config struct {
Services []*ServiceConfig `json:"services"`
Chains []*ChainConfig `yaml:",omitempty" json:"chains,omitempty"`
Hops []*HopConfig `yaml:",omitempty" json:"hops,omitempty"`
Nodes []*NodeConfig `yaml:",omitempty" json:"nodes,omitempty"`
Authers []*AutherConfig `yaml:",omitempty" json:"authers,omitempty"`
Admissions []*AdmissionConfig `yaml:",omitempty" json:"admissions,omitempty"`
Bypasses []*BypassConfig `yaml:",omitempty" json:"bypasses,omitempty"`
@@ -743,6 +785,7 @@ type Config struct {
SDs []*SDConfig `yaml:"sds,omitempty" json:"sds,omitempty"`
Recorders []*RecorderConfig `yaml:",omitempty" json:"recorders,omitempty"`
Rewriters []*RewriterConfig `yaml:",omitempty" json:"rewriters,omitempty"`
P2Ps []*P2PConfig `yaml:"p2ps,omitempty" json:"p2ps,omitempty"`
Caches []*CacheConfig `yaml:",omitempty" json:"caches,omitempty"`
Limiters []*LimiterConfig `yaml:",omitempty" json:"limiters,omitempty"`
Quotas []*QuotaConfig `yaml:",omitempty" json:"quotas,omitempty"`
@@ -780,6 +823,7 @@ func (c *Config) ReadFile(file string) error {
if err := v.ReadInConfig(); err != nil {
return err
}
configFile.Store(file)
return v.Unmarshal(c, viper.DecodeHook(gostDecodeHook()))
}
@@ -793,9 +837,8 @@ func (c *Config) Write(w io.Writer, format string) error {
case "yaml":
fallthrough
default:
enc := yaml.NewEncoder(w)
enc := yaml.NewEncoder(w, yaml.Indent(2))
defer enc.Close()
enc.SetIndent(2)
return enc.Encode(c)
}
+36 -2
View File
@@ -6,6 +6,9 @@
package loader
import (
"fmt"
"strings"
"github.com/go-gost/core/admission"
"github.com/go-gost/core/auth"
"github.com/go-gost/core/bypass"
@@ -22,8 +25,8 @@ import (
"github.com/go-gost/core/recorder"
reg "github.com/go-gost/core/registry"
"github.com/go-gost/core/resolver"
"github.com/go-gost/core/router"
"github.com/go-gost/core/rewriter"
"github.com/go-gost/core/router"
"github.com/go-gost/core/sd"
"github.com/go-gost/x/config"
"github.com/go-gost/x/config/parsing"
@@ -38,6 +41,7 @@ import (
limiter_parser "github.com/go-gost/x/config/parsing/limiter"
logger_parser "github.com/go-gost/x/config/parsing/logger"
observer_parser "github.com/go-gost/x/config/parsing/observer"
p2p_parser "github.com/go-gost/x/config/parsing/p2p"
quota_parser "github.com/go-gost/x/config/parsing/quota"
recorder_parser "github.com/go-gost/x/config/parsing/recorder"
resolver_parser "github.com/go-gost/x/config/parsing/resolver"
@@ -46,6 +50,7 @@ import (
sd_parser "github.com/go-gost/x/config/parsing/sd"
service_parser "github.com/go-gost/x/config/parsing/service"
quota "github.com/go-gost/x/limiter/quota"
xp2p "github.com/go-gost/x/p2p"
"github.com/go-gost/x/registry"
)
@@ -258,6 +263,16 @@ func register(cfg *config.Config) error {
}
}
{
var entries []named[xp2p.Tunnel]
for _, c := range cfg.P2Ps {
entries = append(entries, named[xp2p.Tunnel]{c.Name, p2p_parser.ParseP2P(c)})
}
if err := registerGroup(entries, registry.P2PRegistry()); err != nil {
return err
}
}
{
var entries []named[cache.Cache]
for _, c := range cfg.Caches {
@@ -308,7 +323,26 @@ func register(cfg *config.Config) error {
}
}
// --- hops (references bypasses, resolvers, hosts from registries) ---
// --- nodes (definitions referenced by hops) ---
{
var entries []named[*config.NodeConfig]
for _, c := range cfg.Nodes {
if c == nil {
continue
}
c.Name = strings.TrimSpace(c.Name)
if c.Name == "" {
return fmt.Errorf("global node name is required")
}
entries = append(entries, named[*config.NodeConfig]{c.Name, c})
}
if err := registerGroup(entries, registry.NodeRegistry()); err != nil {
return err
}
}
// --- hops (references nodes, bypasses, resolvers, hosts from registries) ---
{
var entries []named[hop.Hop]
+75
View File
@@ -10,6 +10,7 @@ import (
"testing"
"github.com/go-gost/core/handler"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/listener"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/metadata"
@@ -17,6 +18,9 @@ import (
"github.com/go-gost/x/config"
xlogger "github.com/go-gost/x/logger"
"github.com/go-gost/x/registry"
_ "github.com/go-gost/x/connector/http"
_ "github.com/go-gost/x/dialer/tcp"
)
func TestMain(m *testing.M) {
@@ -465,6 +469,77 @@ func TestRegister_Hops(t *testing.T) {
}
}
func TestRegister_GlobalNodesBeforeHops(t *testing.T) {
nodeName := "test-global-node"
hopName := "test-global-node-hop"
t.Cleanup(func() {
registry.HopRegistry().Unregister(hopName)
registry.NodeRegistry().Unregister(nodeName)
})
cfg := &config.Config{
Nodes: []*config.NodeConfig{{
Name: nodeName,
Addr: "example.com:8080",
}},
Hops: []*config.HopConfig{{
Name: hopName,
Nodes: []*config.NodeConfig{{Name: nodeName}},
}},
}
if err := register(cfg); err != nil {
t.Fatalf("unexpected error: %v", err)
}
definition := registry.NodeRegistry().Get(nodeName)
if definition == nil {
t.Fatal("expected global node to be registered")
}
nodes := registry.HopRegistry().Get(hopName).(hop.NodeList).Nodes()
if len(nodes) != 1 || nodes[0].Name != definition.Name || nodes[0].Addr != definition.Addr {
t.Fatalf("hop nodes = %v, want global definition %q at %q", nodes, definition.Name, definition.Addr)
}
}
func TestRegister_GlobalNodeReferenceFromYAML(t *testing.T) {
const nodeName = "yaml-global-node"
const hopName = "yaml-global-hop"
t.Cleanup(func() {
registry.HopRegistry().Unregister(hopName)
registry.NodeRegistry().Unregister(nodeName)
})
cfg := &config.Config{}
err := cfg.Read(strings.NewReader(`
nodes:
- name: yaml-global-node
addr: global.example:8080
hops:
- name: yaml-global-hop
nodes:
- name: yaml-global-node
- name: inline-node
addr: inline.example:8081
`), "yaml")
if err != nil {
t.Fatalf("read YAML: %v", err)
}
if err := register(cfg); err != nil {
t.Fatalf("register: %v", err)
}
nodes := registry.HopRegistry().Get(hopName).(hop.NodeList).Nodes()
if len(nodes) != 2 {
t.Fatalf("hop nodes = %d, want 2", len(nodes))
}
if nodes[0].Name != nodeName || nodes[0].Addr != "global.example:8080" {
t.Fatalf("referenced node = %q at %q", nodes[0].Name, nodes[0].Addr)
}
if nodes[1].Name != "inline-node" || nodes[1].Addr != "inline.example:8081" {
t.Fatalf("inline node = %q at %q", nodes[1].Name, nodes[1].Addr)
}
}
func TestRegister_Chains(t *testing.T) {
r := registry.ChainRegistry()
name := "test-chain"
+4
View File
@@ -11,6 +11,7 @@ import (
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
"github.com/go-gost/x/registry"
)
@@ -24,6 +25,9 @@ func ParseAdmission(cfg *config.AdmissionConfig) admission.Admission {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+4
View File
@@ -15,6 +15,7 @@ import (
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
"github.com/go-gost/x/registry"
)
@@ -27,6 +28,9 @@ func ParseAuther(cfg *config.AutherConfig) auth.Authenticator {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+4
View File
@@ -11,6 +11,7 @@ import (
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
"github.com/go-gost/x/registry"
)
@@ -23,6 +24,9 @@ func ParseBypass(cfg *config.BypassConfig) bypass.Bypass {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+26
View File
@@ -19,6 +19,8 @@ import (
"github.com/go-gost/x/internal/plugin"
"github.com/go-gost/x/metadata"
mdutil "github.com/go-gost/x/metadata/util"
xplugin "github.com/go-gost/x/plugin"
xs "github.com/go-gost/x/selector"
)
// ParseHop converts a HopConfig into a hop.Hop. It resolves plugin backends
@@ -31,6 +33,9 @@ func ParseHop(cfg *config.HopConfig, log logger.Logger) (hop.Hop, error) {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
@@ -83,6 +88,16 @@ func ParseHop(cfg *config.HopConfig, log logger.Logger) (hop.Hop, error) {
continue
}
var err error
v, err = node_parser.ResolveConfig(v)
if err != nil {
// Fail the whole hop: static config is validated at startup,
// so a dangling reference is a hard error. (Contrast with
// chainHop.parseNode, which warns and skips bad nodes to
// keep hot-reloaded hops resilient.)
return nil, err
}
// Build a merged metadata map for inheritance without mutating
// the original node config.
merged := make(map[string]any)
@@ -158,10 +173,21 @@ func ParseHop(cfg *config.HopConfig, log logger.Logger) (hop.Hop, error) {
sel = selector_parser.DefaultNodeSelector()
}
maxFails, timeout := xs.DefaultMaxFails, xs.DefaultFailTimeout
if cfg.Selector != nil {
if cfg.Selector.MaxFails > 0 {
maxFails = cfg.Selector.MaxFails
}
if cfg.Selector.FailTimeout > 0 {
timeout = cfg.Selector.FailTimeout
}
}
opts := []xhop.Option{
xhop.NameOption(cfg.Name),
xhop.NodeOption(nodes...),
xhop.SelectorOption(sel),
xhop.FailFilterSettingsOption(maxFails, timeout),
xhop.BypassOption(xbypass.BypassGroup(bypass_parser.List(cfg.Bypass, cfg.Bypasses...)...)),
xhop.ReloadPeriodOption(cfg.Reload),
xhop.LoggerOption(log.WithFields(map[string]any{
+77 -1
View File
@@ -4,13 +4,14 @@ import (
"io"
"testing"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/x/config"
"github.com/go-gost/x/config/parsing"
xlogger "github.com/go-gost/x/logger"
mdutil "github.com/go-gost/x/metadata/util"
"github.com/go-gost/x/registry"
// Register connector and dialer implementations needed for node parsing.
_ "github.com/go-gost/x/connector/http"
@@ -185,6 +186,81 @@ func TestParseHop_WithNodeNilEntries(t *testing.T) {
}
}
func TestParseHop_GlobalNodeReference(t *testing.T) {
global := &config.NodeConfig{Name: "global-node", Addr: "example.com:8080"}
registry.NodeRegistry().Register(global.Name, global)
t.Cleanup(func() { registry.NodeRegistry().Unregister(global.Name) })
h, err := ParseHop(&config.HopConfig{
Name: "reference-hop",
Nodes: []*config.NodeConfig{{Name: global.Name}},
}, testLogger())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
nodes := h.(hop.NodeList).Nodes()
if len(nodes) != 1 || nodes[0].Name != global.Name || nodes[0].Addr != global.Addr {
t.Fatalf("referenced nodes = %v, want node %q at %q", nodes, global.Name, global.Addr)
}
}
func TestParseHop_GlobalNodeReferenceNotFound(t *testing.T) {
_, err := ParseHop(&config.HopConfig{
Name: "reference-hop",
Nodes: []*config.NodeConfig{{Name: "missing-global-node"}},
}, testLogger())
if err == nil {
t.Fatal("expected missing global node error")
}
}
func TestParseHop_NodeWithAdditionalFieldRemainsInline(t *testing.T) {
global := &config.NodeConfig{Name: "shared-name", Addr: "global.example:8080"}
registry.NodeRegistry().Register(global.Name, global)
t.Cleanup(func() { registry.NodeRegistry().Unregister(global.Name) })
h, err := ParseHop(&config.HopConfig{
Name: "inline-hop",
Nodes: []*config.NodeConfig{{
Name: global.Name,
Addr: "inline.example:8080",
}},
}, testLogger())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
nodes := h.(hop.NodeList).Nodes()
if len(nodes) != 1 {
t.Fatalf("expected 1 node, got %d", len(nodes))
}
if nodes[0].Addr != "inline.example:8080" {
t.Fatalf("node addr = %q, want inline node address", nodes[0].Addr)
}
}
func TestParseHop_GlobalNodeCreatesIndependentRuntimeNodes(t *testing.T) {
global := &config.NodeConfig{Name: "independent-node", Addr: "example.com:8080"}
registry.NodeRegistry().Register(global.Name, global)
t.Cleanup(func() { registry.NodeRegistry().Unregister(global.Name) })
parse := func(name string) *chain.Node {
h, err := ParseHop(&config.HopConfig{
Name: name, Nodes: []*config.NodeConfig{{Name: global.Name}},
}, testLogger())
if err != nil {
t.Fatalf("ParseHop: %v", err)
}
return h.(hop.NodeList).Nodes()[0]
}
if first, second := parse("hop-1"), parse("hop-2"); first == second {
t.Fatal("global node references must create independent runtime nodes")
}
if global.Connector != nil || global.Dialer != nil {
t.Fatal("parsing a reference must not mutate the global definition")
}
}
func TestParseHop_SockOptsMark(t *testing.T) {
h, err := ParseHop(&config.HopConfig{
Name: "sockopts-hop",
+4
View File
@@ -12,6 +12,7 @@ import (
hosts_plugin "github.com/go-gost/x/hosts/plugin"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
)
// ParseHostMapper converts a HostsConfig into a hosts.HostMapper. It resolves
@@ -23,6 +24,9 @@ func ParseHostMapper(cfg *config.HostsConfig) hosts.HostMapper {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+4
View File
@@ -11,6 +11,7 @@ import (
ingress_plugin "github.com/go-gost/x/ingress/plugin"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
)
// ParseIngress converts an IngressConfig into an ingress.Ingress. It resolves
@@ -22,6 +23,9 @@ func ParseIngress(cfg *config.IngressConfig) ingress.Ingress {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+4
View File
@@ -15,6 +15,7 @@ import (
xrate "github.com/go-gost/x/limiter/rate"
xtraffic "github.com/go-gost/x/limiter/traffic"
traffic_plugin "github.com/go-gost/x/limiter/traffic/plugin"
xplugin "github.com/go-gost/x/plugin"
)
// ParseTrafficLimiter converts a LimiterConfig into a traffic.TrafficLimiter.
@@ -26,6 +27,9 @@ func ParseTrafficLimiter(cfg *config.LimiterConfig) (lim traffic.TrafficLimiter)
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+38 -22
View File
@@ -3,7 +3,6 @@ package logger
import (
"io"
"os"
"path/filepath"
"github.com/go-gost/core/logger"
"github.com/go-gost/x/config"
@@ -25,8 +24,34 @@ func ParseLogger(cfg *config.LoggerConfig) logger.Logger {
xlogger.LevelOption(logger.LogLevel(cfg.Log.Level)),
}
opts = append(opts, xlogger.OutputOption(outputWriter(cfg.Log)))
return xlogger.NewLogger(opts...)
}
// defaultLogRotation is what a file log gets when its config leaves the
// rotation out. It matches what an unconfigured logger has always carried, so
// a log: section that names an output but no rotation is not a way to ask for
// an unbounded file.
//
// The reason this is a default rather than a warning: an unrotated debug log
// is not untidy, it stops the device. A spoke's log section with no rotation:
// key grew to 111MB with no backups, on a phone.
func defaultLogRotation() *config.LogRotationConfig {
return &config.LogRotationConfig{
MaxSize: 10,
MaxAge: 7,
MaxBackups: 10,
LocalTime: true,
}
}
// outputWriter resolves the LogConfig's output to the writer that receives the
// lines. Split out of ParseLogger so the rotation decision — the part that
// decides whether a file log stops growing — is reachable without a logger.
func outputWriter(cfg *config.LogConfig) io.Writer {
var out io.Writer = os.Stderr
switch cfg.Log.Output {
switch cfg.Output {
case "none", "null":
out = io.Discard
case "stdout":
@@ -34,28 +59,19 @@ func ParseLogger(cfg *config.LoggerConfig) logger.Logger {
case "stderr", "":
out = os.Stderr
default:
if cfg.Log.Rotation != nil {
out = &lumberjack.Logger{
Filename: cfg.Log.Output,
MaxSize: cfg.Log.Rotation.MaxSize,
MaxAge: cfg.Log.Rotation.MaxAge,
MaxBackups: cfg.Log.Rotation.MaxBackups,
LocalTime: cfg.Log.Rotation.LocalTime,
Compress: cfg.Log.Rotation.Compress,
}
} else {
os.MkdirAll(filepath.Dir(cfg.Log.Output), 0755)
f, err := os.OpenFile(cfg.Log.Output, os.O_CREATE|os.O_WRONLY|os.O_APPEND, 0666)
if err != nil {
logger.Default().Warn(err)
} else {
out = f
}
if cfg.Rotation == nil {
cfg.Rotation = defaultLogRotation()
}
out = &lumberjack.Logger{
Filename: cfg.Output,
MaxSize: cfg.Rotation.MaxSize,
MaxAge: cfg.Rotation.MaxAge,
MaxBackups: cfg.Rotation.MaxBackups,
LocalTime: cfg.Rotation.LocalTime,
Compress: cfg.Rotation.Compress,
}
}
opts = append(opts, xlogger.OutputOption(out))
return xlogger.NewLogger(opts...)
return out
}
// List resolves one or more logger names from the registry. It returns only
+123 -1
View File
@@ -1,16 +1,28 @@
package logger
import (
"fmt"
"io"
"os"
"os/exec"
"path/filepath"
"testing"
"github.com/go-gost/core/logger"
"github.com/go-gost/x/config"
xlogger "github.com/go-gost/x/logger"
"gopkg.in/natefinch/lumberjack.v2"
)
// probeEnv marks the child process TestParseLogger_UnopenableOutput spawns:
// that test needs a process in which no default logger was ever installed,
// which is the state the failure path used to dereference.
const probeEnv = "X_LOGGER_UNOPENABLE_PROBE"
func TestMain(m *testing.M) {
logger.SetDefault(xlogger.NewLogger(xlogger.OutputOption(io.Discard)))
if os.Getenv(probeEnv) != "1" {
logger.SetDefault(xlogger.NewLogger(xlogger.OutputOption(io.Discard)))
}
m.Run()
}
@@ -140,3 +152,113 @@ func TestList_WithNames(t *testing.T) {
t.Fatal("expected empty list for unregistered names")
}
}
// TestParseLogger_UnopenableOutput: a log file that cannot be opened must leave
// the process logging somewhere, not panic. The failure used to be reported
// through logger.Default(), which is nil until a caller installs one — and this
// call is usually the one doing the installing — so an unwritable path (an app
// whose working directory is not writable, a typo'd directory) dereferenced a
// nil logger and took the whole process down.
//
// The state that matters is process-wide and TestMain sets it for every other
// case here, so the assertion runs in a child process: a fresh one is the only
// place the default logger is still unset, exactly as it is at startup.
func TestParseLogger_UnopenableOutput(t *testing.T) {
if os.Getenv(probeEnv) == "1" {
lg := ParseLogger(&config.LoggerConfig{Log: &config.LogConfig{
Output: "/nonexistent-dir-for-test/wisper.log",
Level: "info",
}})
if lg == nil {
fmt.Fprintln(os.Stderr, "ParseLogger returned nil for an unopenable output")
os.Exit(1)
}
return
}
cmd := exec.Command(os.Args[0], "-test.run=TestParseLogger_UnopenableOutput", "-test.v")
cmd.Env = append(os.Environ(), probeEnv+"=1")
if out, err := cmd.CombinedOutput(); err != nil {
t.Fatalf("a log file that cannot be opened must not be fatal:\n%s", out)
}
}
// TestParseLoggerUnrotatedFileRotates is the defect that filled a phone's
// storage. A log: section without a rotation: key used to get a plain
// *os.File — nothing rotates it — and one debug-level spoke grew it to 111MB
// with no backups, which on a phone is not untidy so much as fatal.
//
// The default rotation is what the unconfigured logger already carries, so
// naming one should not be a way to opt out of it.
func TestParseLoggerUnrotatedFileRotates(t *testing.T) {
file := filepath.Join(t.TempDir(), "wisper.log")
cfg := &config.LoggerConfig{
Log: &config.LogConfig{
Output: file,
Level: "debug",
Format: "json",
// Rotation deliberately absent: the shape wisper.yaml on Android has.
},
}
w := outputWriter(cfg.Log)
if w == nil {
t.Fatal("outputWriter = nil")
}
if _, ok := w.(*lumberjack.Logger); !ok {
t.Fatalf("outputWriter = %T, want a *lumberjack.Logger so the file rotates", w)
}
lg := w.(*lumberjack.Logger)
if lg.Filename != file {
t.Errorf("Filename = %q, want %q", lg.Filename, file)
}
if lg.MaxSize <= 0 || lg.MaxBackups <= 0 {
t.Errorf("rotation is off: MaxSize=%d MaxBackups=%d, want both positive", lg.MaxSize, lg.MaxBackups)
}
}
// TestParseLoggerExplicitRotationWins: a configured rotation is respected as
// written, so the default is only a floor for what was left out.
func TestParseLoggerExplicitRotationWins(t *testing.T) {
file := filepath.Join(t.TempDir(), "wisper.log")
cfg := &config.LoggerConfig{
Log: &config.LogConfig{
Output: file,
Rotation: &config.LogRotationConfig{
MaxSize: 33,
MaxBackups: 5,
},
},
}
lg, ok := outputWriter(cfg.Log).(*lumberjack.Logger)
if !ok {
t.Fatalf("outputWriter = %T, want *lumberjack.Logger", outputWriter(cfg.Log))
}
if lg.MaxSize != 33 || lg.MaxBackups != 5 {
t.Errorf("MaxSize=%d MaxBackups=%d, want 33 and 5 as configured", lg.MaxSize, lg.MaxBackups)
}
}
// TestParseLoggerNotAFileStillGoesToTheSink: stderr/stdout/none are unaffected,
// so defaulting the rotation cannot leak a file into a stream.
func TestParseLoggerNotAFileStillGoesToTheSink(t *testing.T) {
for _, out := range []string{"stderr", "stdout", "none", ""} {
w := outputWriter(&config.LogConfig{Output: out})
switch out {
case "none":
if w != io.Discard {
t.Errorf("none: writer = %T, want io.Discard", w)
}
case "stdout":
if w != os.Stdout {
t.Errorf("stdout: writer = %T, want os.Stdout", w)
}
default:
if w != os.Stderr {
t.Errorf("%q: writer = %T, want os.Stderr", out, w)
}
}
}
}
+312 -47
View File
@@ -22,6 +22,7 @@ import (
tls_util "github.com/go-gost/x/internal/util/tls"
mdx "github.com/go-gost/x/metadata"
mdutil "github.com/go-gost/x/metadata/util"
xp2p "github.com/go-gost/x/p2p"
"github.com/go-gost/x/registry"
"github.com/go-gost/x/routing"
)
@@ -36,6 +37,204 @@ const DefaultMatcherBodySize = 1 << 20 // 1MB
// when a node opts in to body matching.
const MaxMatcherBodySize = 10 << 20 // 10MB
// IsReference reports whether cfg contains only a non-empty name. Such a
// config references a definition from the top-level nodes section.
// Any additional field, even a zero-valued one set explicitly (e.g. an
// empty but non-nil connector), makes the config inline.
// NOTE: keep this in sync with NodeConfig — a new field must be added here,
// otherwise a config setting only that field would misresolve as a reference.
func IsReference(cfg *config.NodeConfig) bool {
if cfg == nil || strings.TrimSpace(cfg.Name) == "" {
return false
}
return cfg.Addr == "" && cfg.Network == "" && cfg.Bypass == "" &&
len(cfg.Bypasses) == 0 && cfg.Resolver == "" && cfg.Hosts == "" &&
cfg.Connector == nil && cfg.Dialer == nil &&
cfg.Interface == "" && cfg.Netns == "" && cfg.SockOpts == nil &&
cfg.Filter == nil && cfg.Matcher == nil && cfg.HTTP == nil &&
cfg.TLS == nil && cfg.Probe == nil && len(cfg.Metadata) == 0
}
// ResolveConfig resolves a name-only config through the global node registry
// and returns an independent copy of the definition. Inline configs are
// returned unchanged to preserve their existing parsing behavior.
func ResolveConfig(cfg *config.NodeConfig) (*config.NodeConfig, error) {
if cfg == nil {
return nil, nil
}
if !IsReference(cfg) {
return cfg, nil
}
name := strings.TrimSpace(cfg.Name)
cfg = registry.NodeRegistry().Get(name)
if cfg == nil {
return nil, fmt.Errorf("node %q not found", name)
}
return cloneConfig(cfg), nil
}
// cloneConfig returns a deep copy of cfg so that parsing (hop-level
// inheritance, connector/dialer type defaults, TLS ServerName defaulting)
// only ever mutates the copy, never the shared global definition.
// NOTE: keep this in sync with NodeConfig — a new reference-type field
// must be cloned here, otherwise hops would share it with the registry.
func cloneConfig(cfg *config.NodeConfig) *config.NodeConfig {
c := *cfg
c.Bypasses = append([]string(nil), cfg.Bypasses...)
if cfg.Metadata != nil {
c.Metadata = make(map[string]any, len(cfg.Metadata))
for k, v := range cfg.Metadata {
c.Metadata[k] = v
}
}
if cfg.Connector != nil {
connector := *cfg.Connector
connector.Auth = cloneAuth(cfg.Connector.Auth)
connector.TLS = cloneTLSConfig(cfg.Connector.TLS)
if cfg.Connector.Metadata != nil {
connector.Metadata = make(map[string]any, len(cfg.Connector.Metadata))
for k, v := range cfg.Connector.Metadata {
connector.Metadata[k] = v
}
}
c.Connector = &connector
}
if cfg.Dialer != nil {
dialer := *cfg.Dialer
dialer.Auth = cloneAuth(cfg.Dialer.Auth)
dialer.TLS = cloneTLSConfig(cfg.Dialer.TLS)
if cfg.Dialer.Metadata != nil {
dialer.Metadata = make(map[string]any, len(cfg.Dialer.Metadata))
for k, v := range cfg.Dialer.Metadata {
dialer.Metadata[k] = v
}
}
c.Dialer = &dialer
}
if cfg.SockOpts != nil {
sockOpts := *cfg.SockOpts
c.SockOpts = &sockOpts
}
if cfg.Filter != nil {
filter := *cfg.Filter
c.Filter = &filter
}
if cfg.Matcher != nil {
matcher := *cfg.Matcher
c.Matcher = &matcher
}
if cfg.HTTP != nil {
http := *cfg.HTTP
http.Header = cloneStringMap(cfg.HTTP.Header)
http.RequestHeader = cloneStringMap(cfg.HTTP.RequestHeader)
http.ResponseHeader = cloneStringMap(cfg.HTTP.ResponseHeader)
http.Rewrite = append([]config.HTTPURLRewriteConfig(nil), cfg.HTTP.Rewrite...)
http.RewriteURL = append([]config.HTTPURLRewriteConfig(nil), cfg.HTTP.RewriteURL...)
http.RewriteBody = append([]config.HTTPBodyRewriteConfig(nil), cfg.HTTP.RewriteBody...)
http.RewriteRequestBody = append([]config.HTTPBodyRewriteConfig(nil), cfg.HTTP.RewriteRequestBody...)
http.RewriteResponseBody = append([]config.HTTPBodyRewriteConfig(nil), cfg.HTTP.RewriteResponseBody...)
http.RewriteRequestHeader = append([]config.HTTPHeaderRewriteConfig(nil), cfg.HTTP.RewriteRequestHeader...)
http.RewriteResponseHeader = append([]config.HTTPHeaderRewriteConfig(nil), cfg.HTTP.RewriteResponseHeader...)
http.Auth = cloneAuth(cfg.HTTP.Auth)
c.HTTP = &http
}
if cfg.TLS != nil {
tls := *cfg.TLS
if cfg.TLS.Options != nil {
options := *cfg.TLS.Options
options.CipherSuites = append([]string(nil), cfg.TLS.Options.CipherSuites...)
options.ALPN = append([]string(nil), cfg.TLS.Options.ALPN...)
tls.Options = &options
}
c.TLS = &tls
}
if cfg.Probe != nil {
probe := *cfg.Probe
probe.HTTPHeaders = cloneStringMap(cfg.Probe.HTTPHeaders)
c.Probe = &probe
}
return &c
}
func cloneAuth(cfg *config.AuthConfig) *config.AuthConfig {
if cfg == nil {
return nil
}
auth := *cfg
return &auth
}
func cloneTLSConfig(cfg *config.TLSConfig) *config.TLSConfig {
if cfg == nil {
return nil
}
tls := *cfg
tls.ServerNames = append([]string(nil), cfg.ServerNames...)
if cfg.Options != nil {
options := *cfg.Options
options.CipherSuites = append([]string(nil), cfg.Options.CipherSuites...)
options.ALPN = append([]string(nil), cfg.Options.ALPN...)
tls.Options = &options
}
return &tls
}
func cloneStringMap(m map[string]string) map[string]string {
if m == nil {
return nil
}
c := make(map[string]string, len(m))
for k, v := range m {
c[k] = v
}
return c
}
// filterToMatcherRule converts a deprecated NodeFilterConfig (host/protocol/
// path) into an equivalent matcher DSL rule. Empty fields are omitted; an
// empty result means the node is unconditional. Path maps to PathPrefix
// (the legacy filter is a prefix match, not exact).
func filterToMatcherRule(filter *config.NodeFilterConfig) string {
if filter == nil {
return ""
}
var parts []string
// Normalize the legacy host forms first: `*.example.com` and
// `.example.com` both meant "apex AND subdomains" in the legacy filter.
// Strip the wildcard so the leading-dot branch below handles both.
host := filter.Host
if strings.HasPrefix(host, "*") {
host = host[1:]
if !strings.HasPrefix(host, ".") {
host = "." + host
}
}
if host != "" {
// A leading-dot host matches the apex AND its subdomains in the
// legacy filter (HasSuffix on the dot-stripped form), but matcher
// Host(.x) only matches subdomains. Emit both to preserve apex
// coverage. Parenthesize the OR when other conjuncts follow, since
// && binds tighter than || in the matcher DSL.
if strings.HasPrefix(host, ".") {
h := "Host(`" + host[1:] + "`) || Host(`" + host + "`)"
if filter.Protocol != "" || filter.Path != "" {
h = "(" + h + ")"
}
parts = append(parts, h)
} else {
parts = append(parts, "Host(`"+host+"`)")
}
}
if prot := filter.Protocol; prot != "" {
parts = append(parts, "Proto(`"+prot+"`)")
}
if path := filter.Path; path != "" {
parts = append(parts, "PathPrefix(`"+path+"`)")
}
return strings.Join(parts, " && ")
}
// ParseNode converts a NodeConfig into a *chain.Node. It resolves the
// connector and dialer from their registries, applies TLS settings, extracts
// metadata-driven options (so_mark, interface, netns, proxy protocol), sets up
@@ -48,8 +247,16 @@ func parseBodyRewrites(vs []config.HTTPBodyRewriteConfig, log logger.Logger) []c
var pattern *regexp.Regexp
var rewriteType string
if js, ok := strings.CutPrefix(v.Match, "json:"); ok {
// json:<path> or json:<path>=<value-regex>
js, ok := strings.CutPrefix(v.Match, "json:")
del := false
if !ok {
// json-:<path> deletes the matched JSON field instead of replacing it.
if js, ok = strings.CutPrefix(v.Match, "json-:"); ok {
del = true
}
}
if ok {
// json:<path>[=<value-regex>] or json-:<path>[=<value-regex>]
path, valRegex, _ := strings.Cut(js, "=")
if valRegex == "" {
valRegex = ".*"
@@ -60,7 +267,11 @@ func parseBodyRewrites(vs []config.HTTPBodyRewriteConfig, log logger.Logger) []c
log.Warnf("invalid JSON value regex %q for path %q: %v", valRegex, path, err)
continue
}
rewriteType = "json:" + path
if del {
rewriteType = "json-:" + path
} else {
rewriteType = "json:" + path
}
} else {
if v.Match != "" {
pattern, _ = regexp.Compile(v.Match)
@@ -87,6 +298,38 @@ func parseBodyRewrites(vs []config.HTTPBodyRewriteConfig, log logger.Logger) []c
return out
}
func parseHeaderRewrites(vs []config.HTTPHeaderRewriteConfig, log logger.Logger) []chain.HTTPHeaderRewriteSettings {
var out []chain.HTTPHeaderRewriteSettings
for _, v := range vs {
var name, pattern *regexp.Regexp
if v.Name != "" {
name, _ = regexp.Compile(v.Name)
}
if v.Match != "" {
pattern, _ = regexp.Compile(v.Match)
}
rw := chain.HTTPHeaderRewriteSettings{
Name: name,
Pattern: pattern,
Replacement: []byte(v.Replacement),
}
if v.Rewriter != "" {
if !registry.RewriterRegistry().IsRegistered(v.Rewriter) {
log.Warnf("rewriter %q not found in registry for rewrite rule", v.Rewriter)
}
rw.Rewriter = registry.RewriterRegistry().Get(v.Rewriter)
}
// Gate on the config string, not the compiled regexp: regexp.Compile("")
// returns a non-nil match-all regexp, so an empty Name with no rewriter
// would otherwise become a silent no-op rule.
if v.Name != "" || rw.Rewriter != nil {
out = append(out, rw)
}
}
return out
}
func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.Node, error) {
if cfg == nil {
return nil, nil
@@ -181,6 +424,25 @@ func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.No
return nil, fmt.Errorf("unregistered dialer: %s", dialCfg.Type)
}
// p2p: wrap the dialer so the node's base transport (the path to this
// node) goes through a tunnel opened by the named p2p plugin instead of
// a direct network dial. The inner dialer's protocol (tls/ws/mux) runs
// unchanged on top of the tunnel. Only whitelisted dialers qualify:
// the wrapper's contract is "inner dials one plain TCP stream via
// options.Dialer". The wrap happens before Init so the chain inits
// exactly once, through the wrapper.
if name := mdutil.GetString(md, parsing.MDKeyP2P); name != "" {
provider := registry.P2PRegistry().Get(name)
if provider == nil {
return nil, fmt.Errorf("unregistered p2p: %s", name)
}
if !xp2p.SupportedDialer(dialCfg.Type) {
return nil, fmt.Errorf("dialer %q does not support p2p", dialCfg.Type)
}
d = xp2p.NewTunnelDialer(d, provider,
xp2p.WithTunnelNetwork(mdutil.GetString(md, parsing.MDKeyP2PNetwork)))
}
if err := d.Init(mdx.NewMetadata(dialCfg.Metadata)); err != nil {
dialerLogger.Error("init: ", err)
return nil, err
@@ -209,56 +471,53 @@ func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.No
chain.NetworkNodeOption(cfg.Network),
}
if filter := cfg.Filter; filter != nil {
// convert *.example.com to .example.com
// convert *example.com to example.com
host := filter.Host
if strings.HasPrefix(host, "*") {
host = host[1:]
if !strings.HasPrefix(host, ".") {
host = "." + host
}
}
var (
rule string
priority int
bodySize int
)
settings := &chain.NodeFilterSettings{
Protocol: filter.Protocol,
Host: host,
Path: filter.Path,
}
opts = append(opts, chain.NodeFilterOption(settings))
// Deprecated filter: normalize into an equivalent matcher DSL rule so the
// runtime keeps a single filtering logic. Explicit -1 priority opts out of
// auto-priority and the priority short-circuit, preserving the legacy
// filter behavior (always through the selector).
if filter := cfg.Filter; filter != nil {
rule = filterToMatcherRule(filter)
priority = -1
bodySize = DefaultMatcherBodySize
}
if cfg.Matcher != nil {
priority := cfg.Matcher.Priority
if rule := strings.TrimSpace(cfg.Matcher.Rule); rule != "" {
if matcher, err := routing.NewMatcher(rule); err == nil {
log.Debugf("new matcher for node %s with rule %s", cfg.Name, cfg.Matcher.Rule)
// Priority 0 means "use default": automatically set to the
// rule length so longer (more specific) rules outrank shorter
// ones. Use a negative priority to opt out of this behavior
// and always go through the selector.
if priority == 0 {
priority = len(cfg.Matcher.Rule)
}
opts = append(opts, chain.MatcherNodeOption(matcher))
} else {
log.Error(err)
priority = -1
}
}
bodySize := cfg.Matcher.BodySize
if bodySize <= 0 {
bodySize = DefaultMatcherBodySize
} else if bodySize > MaxMatcherBodySize {
bodySize = MaxMatcherBodySize
}
opts = append(opts, chain.MatcherBodySizeNodeOption(bodySize))
opts = append(opts, chain.PriorityNodeOption(priority))
priority = cfg.Matcher.Priority
rule = cfg.Matcher.Rule
bodySize = cfg.Matcher.BodySize
}
if rule = strings.TrimSpace(rule); rule != "" {
if matcher, err := routing.NewMatcher(rule); err == nil {
log.Debugf("new matcher for node %s with rule %s", cfg.Name, rule)
// Priority 0 means "use default": automatically set to the
// rule length so longer (more specific) rules outrank shorter
// ones. Use a negative priority to opt out of this behavior
// and always go through the selector.
if priority == 0 {
priority = len(rule)
}
opts = append(opts, chain.MatcherNodeOption(matcher))
} else {
log.Error(err)
priority = -1
}
}
if bodySize <= 0 {
bodySize = DefaultMatcherBodySize
} else if bodySize > MaxMatcherBodySize {
bodySize = MaxMatcherBodySize
}
opts = append(opts, chain.MatcherBodySizeNodeOption(bodySize))
opts = append(opts, chain.PriorityNodeOption(priority))
if cfg.HTTP != nil {
settings := &chain.HTTPNodeSettings{
Host: cfg.HTTP.Host,
@@ -269,6 +528,10 @@ func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.No
settings.RequestHeader = cfg.HTTP.Header
}
if v := cfg.HTTP.HostPattern; v != "" {
settings.HostPattern, _ = regexp.Compile(v)
}
if auth := cfg.HTTP.Auth; auth != nil && auth.Username != "" {
settings.Auther = xauth.NewAuthenticator(
xauth.AuthsOption(map[string]string{auth.Username: auth.Password}),
@@ -295,6 +558,8 @@ func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.No
settings.RewriteResponseBody = append(settings.RewriteResponseBody, parseBodyRewrites(cfg.HTTP.RewriteBody, log)...)
settings.RewriteResponseBody = append(settings.RewriteResponseBody, parseBodyRewrites(cfg.HTTP.RewriteResponseBody, log)...)
settings.RewriteRequestBody = append(settings.RewriteRequestBody, parseBodyRewrites(cfg.HTTP.RewriteRequestBody, log)...)
settings.RewriteRequestHeader = parseHeaderRewrites(cfg.HTTP.RewriteRequestHeader, log)
settings.RewriteResponseHeader = parseHeaderRewrites(cfg.HTTP.RewriteResponseHeader, log)
if v := strings.TrimSpace(cfg.HTTP.FailCodes); v != "" {
settings.FailCodes = parseFailCodes(v, nodeLogger)
+172
View File
@@ -8,6 +8,7 @@ import (
"github.com/go-gost/x/config"
"github.com/go-gost/x/config/parsing"
xlogger "github.com/go-gost/x/logger"
"github.com/go-gost/x/registry"
// Register connector and dialer implementations needed for node parsing.
_ "github.com/go-gost/x/connector/http"
@@ -387,6 +388,44 @@ func TestParseNode_WithHTTPDeprecatedHeader(t *testing.T) {
}
}
func TestParseNode_WithHeaderRewrite(t *testing.T) {
n, err := ParseNode("test-hop", &config.NodeConfig{
Name: "hdr-node",
Addr: "example.com:8080",
HTTP: &config.HTTPNodeConfig{
RewriteRequestHeader: []config.HTTPHeaderRewriteConfig{
{Name: "(?i)^referer$", Match: ".*", Replacement: ""},
},
RewriteResponseHeader: []config.HTTPHeaderRewriteConfig{
{Name: "(?i)^location$", Match: "https://github.com", Replacement: "https://127.0.0.1:8080"},
},
},
}, testLogger())
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if n == nil {
t.Fatal("expected non-nil node")
}
settings := n.Options().HTTP
if settings == nil {
t.Fatal("expected HTTP settings")
}
if len(settings.RewriteRequestHeader) != 1 || len(settings.RewriteResponseHeader) != 1 {
t.Fatalf("rewrite header counts = %d/%d, want 1/1", len(settings.RewriteRequestHeader), len(settings.RewriteResponseHeader))
}
}
func TestParseHeaderRewrites_EmptyNameNoRewriter(t *testing.T) {
// Empty Name with no Rewriter must not be appended (silent no-op guard).
out := parseHeaderRewrites([]config.HTTPHeaderRewriteConfig{
{Name: "", Match: ".*", Replacement: ""},
}, testLogger())
if len(out) != 0 {
t.Fatalf("out = %d rules, want 0", len(out))
}
}
func TestParseNode_WithTLSNode(t *testing.T) {
n, err := ParseNode("test-hop", &config.NodeConfig{
Name: "tls-node",
@@ -501,3 +540,136 @@ func TestParseNode_DeprecatedFilterFields(t *testing.T) {
t.Fatal("expected non-nil node")
}
}
func TestFilterToMatcherRule(t *testing.T) {
tests := []struct {
name string
filter *config.NodeFilterConfig
want string
}{
{"nil", nil, ""},
{"all empty", &config.NodeFilterConfig{}, ""},
{"host only", &config.NodeFilterConfig{Host: "example.com"}, "Host(`example.com`)"},
{"dot host", &config.NodeFilterConfig{Host: ".example.com"},
"Host(`example.com`) || Host(`.example.com`)"},
{"star host", &config.NodeFilterConfig{Host: "*.example.com"},
"Host(`example.com`) || Host(`.example.com`)"},
{"protocol only", &config.NodeFilterConfig{Protocol: "ssh"}, "Proto(`ssh`)"},
{"path only", &config.NodeFilterConfig{Path: "/api"}, "PathPrefix(`/api`)"},
{"combined", &config.NodeFilterConfig{
Host: ".example.com",
Protocol: "http",
Path: "/api",
}, "(Host(`example.com`) || Host(`.example.com`)) && Proto(`http`) && PathPrefix(`/api`)"},
{"star combined", &config.NodeFilterConfig{
Host: "*.example.com",
Protocol: "http",
Path: "/api",
}, "(Host(`example.com`) || Host(`.example.com`)) && Proto(`http`) && PathPrefix(`/api`)"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := filterToMatcherRule(tt.filter); got != tt.want {
t.Errorf("filterToMatcherRule(%v) = %q, want %q", tt.filter, got, tt.want)
}
})
}
}
func TestIsReference(t *testing.T) {
tests := []struct {
name string
cfg *config.NodeConfig
want bool
}{
{"nil", nil, false},
{"empty name", &config.NodeConfig{}, false},
{"blank name", &config.NodeConfig{Name: " "}, false},
{"name only", &config.NodeConfig{Name: "shared"}, true},
{"name and addr", &config.NodeConfig{Name: "shared", Addr: "example.com:80"}, false},
{"name and network", &config.NodeConfig{Name: "shared", Network: "tcp"}, false},
{"name and empty-but-non-nil connector", &config.NodeConfig{Name: "shared", Connector: &config.ConnectorConfig{}}, false},
{"name and empty bypasses slice", &config.NodeConfig{Name: "shared", Bypasses: []string{}}, true},
{"name and matcher", &config.NodeConfig{Name: "shared", Matcher: &config.NodeMatcherConfig{Rule: "Host(`x`)"}}, false},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := IsReference(tt.cfg); got != tt.want {
t.Errorf("IsReference(%+v) = %v, want %v", tt.cfg, got, tt.want)
}
})
}
}
func TestResolveConfig_TrimsName(t *testing.T) {
const name = "spaced-global-node"
registry.NodeRegistry().Register(name, &config.NodeConfig{Name: name, Addr: "example.com:8080"})
t.Cleanup(func() { registry.NodeRegistry().Unregister(name) })
got, err := ResolveConfig(&config.NodeConfig{Name: " " + name + " "})
if err != nil {
t.Fatalf("ResolveConfig: %v", err)
}
if got == nil || got.Addr != "example.com:8080" {
t.Fatalf("resolved = %+v, want addr %q", got, "example.com:8080")
}
}
func TestResolveConfig_DeepCopyIsolation(t *testing.T) {
const name = "rich-global-node"
global := &config.NodeConfig{
Name: name,
Addr: "example.com:8080",
Metadata: map[string]any{"k": "v"},
Connector: &config.ConnectorConfig{
Type: "http",
Auth: &config.AuthConfig{Username: "u"},
Metadata: map[string]any{"ck": "cv"},
TLS: &config.TLSConfig{ServerName: "sni.example", Options: &config.TLSOptions{ALPN: []string{"h2"}}},
},
Dialer: &config.DialerConfig{
Type: "tcp",
Metadata: map[string]any{"dk": "dv"},
},
Matcher: &config.NodeMatcherConfig{Rule: "Host(`x`)"},
HTTP: &config.HTTPNodeConfig{
Host: "h",
RequestHeader: map[string]string{"A": "b"},
Auth: &config.AuthConfig{Username: "hu"},
},
TLS: &config.TLSNodeConfig{ServerName: "tls.example", Options: &config.TLSOptions{MinVersion: "1.2"}},
Probe: &config.ProbeConfig{Type: "tcp", Addr: "example.com:80", HTTPHeaders: map[string]string{"H": "v"}},
}
registry.NodeRegistry().Register(name, global)
t.Cleanup(func() { registry.NodeRegistry().Unregister(name) })
got, err := ResolveConfig(&config.NodeConfig{Name: name})
if err != nil {
t.Fatalf("ResolveConfig: %v", err)
}
// Mutate every nested value on the copy; the global must not change.
got.Metadata["k"] = "mut"
got.Connector.Auth.Username = "mut"
got.Connector.Metadata["ck"] = "mut"
got.Connector.TLS.Options.ALPN[0] = "mut"
got.Dialer.Metadata["dk"] = "mut"
got.Matcher.Rule = "mut"
got.HTTP.RequestHeader["A"] = "mut"
got.HTTP.Auth.Username = "mut"
got.TLS.Options.MinVersion = "mut"
got.Probe.HTTPHeaders["H"] = "mut"
if global.Metadata["k"] != "v" ||
global.Connector.Auth.Username != "u" ||
global.Connector.Metadata["ck"] != "cv" ||
global.Connector.TLS.Options.ALPN[0] != "h2" ||
global.Dialer.Metadata["dk"] != "dv" ||
global.Matcher.Rule != "Host(`x`)" ||
global.HTTP.RequestHeader["A"] != "b" ||
global.HTTP.Auth.Username != "hu" ||
global.TLS.Options.MinVersion != "1.2" ||
global.Probe.HTTPHeaders["H"] != "v" {
t.Fatalf("global definition was mutated via the resolved copy: %+v", global)
}
}
+35
View File
@@ -0,0 +1,35 @@
package node
import (
"testing"
"github.com/go-gost/x/config"
coreRouting "github.com/go-gost/core/routing"
"github.com/go-gost/x/routing"
)
func TestFilterToMatcherRule_Precedence(t *testing.T) {
for _, fc := range []struct{ name, host string }{
{"dot", ".example.com"},
{"star", "*.example.com"},
} {
t.Run(fc.name, func(t *testing.T) {
rule := filterToMatcherRule(&config.NodeFilterConfig{
Host: fc.host, Protocol: "http", Path: "/api",
})
m, err := routing.NewMatcher(rule)
if err != nil {
t.Fatalf("parse %q: %v", rule, err)
}
if m.Match(&coreRouting.Request{Host: "example.com", Protocol: "https", Path: "/other"}) {
t.Errorf("rule %q matched apex+https+/other (should reject)", rule)
}
if !m.Match(&coreRouting.Request{Host: "example.com", Protocol: "http", Path: "/api/v1"}) {
t.Errorf("rule %q rejected apex+http+/api/v1 (should accept)", rule)
}
if !m.Match(&coreRouting.Request{Host: "www.example.com", Protocol: "http", Path: "/api"}) {
t.Errorf("rule %q rejected www+http+/api (should accept)", rule)
}
})
}
}
+5
View File
@@ -4,10 +4,12 @@ import (
"crypto/tls"
"strings"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/observer"
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/plugin"
observer_plugin "github.com/go-gost/x/observer/plugin"
xplugin "github.com/go-gost/x/plugin"
)
// ParseObserver converts an ObserverConfig into an observer.Observer. It only
@@ -18,6 +20,9 @@ func ParseObserver(cfg *config.ObserverConfig) observer.Observer {
return nil
}
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+47
View File
@@ -0,0 +1,47 @@
// Package p2p converts a [config.P2PConfig] into a p2p.Tunnel.
package p2p
import (
"crypto/tls"
"strings"
"github.com/go-gost/core/logger"
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/plugin"
xp2p "github.com/go-gost/x/p2p"
p2p_plugin "github.com/go-gost/x/p2p/plugin"
xplugin "github.com/go-gost/x/plugin"
)
// ParseP2P converts a P2PConfig into a p2p.Tunnel backed by an
// external gRPC plugin. Returns nil when cfg is nil or no plugin is
// configured. The HTTP plugin variant is not implemented; configuring it
// logs a deprecation notice and falls back to gRPC.
func ParseP2P(cfg *config.P2PConfig) (p xp2p.Tunnel) {
if cfg == nil || cfg.Plugin == nil {
return nil
}
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
ServerName: cfg.Plugin.TLS.ServerName,
InsecureSkipVerify: !cfg.Plugin.TLS.Secure,
}
}
switch strings.ToLower(cfg.Plugin.Type) {
case "http":
logger.Default().Warn("p2p: http plugin type is deprecated/unimplemented, falling back to grpc")
fallthrough
default:
return p2p_plugin.NewGRPCPlugin(
cfg.Name, cfg.Plugin.Addr,
plugin.TokenOption(cfg.Plugin.Token),
plugin.TLSConfigOption(tlsCfg),
)
}
}
+10
View File
@@ -96,4 +96,14 @@ const (
// MDKeyLabels holds static key/value labels attached to a service's
// records and logs.
MDKeyLabels = "labels"
// MDKeyP2P selects the p2p tunnel provider for a chain node's base
// transport. The value is the name of a registered p2ps: component.
MDKeyP2P = "p2p"
// MDKeyP2PNetwork overrides the tunnel network a p2p-wrapped node asks the
// provider for (see xp2p.WithTunnelNetwork). Empty derives it from the inner
// dialer. A tun link sets it to "ip": datagram-shaped like udp, but
// session-scoped on the host, so the link ends on a lost peer session.
MDKeyP2PNetwork = "p2p.network"
)
+5
View File
@@ -43,6 +43,7 @@ func TestMergeConfig_AppendsAllSlices(t *testing.T) {
Services: []*config.ServiceConfig{{Name: "s1"}},
Chains: []*config.ChainConfig{{Name: "c1"}},
Hops: []*config.HopConfig{{Name: "h1"}},
Nodes: []*config.NodeConfig{{Name: "n1", Addr: "n1.example:80"}},
Authers: []*config.AutherConfig{{Name: "a1"}},
Admissions: []*config.AdmissionConfig{{Name: "ad1"}},
Bypasses: []*config.BypassConfig{{Name: "b1"}},
@@ -63,6 +64,7 @@ func TestMergeConfig_AppendsAllSlices(t *testing.T) {
Services: []*config.ServiceConfig{{Name: "s2"}},
Chains: []*config.ChainConfig{{Name: "c2"}},
Hops: []*config.HopConfig{{Name: "h2"}},
Nodes: []*config.NodeConfig{{Name: "n2", Addr: "n2.example:80"}},
Authers: []*config.AutherConfig{{Name: "a2"}},
Admissions: []*config.AdmissionConfig{{Name: "ad2"}},
Bypasses: []*config.BypassConfig{{Name: "b2"}},
@@ -91,6 +93,9 @@ func TestMergeConfig_AppendsAllSlices(t *testing.T) {
if len(got.Hops) != 2 {
t.Fatal("hops not properly appended")
}
if len(got.Nodes) != 2 {
t.Fatal("nodes not properly appended")
}
if len(got.Authers) != 2 {
t.Fatal("authers not properly appended")
}
+2
View File
@@ -320,6 +320,7 @@ func mergeConfig(cfg1, cfg2 *config.Config) *config.Config {
Services: append(cfg1.Services, cfg2.Services...),
Chains: append(cfg1.Chains, cfg2.Chains...),
Hops: append(cfg1.Hops, cfg2.Hops...),
Nodes: append(cfg1.Nodes, cfg2.Nodes...),
Authers: append(cfg1.Authers, cfg2.Authers...),
Admissions: append(cfg1.Admissions, cfg2.Admissions...),
Bypasses: append(cfg1.Bypasses, cfg2.Bypasses...),
@@ -329,6 +330,7 @@ func mergeConfig(cfg1, cfg2 *config.Config) *config.Config {
SDs: append(cfg1.SDs, cfg2.SDs...),
Recorders: append(cfg1.Recorders, cfg2.Recorders...),
Rewriters: append(cfg1.Rewriters, cfg2.Rewriters...),
P2Ps: append(cfg1.P2Ps, cfg2.P2Ps...),
Caches: append(cfg1.Caches, cfg2.Caches...),
Limiters: append(cfg1.Limiters, cfg2.Limiters...),
Quotas: append(cfg1.Quotas, cfg2.Quotas...),
+4
View File
@@ -12,6 +12,7 @@ import (
"github.com/go-gost/core/recorder"
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
xrecorder "github.com/go-gost/x/recorder"
recorder_plugin "github.com/go-gost/x/recorder/plugin"
"gopkg.in/natefinch/lumberjack.v2"
@@ -33,6 +34,9 @@ func ParseRecorder(cfg *config.RecorderConfig) (r recorder.Recorder) {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+4
View File
@@ -9,6 +9,7 @@ import (
"github.com/go-gost/core/resolver"
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
"github.com/go-gost/x/registry"
xresolver "github.com/go-gost/x/resolver"
resolver_plugin "github.com/go-gost/x/resolver/plugin"
@@ -24,6 +25,9 @@ func ParseResolver(cfg *config.ResolverConfig) (resolver.Resolver, error) {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+5
View File
@@ -4,9 +4,11 @@ import (
"crypto/tls"
"strings"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/rewriter"
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
rewriter_plugin "github.com/go-gost/x/rewriter/plugin"
)
@@ -19,6 +21,9 @@ func ParseRewriter(cfg *config.RewriterConfig) rewriter.Rewriter {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+4
View File
@@ -10,6 +10,7 @@ import (
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
xrouter "github.com/go-gost/x/router"
router_plugin "github.com/go-gost/x/router/plugin"
)
@@ -23,6 +24,9 @@ func ParseRouter(cfg *config.RouterConfig) router.Router {
}
if cfg.Plugin != nil {
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+5
View File
@@ -4,9 +4,11 @@ import (
"crypto/tls"
"strings"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/sd"
"github.com/go-gost/x/config"
"github.com/go-gost/x/internal/plugin"
xplugin "github.com/go-gost/x/plugin"
sd_plugin "github.com/go-gost/x/sd/plugin"
)
@@ -17,6 +19,9 @@ func ParseSD(cfg *config.SDConfig) sd.SD {
return nil
}
if err := xplugin.Spawn(cfg.Plugin.Command); err != nil {
logger.Default().Errorf("spawn plugin %s: %v", cfg.Name, err)
}
var tlsCfg *tls.Config
if cfg.Plugin.TLS != nil {
tlsCfg = &tls.Config{
+328
View File
@@ -0,0 +1,328 @@
package masque
import (
"context"
"errors"
"net"
"os"
"sync"
"time"
"github.com/go-gost/core/logger"
)
const udpAssociationBufferSize = 64 * 1024
var errMissingDestination = errors.New("masque: destination address required")
type udpAssociationDialFunc func(ctx context.Context, addr net.Addr) (net.PacketConn, error)
type udpAssociationResult struct {
data []byte
addr net.Addr
}
type udpAssociationTunnel struct {
conn net.PacketConn
}
type udpAssociationConn struct {
ctx context.Context
cancel context.CancelFunc
localAddr net.Addr
dial udpAssociationDialFunc
closeIdle func() error
log logger.Logger
timeout time.Duration
closed chan struct{}
closeOnce sync.Once
closeErr error
results chan udpAssociationResult
mu sync.Mutex
tunnels map[string]*udpAssociationTunnel
readDeadline time.Time
readDeadlineChanged chan struct{}
}
func newUDPAssociationConn(
ctx context.Context,
localAddr net.Addr,
timeout time.Duration,
dial udpAssociationDialFunc,
closeIdle func() error,
log logger.Logger,
) *udpAssociationConn {
ctx, cancel := context.WithCancel(context.WithoutCancel(ctx))
c := &udpAssociationConn{
ctx: ctx,
cancel: cancel,
localAddr: localAddr,
dial: dial,
closeIdle: closeIdle,
log: log,
timeout: timeout,
closed: make(chan struct{}),
results: make(chan udpAssociationResult, 32),
tunnels: make(map[string]*udpAssociationTunnel),
readDeadlineChanged: make(chan struct{}),
}
return c
}
func (c *udpAssociationConn) ReadFrom(b []byte) (n int, addr net.Addr, err error) {
for {
select {
case <-c.closed:
return 0, nil, net.ErrClosed
default:
}
c.mu.Lock()
deadline := c.readDeadline
deadlineChanged := c.readDeadlineChanged
c.mu.Unlock()
var (
timer *time.Timer
timeout <-chan time.Time
)
if !deadline.IsZero() {
remaining := time.Until(deadline)
if remaining <= 0 {
return 0, nil, os.ErrDeadlineExceeded
}
timer = time.NewTimer(remaining)
timeout = timer.C
}
select {
case result := <-c.results:
if timer != nil {
timer.Stop()
}
return copy(b, result.data), result.addr, nil
case <-deadlineChanged:
if timer != nil {
timer.Stop()
}
continue
case <-timeout:
return 0, nil, os.ErrDeadlineExceeded
case <-c.closed:
if timer != nil {
timer.Stop()
}
return 0, nil, net.ErrClosed
}
}
}
func (c *udpAssociationConn) WriteTo(b []byte, addr net.Addr) (int, error) {
if addr == nil {
return 0, errMissingDestination
}
tunnel, err := c.tunnel(addr)
if err != nil {
if c.isClosed() {
return 0, net.ErrClosed
}
c.logError(addr, err)
return len(b), nil
}
n, err := tunnel.conn.WriteTo(b, addr)
if err == nil {
return n, nil
}
if c.isClosed() {
return 0, net.ErrClosed
}
if c.removeTunnel(addr.String(), tunnel) {
c.logError(addr, err)
}
return len(b), nil
}
func (c *udpAssociationConn) tunnel(addr net.Addr) (*udpAssociationTunnel, error) {
key := addr.String()
c.mu.Lock()
select {
case <-c.closed:
c.mu.Unlock()
return nil, net.ErrClosed
default:
}
if tunnel := c.tunnels[key]; tunnel != nil {
c.mu.Unlock()
return tunnel, nil
}
c.mu.Unlock()
ctx := c.ctx
if c.timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, c.timeout)
defer cancel()
}
conn, err := c.dial(ctx, addr)
if err != nil {
return nil, err
}
if conn == nil {
return nil, errors.New("masque: nil UDP tunnel")
}
tunnel := &udpAssociationTunnel{conn: conn}
c.mu.Lock()
select {
case <-c.closed:
c.mu.Unlock()
conn.Close()
return nil, net.ErrClosed
default:
}
if current := c.tunnels[key]; current != nil {
c.mu.Unlock()
conn.Close()
return current, nil
}
c.tunnels[key] = tunnel
c.mu.Unlock()
go c.readTunnel(key, tunnel)
return tunnel, nil
}
func (c *udpAssociationConn) readTunnel(key string, tunnel *udpAssociationTunnel) {
buf := make([]byte, udpAssociationBufferSize)
for {
n, addr, err := tunnel.conn.ReadFrom(buf)
if err != nil {
if c.isClosed() {
return
}
if c.removeTunnel(key, tunnel) {
c.logError(key, err)
}
return
}
data := append([]byte(nil), buf[:n]...)
select {
case c.results <- udpAssociationResult{data: data, addr: addr}:
case <-c.closed:
return
}
}
}
func (c *udpAssociationConn) isClosed() bool {
select {
case <-c.closed:
return true
default:
return false
}
}
func (c *udpAssociationConn) logError(addr any, err error) {
if c.log != nil {
c.log.Warnf("masque: UDP tunnel %v: %v", addr, err)
}
}
func (c *udpAssociationConn) removeTunnel(key string, tunnel *udpAssociationTunnel) bool {
c.mu.Lock()
removed := c.tunnels[key] == tunnel
if c.tunnels[key] == tunnel {
delete(c.tunnels, key)
}
c.mu.Unlock()
tunnel.conn.Close()
return removed
}
func (c *udpAssociationConn) Read(b []byte) (int, error) {
n, _, err := c.ReadFrom(b)
return n, err
}
func (c *udpAssociationConn) Write(b []byte) (int, error) {
return 0, errMissingDestination
}
func (c *udpAssociationConn) Close() error {
c.closeOnce.Do(func() {
close(c.closed)
c.cancel()
c.mu.Lock()
close(c.readDeadlineChanged)
tunnels := make([]*udpAssociationTunnel, 0, len(c.tunnels))
for _, tunnel := range c.tunnels {
tunnels = append(tunnels, tunnel)
}
clear(c.tunnels)
c.mu.Unlock()
var errs []error
for _, tunnel := range tunnels {
if err := tunnel.conn.Close(); err != nil {
errs = append(errs, err)
}
}
if c.closeIdle != nil {
if err := c.closeIdle(); err != nil {
errs = append(errs, err)
}
}
c.closeErr = errors.Join(errs...)
})
return c.closeErr
}
func (c *udpAssociationConn) LocalAddr() net.Addr {
return c.localAddr
}
func (c *udpAssociationConn) RemoteAddr() net.Addr {
return &net.UDPAddr{}
}
func (c *udpAssociationConn) SetDeadline(t time.Time) error {
if err := c.SetReadDeadline(t); err != nil {
return err
}
return c.SetWriteDeadline(t)
}
func (c *udpAssociationConn) SetReadDeadline(t time.Time) error {
c.mu.Lock()
defer c.mu.Unlock()
select {
case <-c.closed:
return net.ErrClosed
default:
}
c.readDeadline = t
close(c.readDeadlineChanged)
c.readDeadlineChanged = make(chan struct{})
return nil
}
func (c *udpAssociationConn) SetWriteDeadline(t time.Time) error {
select {
case <-c.closed:
return net.ErrClosed
default:
}
return nil
}
var (
_ net.Conn = (*udpAssociationConn)(nil)
_ net.PacketConn = (*udpAssociationConn)(nil)
)
+321
View File
@@ -0,0 +1,321 @@
package masque
import (
"context"
"errors"
"net"
"sync"
"testing"
"time"
)
func TestUDPAssociationConnRoutesByDestination(t *testing.T) {
type dialCall struct {
addr net.Addr
conn *fakePacketConn
}
var (
mu sync.Mutex
calls []dialCall
)
dial := func(ctx context.Context, addr net.Addr) (net.PacketConn, error) {
conn := newFakePacketConn()
mu.Lock()
calls = append(calls, dialCall{addr: addr, conn: conn})
mu.Unlock()
return conn, nil
}
ctx, cancel := context.WithCancel(context.Background())
conn := newUDPAssociationConn(ctx, &net.UDPAddr{}, 0, dial, nil, nil)
cancel() // The route dial context ends before the UDP association is used.
t.Cleanup(func() { conn.Close() })
addr1 := &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 53}
addr2 := &net.UDPAddr{IP: net.ParseIP("8.8.8.8"), Port: 53}
if _, err := conn.WriteTo([]byte("first"), addr1); err != nil {
t.Fatalf("first WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("second"), addr1); err != nil {
t.Fatalf("second WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("third"), addr2); err != nil {
t.Fatalf("third WriteTo failed: %v", err)
}
mu.Lock()
gotCalls := append([]dialCall(nil), calls...)
mu.Unlock()
if len(gotCalls) != 2 {
t.Fatalf("expected one tunnel per destination, got %d", len(gotCalls))
}
if gotCalls[0].addr.String() != addr1.String() || gotCalls[1].addr.String() != addr2.String() {
t.Fatalf("unexpected tunnel destinations: %s, %s", gotCalls[0].addr, gotCalls[1].addr)
}
if got := gotCalls[0].conn.writtenData(); len(got) != 2 || string(got[0]) != "first" || string(got[1]) != "second" {
t.Fatalf("unexpected writes for first destination: %q", got)
}
if got := gotCalls[1].conn.writtenData(); len(got) != 1 || string(got[0]) != "third" {
t.Fatalf("unexpected writes for second destination: %q", got)
}
replyAddr := &net.UDPAddr{IP: net.ParseIP("127.0.0.1"), Port: 1}
gotCalls[1].conn.queueRead([]byte("reply"), replyAddr)
buf := make([]byte, 32)
n, addr, err := conn.ReadFrom(buf)
if err != nil {
t.Fatalf("ReadFrom failed: %v", err)
}
if string(buf[:n]) != "reply" {
t.Fatalf("unexpected reply: %q", buf[:n])
}
if addr.String() != replyAddr.String() {
t.Fatalf("expected reply from %s, got %s", replyAddr, addr)
}
}
func TestUDPAssociationConnClose(t *testing.T) {
upstream := newFakePacketConn()
idleClosed := 0
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) { return upstream, nil },
func() error {
idleClosed++
return nil
},
nil,
)
if _, err := conn.WriteTo([]byte("request"), &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 53}); err != nil {
t.Fatalf("WriteTo failed: %v", err)
}
if err := conn.Close(); err != nil {
t.Fatalf("Close failed: %v", err)
}
if err := conn.Close(); err != nil {
t.Fatalf("second Close failed: %v", err)
}
if idleClosed != 1 {
t.Fatalf("expected idle stream to be closed once, got %d", idleClosed)
}
if !upstream.isClosed() {
t.Fatal("expected destination tunnel to be closed")
}
if _, err := conn.WriteTo([]byte("request"), &net.UDPAddr{}); !errors.Is(err, net.ErrClosed) {
t.Fatalf("expected net.ErrClosed after Close, got %v", err)
}
}
func TestUDPAssociationConnTunnelFailureIsIsolated(t *testing.T) {
var (
mu sync.Mutex
calls []*fakePacketConn
)
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) {
pc := newFakePacketConn()
mu.Lock()
calls = append(calls, pc)
mu.Unlock()
return pc, nil
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
addr1 := &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443}
addr2 := &net.UDPAddr{IP: net.ParseIP("8.8.8.8"), Port: 443}
if _, err := conn.WriteTo([]byte("first"), addr1); err != nil {
t.Fatalf("first WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("second"), addr2); err != nil {
t.Fatalf("second WriteTo failed: %v", err)
}
mu.Lock()
first, second := calls[0], calls[1]
mu.Unlock()
first.queueReadError(errors.New("stream reset"))
select {
case <-first.closed:
case <-time.After(time.Second):
t.Fatal("failed tunnel was not removed")
}
second.queueRead([]byte("reply"), addr2)
buf := make([]byte, 32)
n, addr, err := conn.ReadFrom(buf)
if err != nil {
t.Fatalf("healthy tunnel failed after peer reset: %v", err)
}
if string(buf[:n]) != "reply" || addr.String() != addr2.String() {
t.Fatalf("unexpected healthy tunnel reply %q from %v", buf[:n], addr)
}
if _, err := conn.WriteTo([]byte("retry"), addr1); err != nil {
t.Fatalf("retry WriteTo failed: %v", err)
}
mu.Lock()
callCount := len(calls)
mu.Unlock()
if callCount != 3 {
t.Fatalf("expected failed destination to redial, got %d dials", callCount)
}
}
func TestUDPAssociationConnDialTimeout(t *testing.T) {
dialErr := make(chan error, 1)
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
20*time.Millisecond,
func(ctx context.Context, _ net.Addr) (net.PacketConn, error) {
<-ctx.Done()
dialErr <- ctx.Err()
return nil, ctx.Err()
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
payload := []byte("request")
n, err := conn.WriteTo(payload, &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443})
if err != nil || n != len(payload) {
t.Fatalf("timed out datagram was not dropped cleanly: n=%d err=%v", n, err)
}
if err := <-dialErr; !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("expected dial deadline, got %v", err)
}
}
func TestUDPAssociationConnDialDoesNotHoldAssociationLock(t *testing.T) {
started := make(chan struct{})
release := make(chan struct{})
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) {
close(started)
<-release
return newFakePacketConn(), nil
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
writeDone := make(chan struct{})
go func() {
conn.WriteTo([]byte("request"), &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443})
close(writeDone)
}()
<-started
deadlineDone := make(chan error, 1)
go func() {
deadlineDone <- conn.SetReadDeadline(time.Now().Add(time.Second))
}()
select {
case err := <-deadlineDone:
if err != nil {
t.Fatalf("SetReadDeadline failed: %v", err)
}
case <-time.After(time.Second):
t.Fatal("dial held the association lock")
}
close(release)
<-writeDone
}
type fakePacket struct {
data []byte
addr net.Addr
err error
}
type fakePacketConn struct {
reads chan fakePacket
closed chan struct{}
closeOnce sync.Once
mu sync.Mutex
writes [][]byte
}
func newFakePacketConn() *fakePacketConn {
return &fakePacketConn{
reads: make(chan fakePacket, 1),
closed: make(chan struct{}),
}
}
func (c *fakePacketConn) queueRead(data []byte, addr net.Addr) {
c.reads <- fakePacket{data: append([]byte(nil), data...), addr: addr}
}
func (c *fakePacketConn) queueReadError(err error) {
c.reads <- fakePacket{err: err}
}
func (c *fakePacketConn) writtenData() [][]byte {
c.mu.Lock()
defer c.mu.Unlock()
result := make([][]byte, len(c.writes))
for i, data := range c.writes {
result[i] = append([]byte(nil), data...)
}
return result
}
func (c *fakePacketConn) isClosed() bool {
select {
case <-c.closed:
return true
default:
return false
}
}
func (c *fakePacketConn) ReadFrom(b []byte) (int, net.Addr, error) {
select {
case packet := <-c.reads:
if packet.err != nil {
return 0, nil, packet.err
}
return copy(b, packet.data), packet.addr, nil
case <-c.closed:
return 0, nil, net.ErrClosed
}
}
func (c *fakePacketConn) WriteTo(b []byte, _ net.Addr) (int, error) {
select {
case <-c.closed:
return 0, net.ErrClosed
default:
}
c.mu.Lock()
c.writes = append(c.writes, append([]byte(nil), b...))
c.mu.Unlock()
return len(b), nil
}
func (c *fakePacketConn) Close() error {
c.closeOnce.Do(func() { close(c.closed) })
return nil
}
func (c *fakePacketConn) LocalAddr() net.Addr { return &net.UDPAddr{} }
func (c *fakePacketConn) SetDeadline(time.Time) error { return nil }
func (c *fakePacketConn) SetReadDeadline(time.Time) error { return nil }
func (c *fakePacketConn) SetWriteDeadline(time.Time) error { return nil }
+82 -15
View File
@@ -10,6 +10,7 @@ import (
"net/url"
"strconv"
"strings"
"sync"
"time"
"github.com/go-gost/core/connector"
@@ -19,6 +20,7 @@ import (
masque_dialer "github.com/go-gost/x/dialer/http3/masque"
masque_util "github.com/go-gost/x/internal/util/masque"
"github.com/go-gost/x/registry"
"github.com/quic-go/quic-go/http3"
)
func init() {
@@ -73,6 +75,16 @@ func (c *masqueConnector) Connect(ctx context.Context, conn net.Conn, network, a
return nil, fmt.Errorf("%w: %s", ErrUnsupportedNetwork, network)
}
func newTCPConnectRequest(address string) *http.Request {
return &http.Request{
Method: http.MethodConnect,
URL: &url.URL{},
Host: address,
Header: http.Header{},
ProtoMajor: 3,
}
}
func (c *masqueConnector) connectTCP(ctx context.Context, conn net.Conn, address string, log logger.Logger) (net.Conn, error) {
log.Debugf("connect-tcp %s", address)
@@ -103,13 +115,7 @@ func (c *masqueConnector) connectTCP(ctx context.Context, conn net.Conn, address
// Create standard HTTP/3 CONNECT request (RFC 9114)
// No :protocol pseudo-header for standard CONNECT
req := &http.Request{
Method: http.MethodConnect,
URL: &url.URL{},
Host: address, // Target address goes in :authority
Header: http.Header{},
Proto: "HTTP/3.0",
}
req := newTCPConnectRequest(address)
// Add proxy authentication if configured
if c.options.Auth != nil {
@@ -172,9 +178,69 @@ func (c *masqueConnector) connectUDP(ctx context.Context, conn net.Conn, address
return nil, ErrInvalidConnection
}
// Get pre-opened stream from dialer (stream opening happens there for dead connection detection)
reqStream := masqueConn.GetRequestStream()
proxyHost := masqueConn.GetHost()
if address == "" {
return c.connectUDPAssociation(ctx, masqueConn, log), nil
}
return c.connectUDPTarget(
ctx,
conn.LocalAddr(),
masqueConn.GetHost(),
masqueConn.GetRequestStream(),
address,
nil,
log,
)
}
func (c *masqueConnector) connectUDPAssociation(ctx context.Context, conn *masque_dialer.MasqueConn, log logger.Logger) net.Conn {
var (
mu sync.Mutex
firstStream = conn.GetRequestStream()
)
nextStream := func(ctx context.Context) (*http3.RequestStream, error) {
mu.Lock()
defer mu.Unlock()
if firstStream != nil {
stream := firstStream
firstStream = nil
return stream, nil
}
return conn.OpenRequestStream(ctx)
}
closeIdle := func() error {
mu.Lock()
defer mu.Unlock()
if firstStream == nil {
return nil
}
err := firstStream.Close()
firstStream = nil
return err
}
dial := func(ctx context.Context, addr net.Addr) (net.PacketConn, error) {
stream, err := nextStream(ctx)
if err != nil {
return nil, err
}
return c.connectUDPTarget(ctx, conn.LocalAddr(), conn.GetHost(), stream, addr.String(), addr, log)
}
return newUDPAssociationConn(ctx, conn.LocalAddr(), c.md.connectTimeout, dial, closeIdle, log)
}
func (c *masqueConnector) connectUDPTarget(
ctx context.Context,
localAddr net.Addr,
proxyHost string,
reqStream *http3.RequestStream,
address string,
remoteAddr net.Addr,
log logger.Logger,
) (*masque_util.DatagramConn, error) {
// Apply connect timeout to the actual stream
if c.md.connectTimeout > 0 {
@@ -259,14 +325,15 @@ func (c *masqueConnector) connectUDP(ctx context.Context, conn net.Conn, address
// Get the underlying HTTP/3 stream for datagrams
stream := reqStream
// Resolve target address
raddr, err := net.ResolveUDPAddr("udp", address)
if err != nil {
return nil, err
if remoteAddr == nil {
remoteAddr, err = net.ResolveUDPAddr("udp", address)
if err != nil {
return nil, err
}
}
// Create datagram connection wrapping the request stream
datagramConn := masque_util.NewDatagramConnFromRequestStream(stream, conn.LocalAddr(), raddr)
datagramConn := masque_util.NewDatagramConnFromRequestStream(stream, localAddr, remoteAddr)
success = true // Prevent defer from closing stream - datagramConn now owns it
return datagramConn, nil
+24
View File
@@ -0,0 +1,24 @@
package masque
import (
"net/http"
"testing"
)
func TestNewTCPConnectRequestUsesStandardConnect(t *testing.T) {
req := newTCPConnectRequest("example.com:443")
if req.Method != http.MethodConnect {
t.Fatalf("expected CONNECT method, got %s", req.Method)
}
if req.Host != "example.com:443" {
t.Fatalf("expected target authority, got %s", req.Host)
}
if req.Proto != "" {
// quic-go treats any non-empty Proto as an extended CONNECT :protocol.
t.Fatalf("standard CONNECT must not set :protocol, got %q", req.Proto)
}
if req.ProtoMajor != 3 {
t.Fatalf("expected HTTP/3 request, got HTTP/%d", req.ProtoMajor)
}
}
+7 -6
View File
@@ -9,7 +9,6 @@ import (
"strings"
"github.com/go-gost/core/common/bufpool"
"github.com/go-gost/core/connector"
md "github.com/go-gost/core/metadata"
"github.com/go-gost/go-shadowsocks2/core"
@@ -97,10 +96,13 @@ func (c *ssConnector) Connect(ctx context.Context, conn net.Conn, network, addre
log.Error(err)
return nil, err
}
rawaddr := bufpool.Get(512)
defer bufpool.Put(rawaddr)
// The wrapped conn stores this target and only writes it on first I/O,
// so it must be owned by this connection, not a shared pooled buffer
// (see go-gost/gost#896).
const maxAddrLen = 1 + 1 + 255 + 2 // SOCKS5 addr: atyp + len + host + port
n, err := addr.Encode(rawaddr)
target := make([]byte, maxAddrLen)
n, err := addr.Encode(target)
if err != nil {
log.Error("encoding addr: ", err)
return nil, err
@@ -111,8 +113,7 @@ func (c *ssConnector) Connect(ctx context.Context, conn net.Conn, network, addre
defer conn.SetDeadline(time.Time{})
}
target := socks.Addr(rawaddr[:n])
conn, err = c.client.WrapConn(conn, target)
conn, err = c.client.WrapConn(conn, socks.Addr(target[:n]))
if err != nil {
return nil, err
}
+81
View File
@@ -0,0 +1,81 @@
package ss
import (
"bytes"
"context"
"io"
"net"
"net/url"
"testing"
"time"
"github.com/go-gost/core/connector"
"github.com/go-gost/go-shadowsocks2/socks"
"github.com/go-gost/x/internal/util/ss/none"
xlogger "github.com/go-gost/x/logger"
mdx "github.com/go-gost/x/metadata"
)
// captureConn records everything written to it, so a test can inspect the
// bytes a connector actually puts on the wire.
type captureConn struct {
buf bytes.Buffer
}
func (c *captureConn) Read(b []byte) (int, error) { return 0, io.EOF }
func (c *captureConn) Write(b []byte) (int, error) { return c.buf.Write(b) }
func (c *captureConn) Close() error { return nil }
func (c *captureConn) LocalAddr() net.Addr { return &net.TCPAddr{} }
func (c *captureConn) RemoteAddr() net.Addr { return &net.TCPAddr{} }
func (c *captureConn) SetDeadline(time.Time) error { return nil }
func (c *captureConn) SetReadDeadline(time.Time) error { return nil }
func (c *captureConn) SetWriteDeadline(time.Time) error { return nil }
// TestConnectTargetNotOverwrittenByPool is a regression test for
// https://github.com/go-gost/gost/issues/896: the connector previously passed
// a slice aliasing the bufpool buffer to WrapConn, and the buffer was returned
// to the pool before the wrapped conn lazily wrote the target on first I/O. A
// second Connect could then overwrite the bytes and the first conn would send
// the second conn's target. The none cipher keeps the wire plaintext so the
// written target is directly assertable.
func TestConnectTargetNotOverwrittenByPool(t *testing.T) {
c := NewConnector(
connector.AuthOption(url.UserPassword("none", "x")),
connector.LoggerOption(xlogger.NewLogger()),
)
if err := c.Init(mdx.NewMetadata(nil)); err != nil {
t.Fatalf("Init: %v", err)
}
ctx := context.Background()
cap1 := &captureConn{}
conn1, err := c.Connect(ctx, cap1, "tcp", "1.2.3.4:80")
if err != nil {
t.Fatalf("first Connect: %v", err)
}
// Second Connect reuses the same 512-byte bufpool buffer that conn1's
// target used to alias (single pool tier, same goroutine).
if _, err := c.Connect(ctx, &captureConn{}, "tcp", "5.6.7.8:443"); err != nil {
t.Fatalf("second Connect: %v", err)
}
if _, err := conn1.Write([]byte("X")); err != nil {
t.Fatalf("first Write: %v", err)
}
// initWriter frames the target as [salt][2-byte length][addr].
data := cap1.buf.Bytes()
payload := data[none.Cipher.SaltSize():]
if len(payload) < 2 {
t.Fatalf("captured payload too short: %d bytes: %x", len(payload), data)
}
n := int(payload[0])<<8 | int(payload[1])
if len(payload) < 2+n {
t.Fatalf("captured payload too short: len=%d, need %d: %x", len(payload), 2+n, data)
}
addr := socks.SplitAddr(payload[2 : 2+n])
if got, want := addr.String(), "1.2.3.4:80"; got != want {
t.Fatalf("first conn wrote target %q, want %q (captured %x)", got, want, data)
}
}
+16
View File
@@ -1,6 +1,7 @@
package tunnel
import (
"context"
"encoding/binary"
"errors"
"fmt"
@@ -11,6 +12,7 @@ import (
"github.com/go-gost/core/common/bufpool"
mdata "github.com/go-gost/core/metadata"
"github.com/go-gost/relay"
ictx "github.com/go-gost/x/internal/ctx"
xrelay "github.com/go-gost/x/internal/util/relay"
)
@@ -94,6 +96,14 @@ func (c *bindConn) Metadata() mdata.Metadata {
return c.md
}
// Context implements xctx.Context, exposing the connection metadata (e.g. the
// tunneled hostname) to the handler's context. The service accept loop derives
// the handler context from a conn that implements this interface, which lets
// reverse-forward handlers (rtcp) route on the hostname carried by the tunnel.
func (c *bindConn) Context() context.Context {
return ictx.ContextWithMetadata(context.Background(), c.md)
}
type bindUDPConn struct {
net.Conn
localAddr net.Addr
@@ -162,6 +172,12 @@ func (c *bindUDPConn) Metadata() mdata.Metadata {
return c.md
}
// Context implements xctx.Context, exposing the connection metadata (e.g. the
// tunneled hostname) to the handler's context. See bindConn.Context.
func (c *bindUDPConn) Context() context.Context {
return ictx.ContextWithMetadata(context.Background(), c.md)
}
type bindAddr struct {
network string
addr string
+45
View File
@@ -0,0 +1,45 @@
package tunnel
import (
"net"
"testing"
mdata "github.com/go-gost/core/metadata"
ictx "github.com/go-gost/x/internal/ctx"
xmd "github.com/go-gost/x/metadata"
mdutil "github.com/go-gost/x/metadata/util"
)
// TestBindConnContext verifies that a bindConn exposes its connection metadata
// (the tunneled hostname) via the xctx.Context interface, so the service accept
// loop forwards it into the handler context for filter.host routing (gost#898).
func TestBindConnContext(t *testing.T) {
md := mdata.Metadata(xmd.NewMetadata(map[string]any{"host": "example.local:80"}))
cn := &bindConn{
Conn: &net.TCPConn{},
localAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0},
remoteAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 80},
md: md,
}
got := mdutil.GetString(ictx.MetadataFromContext(cn.Context()), "host")
if got != "example.local:80" {
t.Errorf("host from Context() = %q, want %q", got, "example.local:80")
}
}
// TestBindUDPConnContext is the same check for the UDP reverse-tunnel conn.
func TestBindUDPConnContext(t *testing.T) {
md := mdata.Metadata(xmd.NewMetadata(map[string]any{"host": "example.local:80"}))
cn := &bindUDPConn{
Conn: &net.TCPConn{},
localAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0},
remoteAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 80},
md: md,
}
got := mdutil.GetString(ictx.MetadataFromContext(cn.Context()), "host")
if got != "example.local:80" {
t.Errorf("host from Context() = %q, want %q", got, "example.local:80")
}
}
+9
View File
@@ -102,6 +102,15 @@ func (c *MasqueConn) GetRequestStream() *http3.RequestStream {
return c.reqStream
}
// OpenRequestStream opens another request stream on the shared HTTP/3 connection.
// CONNECT-UDP associations use one stream per destination.
func (c *MasqueConn) OpenRequestStream(ctx context.Context) (*http3.RequestStream, error) {
if c.clientConn == nil {
return nil, errors.New("masque: connection is closed")
}
return c.clientConn.OpenRequestStream(ctx)
}
// GetHost returns the proxy host.
func (c *MasqueConn) GetHost() string {
return c.host
+20 -4
View File
@@ -2,7 +2,6 @@ package mtcp
import (
"context"
"errors"
"net"
"sync"
"time"
@@ -61,7 +60,17 @@ func (d *mtcpDialer) Dial(ctx context.Context, addr string, opts ...dialer.DialO
session, ok := d.sessions[addr]
if session != nil && session.IsClosed() {
delete(d.sessions, addr) // session is dead
// A session still waiting for its handshake is not dead: its conn
// belongs to the request that dialed it, whose Handshake either
// builds the session on it or closes it. Only an established
// session's base conn is closed here.
if session.session != nil {
session.Close()
if session.conn != nil {
session.conn.Close() // base conn would otherwise leak (and hold a p2p tunnel open)
}
}
delete(d.sessions, addr)
ok = false
}
if !ok {
@@ -114,8 +123,15 @@ func (d *mtcpDialer) Handshake(ctx context.Context, conn net.Conn, options ...di
session, ok := d.sessions[opts.Addr]
if session != nil && session.conn != conn {
conn.Close()
return nil, errors.New("mtls: unrecognized connection")
// Another Dial registered a conn of its own while this one waited
// for the lock. If it has become a live session, use that and drop
// this conn; otherwise build the session on this conn, and the other
// request's Handshake will find it here and use it.
if session.IsClosed() {
ok = false
} else {
conn.Close()
}
}
if !ok || session.session == nil {
+93
View File
@@ -0,0 +1,93 @@
package mtcp
import (
"context"
"io"
"net"
"testing"
"github.com/go-gost/core/dialer"
"github.com/go-gost/core/listener"
mtcp_listener "github.com/go-gost/x/listener/mtcp"
xlogger "github.com/go-gost/x/logger"
xmd "github.com/go-gost/x/metadata"
)
type netDialer struct{}
func (netDialer) Dial(ctx context.Context, network, addr string) (net.Conn, error) {
var d net.Dialer
return d.DialContext(ctx, network, addr)
}
// echoServer serves mtcp on a local port and echoes every stream.
func echoServer(t *testing.T) string {
t.Helper()
ln := mtcp_listener.NewListener(
listener.AddrOption("127.0.0.1:0"),
listener.LoggerOption(xlogger.Nop()),
)
if err := ln.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { ln.Close() })
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
go func() {
defer conn.Close()
io.Copy(conn, conn)
}()
}
}()
return ln.Addr().String()
}
// Concurrent requests that find no live session each dial a conn of their
// own, and a Dial that finds a session still waiting for its handshake
// replaces it. Both handshakes must still end on one usable session,
// whichever runs first; they used to fail with "unrecognized connection",
// or on a conn the later Dial had closed.
func TestHandshakeAfterAnotherDial(t *testing.T) {
for _, first := range []string{"earlier", "later"} {
t.Run(first+" conn first", func(t *testing.T) {
addr := echoServer(t)
d := NewDialer(
dialer.LoggerOption(xlogger.Nop()),
)
if err := d.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatal(err)
}
ctx := context.Background()
earlier, err := d.Dial(ctx, addr, dialer.NetDialerDialOption(netDialer{}))
if err != nil {
t.Fatal(err)
}
later, err := d.Dial(ctx, addr, dialer.NetDialerDialOption(netDialer{}))
if err != nil {
t.Fatal(err)
}
conns := []net.Conn{earlier, later}
if first == "later" {
conns = []net.Conn{later, earlier}
}
for i, conn := range conns {
cc, err := d.(dialer.Handshaker).Handshake(ctx, conn, dialer.AddrHandshakeOption(addr))
if err != nil {
t.Fatalf("handshake %d: %v", i+1, err)
}
if _, err := cc.Write([]byte("ping")); err != nil {
t.Fatalf("handshake %d: write: %v", i+1, err)
}
if _, err := io.ReadFull(cc, make([]byte, 4)); err != nil {
t.Fatalf("handshake %d: read: %v", i+1, err)
}
cc.Close()
}
})
}
}
+20 -4
View File
@@ -3,7 +3,6 @@ package mtls
import (
"context"
"crypto/tls"
"errors"
"net"
"sync"
"time"
@@ -62,7 +61,17 @@ func (d *mtlsDialer) Dial(ctx context.Context, addr string, opts ...dialer.DialO
session, ok := d.sessions[addr]
if session != nil && session.IsClosed() {
delete(d.sessions, addr) // session is dead
// A session still waiting for its handshake is not dead: its conn
// belongs to the request that dialed it, whose Handshake either
// builds the session on it or closes it. Only an established
// session's base conn is closed here.
if session.session != nil {
session.Close()
if session.conn != nil {
session.conn.Close() // base conn would otherwise leak (and hold a p2p tunnel open)
}
}
delete(d.sessions, addr)
ok = false
}
if !ok {
@@ -115,8 +124,15 @@ func (d *mtlsDialer) Handshake(ctx context.Context, conn net.Conn, options ...di
session, ok := d.sessions[opts.Addr]
if session != nil && session.conn != conn {
conn.Close()
return nil, errors.New("mtls: unrecognized connection")
// Another Dial registered a conn of its own while this one waited
// for the lock. If it has become a live session, use that and drop
// this conn; otherwise build the session on this conn, and the other
// request's Handshake will find it here and use it.
if session.IsClosed() {
ok = false
} else {
conn.Close()
}
}
if !ok || session.session == nil {
+123
View File
@@ -0,0 +1,123 @@
package mtls
import (
"context"
"crypto/ecdsa"
"crypto/elliptic"
"crypto/rand"
"crypto/tls"
"crypto/x509"
"crypto/x509/pkix"
"io"
"math/big"
"net"
"testing"
"time"
"github.com/go-gost/core/dialer"
"github.com/go-gost/core/listener"
mtls_listener "github.com/go-gost/x/listener/mtls"
xlogger "github.com/go-gost/x/logger"
xmd "github.com/go-gost/x/metadata"
)
type netDialer struct{}
func (netDialer) Dial(ctx context.Context, network, addr string) (net.Conn, error) {
var d net.Dialer
return d.DialContext(ctx, network, addr)
}
// echoServer serves mtls on a local port and echoes every stream.
func echoServer(t *testing.T) string {
t.Helper()
ln := mtls_listener.NewListener(
listener.AddrOption("127.0.0.1:0"),
listener.TLSConfigOption(&tls.Config{Certificates: []tls.Certificate{selfSignedCert(t)}}),
listener.LoggerOption(xlogger.Nop()),
)
if err := ln.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { ln.Close() })
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
go func() {
defer conn.Close()
io.Copy(conn, conn)
}()
}
}()
return ln.Addr().String()
}
func selfSignedCert(t *testing.T) tls.Certificate {
t.Helper()
key, err := ecdsa.GenerateKey(elliptic.P256(), rand.Reader)
if err != nil {
t.Fatal(err)
}
tmpl := &x509.Certificate{
SerialNumber: big.NewInt(1),
Subject: pkix.Name{CommonName: "127.0.0.1"},
IPAddresses: []net.IP{net.IPv4(127, 0, 0, 1)},
NotBefore: time.Now().Add(-time.Hour),
NotAfter: time.Now().Add(time.Hour),
}
der, err := x509.CreateCertificate(rand.Reader, tmpl, tmpl, &key.PublicKey, key)
if err != nil {
t.Fatal(err)
}
return tls.Certificate{Certificate: [][]byte{der}, PrivateKey: key}
}
// Concurrent requests that find no live session each dial a conn of their
// own, and a Dial that finds a session still waiting for its handshake
// replaces it. Both handshakes must still end on one usable session,
// whichever runs first; they used to fail with "unrecognized connection",
// or on a conn the later Dial had closed.
func TestHandshakeAfterAnotherDial(t *testing.T) {
for _, first := range []string{"earlier", "later"} {
t.Run(first+" conn first", func(t *testing.T) {
addr := echoServer(t)
d := NewDialer(
dialer.TLSConfigOption(&tls.Config{InsecureSkipVerify: true}),
dialer.LoggerOption(xlogger.Nop()),
)
if err := d.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatal(err)
}
ctx := context.Background()
earlier, err := d.Dial(ctx, addr, dialer.NetDialerDialOption(netDialer{}))
if err != nil {
t.Fatal(err)
}
later, err := d.Dial(ctx, addr, dialer.NetDialerDialOption(netDialer{}))
if err != nil {
t.Fatal(err)
}
conns := []net.Conn{earlier, later}
if first == "later" {
conns = []net.Conn{later, earlier}
}
for i, conn := range conns {
cc, err := d.(dialer.Handshaker).Handshake(ctx, conn, dialer.AddrHandshakeOption(addr))
if err != nil {
t.Fatalf("handshake %d: %v", i+1, err)
}
if _, err := cc.Write([]byte("ping")); err != nil {
t.Fatalf("handshake %d: write: %v", i+1, err)
}
if _, err := io.ReadFull(cc, make([]byte, 4)); err != nil {
t.Fatalf("handshake %d: read: %v", i+1, err)
}
cc.Close()
}
})
}
}
+20 -6
View File
@@ -2,7 +2,6 @@ package mws
import (
"context"
"errors"
"net"
"net/url"
"sync"
@@ -76,7 +75,17 @@ func (d *mwsDialer) Dial(ctx context.Context, addr string, opts ...dialer.DialOp
session, ok := d.sessions[addr]
if session != nil && session.IsClosed() {
delete(d.sessions, addr) // session is dead
// A session still waiting for its handshake is not dead: its conn
// belongs to the request that dialed it, whose Handshake either
// builds the session on it or closes it. Only an established
// session's base conn is closed here.
if session.session != nil {
session.Close()
if session.conn != nil {
session.conn.Close() // base conn would otherwise leak (and hold a p2p tunnel open)
}
}
delete(d.sessions, addr)
ok = false
}
if !ok {
@@ -129,10 +138,15 @@ func (d *mwsDialer) Handshake(ctx context.Context, conn net.Conn, options ...dia
session, ok := d.sessions[opts.Addr]
if session != nil && session.conn != conn {
err := errors.New("mws: unrecognized connection")
log.Error(err)
conn.Close()
return nil, err
// Another Dial registered a conn of its own while this one waited
// for the lock. If it has become a live session, use that and drop
// this conn; otherwise build the session on this conn, and the other
// request's Handshake will find it here and use it.
if session.IsClosed() {
ok = false
} else {
conn.Close()
}
}
if !ok || session.session == nil {
+93
View File
@@ -0,0 +1,93 @@
package mws
import (
"context"
"io"
"net"
"testing"
"github.com/go-gost/core/dialer"
"github.com/go-gost/core/listener"
mws_listener "github.com/go-gost/x/listener/mws"
xlogger "github.com/go-gost/x/logger"
xmd "github.com/go-gost/x/metadata"
)
type netDialer struct{}
func (netDialer) Dial(ctx context.Context, network, addr string) (net.Conn, error) {
var d net.Dialer
return d.DialContext(ctx, network, addr)
}
// echoServer serves mws on a local port and echoes every stream.
func echoServer(t *testing.T) string {
t.Helper()
ln := mws_listener.NewListener(
listener.AddrOption("127.0.0.1:0"),
listener.LoggerOption(xlogger.Nop()),
)
if err := ln.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatal(err)
}
t.Cleanup(func() { ln.Close() })
go func() {
for {
conn, err := ln.Accept()
if err != nil {
return
}
go func() {
defer conn.Close()
io.Copy(conn, conn)
}()
}
}()
return ln.Addr().String()
}
// Concurrent requests that find no live session each dial a conn of their
// own, and a Dial that finds a session still waiting for its handshake
// replaces it. Both handshakes must still end on one usable session,
// whichever runs first; they used to fail with "unrecognized connection",
// or on a conn the later Dial had closed.
func TestHandshakeAfterAnotherDial(t *testing.T) {
for _, first := range []string{"earlier", "later"} {
t.Run(first+" conn first", func(t *testing.T) {
addr := echoServer(t)
d := NewDialer(
dialer.LoggerOption(xlogger.Nop()),
)
if err := d.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatal(err)
}
ctx := context.Background()
earlier, err := d.Dial(ctx, addr, dialer.NetDialerDialOption(netDialer{}))
if err != nil {
t.Fatal(err)
}
later, err := d.Dial(ctx, addr, dialer.NetDialerDialOption(netDialer{}))
if err != nil {
t.Fatal(err)
}
conns := []net.Conn{earlier, later}
if first == "later" {
conns = []net.Conn{later, earlier}
}
for i, conn := range conns {
cc, err := d.(dialer.Handshaker).Handshake(ctx, conn, dialer.AddrHandshakeOption(addr))
if err != nil {
t.Fatalf("handshake %d: %v", i+1, err)
}
if _, err := cc.Write([]byte("ping")); err != nil {
t.Fatalf("handshake %d: write: %v", i+1, err)
}
if _, err := io.ReadFull(cc, make([]byte, 4)); err != nil {
t.Fatalf("handshake %d: read: %v", i+1, err)
}
cc.Close()
}
})
}
}
+8 -3
View File
@@ -4,16 +4,21 @@ import (
"net"
)
// conn presents a dialed connection as a connected datagram socket: one
// Write/Read per datagram, the peer address always the connected one, and the
// address argument of WriteTo ignored. It covers both a raw *net.UDPConn and a
// wrapped base dialer (e.g. a p2p tunnel endpoint), neither of which the caller
// needs to tell apart.
type conn struct {
*net.UDPConn
net.Conn
}
func (c *conn) WriteTo(b []byte, addr net.Addr) (int, error) {
return c.UDPConn.Write(b)
return c.Write(b)
}
func (c *conn) ReadFrom(b []byte) (n int, addr net.Addr, err error) {
n, err = c.UDPConn.Read(b)
n, err = c.Read(b)
addr = c.RemoteAddr()
return
}
+4 -1
View File
@@ -49,8 +49,11 @@ func (d *udpDialer) Dial(ctx context.Context, addr string, opts ...dialer.DialOp
return nil, err
}
// The base may be a raw *net.UDPConn (a direct dial) or any conn carrying
// datagrams (a wrapped base dialer, e.g. a p2p tunnel endpoint); the
// wrapper treats both the same.
c = &conn{
UDPConn: c.(*net.UDPConn),
Conn: c,
}
c = proxyproto.WrapClientConn(
+129
View File
@@ -0,0 +1,129 @@
# Plan: obfs4 Pluggable Transport Support (go-gost/gost#182)
## Context
Issue [#182](https://github.com/go-gost/gost/issues/182) requests obfs4 support. GOST v3 currently has `ohttp` (HTTP WebSocket upgrade) and `otls` (fake TLS handshake) obfuscation, but not obfs4 — the "look-like-nothing" pluggable transport used by Tor bridges. The maintainer confirmed in the issue comments that obfs4 was not yet implemented. The `feature` label indicates this is desired functionality.
## Approach: Use external library
Use **`github.com/refraction-networking/obfs4`** (BSD-licensed) — the same organization that provides `refraction-networking/utls` already depended on by GOST. This library implements the full obfs4 protocol (ntor handshake, Elligator 2 key obfuscation, NaCl secretbox framing, SipHash length masking, replay filtering).
### Why this library vs implementing from scratch
| Factor | Library | From scratch |
|--------|---------|-------------|
| Cryptographic safety | Battle-tested (used by Tor/Refraction Networking) | Risk of subtle bugs in Elligator 2, ntor, constant-time ops |
| Code volume | ~200 lines of GOST wrappers | ~1500+ lines of complex crypto code |
| Maintenance | Upstream maintains protocol compliance | GOST maintainers own all crypto code |
| Dependency footprint | Adds ~3 transitive deps | None new |
| Integration fit | `ServerFactory.WrapConn()` is a perfect match for Accept(); `ClientFactory.Dial()` needs a thin bridge in Handshake() | Full control |
The library approach matches how GOST already uses `refraction-networking/utls` for TLS fingerprinting.
## Files to Create
### 1. `x/listener/obfs/obfs4/listener.go`
- Registers `"obfs4"` in `ListenerRegistry`
- `obfsListener` struct following the same pattern as `listener/obfs/tls/listener.go` and `listener/obfs/http/listener.go`
- `NewListener()` factory, `Init(metadata)`, `Accept()` methods
- In `Accept()`: wrap accepted conn with limiter, then call `serverFactory.WrapConn(conn)` to perform server-side obfs4 handshake
### 2. `x/listener/obfs/obfs4/conn.go`
- Thin: the library's `WrapConn` already returns a `net.Conn` that handles framing transparently
- May not even need a custom conn type — `WrapConn` returns a fully functional `net.Conn`
### 3. `x/listener/obfs/obfs4/metadata.go`
- Parse server-side config: `nodeId` (hex), `privateKey` (base64), `iatMode` (int, default 0)
- Construct `pt.Args` from metadata for `ServerFactory()`
### 4. `x/dialer/obfs/obfs4/dialer.go`
- Registers `"obfs4"` in `DialerRegistry`
- `obfs4Dialer` struct following the same pattern as `dialer/obfs/tls/dialer.go`
- `NewDialer()`, `Init(metadata)`, `Dial()`, `Handshake()` methods
- `Dial()`: delegates to options.Dialer (standard raw TCP connection)
- `Handshake()`: uses `ClientFactory.Dial()` with a passthrough `DialFunc` that returns the already-established conn — the library then performs the obfs4 client handshake on it
### 5. `x/dialer/obfs/obfs4/metadata.go`
- Parse client-side config: `cert` (base64, required), `iatMode` (int)
- Store state dir path for client factory
### 6. Update `gost/cmd/gost/register.go`
- Add blank imports:
```go
_ "github.com/go-gost/x/listener/obfs/obfs4"
_ "github.com/go-gost/x/dialer/obfs/obfs4"
```
## Key Design Decisions
### Server-side: Use `ServerFactory.WrapConn()`
The library's `base.ServerFactory` interface has `WrapConn(conn net.Conn) (net.Conn, error)` which is an exact match for GOST's `Accept()` pattern — it takes a raw TCP connection, performs the obfs4 server handshake, and returns a wrapped connection. No custom conn type needed.
### Client-side: Bridge Dial/Handshake with passthrough DialFunc
The library's `ClientFactory.Dial(network, address, DialFunc, args)` both connects AND handshakes. GOST separates these. Solution:
```go
func (d *obfs4Dialer) Handshake(ctx context.Context, conn net.Conn, ...) (net.Conn, error) {
cf, _ := d.transport.ClientFactory(d.md.stateDir)
args, _ := cf.ParseArgs(d.md.ptArgs)
return cf.Dial("tcp", "", func(network, address string) (net.Conn, error) {
return conn, nil // passthrough — conn already established
}, args)
}
```
### Config interface
Users configure obfs4 via YAML/JSON metadata, mirroring Tor's bridge line format:
**Server** (`listener`):
```yaml
services:
- name: obfs4-bridge
addr: :8080
listener:
type: obfs4
metadata:
nodeId: "0123456789abcdef0123456789abcdef01234567"
privateKey: "base64-encoded-curve25519-private-key"
iatMode: 0
handler:
type: auto
```
**Client** (`hop node`):
```yaml
chains:
- name: obfs4-chain
hops:
- nodes:
- addr: bridge.example.com:8080
dialer:
type: obfs4
metadata:
cert: "base64-encoded-server-cert"
iatMode: 0
connector:
type: http
```
### follow existing patterns exactly
- Same `init()` registration pattern
- Same `metadata` struct + `parseMetadata()` pattern
- Same `mdutil.GetString/GetInt` helpers for metadata extraction
- Same listener wrapper chain (proxyproto → metrics → stats → admission → traffic limiter → conn limiter)
- Same dialer factory + Handshake pattern
## Risks
1. **Library API stability**: The refraction-networking/obfs4 library is used by the Tor anti-censorship community. API changes are unlikely but possible. Mitigation: pin to a specific version in go.mod.
2. **State directory requirement**: The library requires a state directory for replay filter persistence. For stateless deployments (containers), we'll default to a temp directory that's cleaned up on exit.
3. **Interop with real Tor obfs4 bridges**: This should work since we're using the same protocol implementation library that Tor uses.
## Verification
1. Build: `cd x && go build ./...` and `cd gost && go build ./cmd/gost/...`
2. Vet: `cd x && go vet ./...`
3. Smoke test: Create a minimal config with obfs4 listener + handler, verify the binary starts without error
4. Integration test: Start a server with obfs4 listener and client with obfs4 dialer, verify connectivity. This can't be tested with real Tor bridges without credentials, but local loopback testing validates the integration layer.
+75
View File
@@ -0,0 +1,75 @@
# Plan: Add uTLS Fingerprint Dialer (`utls`)
## Context
**Issue**: [go-gost/gost#31](https://github.com/go-gost/gost/issues/31) — Feature request to add TLS ClientHello fingerprint simulation using the [uTLS](https://github.com/refraction-networking/utls) library. GOST currently uses Go's `crypto/tls` which produces a distinctive Go TLS fingerprint that's easily blocked. Users want to mimic Chrome/Firefox/Safari/etc. fingerprints.
**Approach**: New `"utls"` dialer type (standalone, not modifying the existing `"tls"` dialer). The fingerprint is dialer-only metadata — not on the shared `TLSConfig` struct. This keeps the standard TLS dialer untouched and avoids pulling in the uTLS dependency for all users.
## Files to Create
### 1. `x/dialer/utls/fingerprint.go` — String→ClientHelloID map
Maps user-facing fingerprint names to `utls.ClientHelloID` presets. Supported names: `chrome`, `firefox`, `ios`, `safari`, `edge`, `randomized`, `randomized-alpn`, `randomized-noalpn`, `golang`, `custom`. Empty string and `"golang"` both mean "fall through to standard crypto/tls" (return `ok=false`). Unknown names get a warning log and also fall through.
### 2. `x/dialer/utls/metadata.go` — Metadata parsing
Mirrors `dialer/tls/metadata.go` exactly, adding a `fingerprint string` field. Parsed keys:
- `handshakeTimeout` (duration)
- `keepalive`, `keepalive.idle`, `keepalive.interval`, `keepalive.count` (TCP keepalive)
- `fingerprint` (string) — **new**
### 3. `x/dialer/utls/dialer.go` — Core dialer
Mirrors `dialer/tls/dialer.go`. Registered as `"utls"`. The `Dial()` method is identical to the TLS dialer. The `Handshake()` method diverges: looks up the fingerprint; if `ok`, uses `utls.UClient(conn, tlsConfig, clientHelloID)`; otherwise falls through to `crypto/tls.Client()`.
## Files to Modify
### 4. `x/go.mod` — Add uTLS dependency
Add `github.com/refraction-networking/utls v1.8.2` to the require block, then run `go mod tidy` from `x/`.
### 5. `gost/cmd/gost/register.go` — Blank import
Add `_ "github.com/go-gost/x/dialer/utls"` in the "Register dialers" section, alphabetically between the `unix` and `ws` imports.
## No Changes Needed
- **Config parsing** — `node/parse.go` already passes `dialCfg.Metadata` to `d.Init()`; no changes required to any parsing code.
- **`config/config.go`** — `DialerConfig.Metadata` already supports arbitrary keys; `fingerprint` flows through naturally.
- **Metadata key constants** — No new constant in `config/parsing/parse.go`; local `const` in `parseMetadata()` follows existing dialer convention.
## Usage Example
```yaml
services:
- name: service-0
addr: :8080
handler:
type: tcp
listener:
type: tcp
forwarder:
nodes:
- name: target-0
addr: example.com:443
dialer:
type: utls
metadata:
fingerprint: chrome
```
## Verification
```bash
# 1. Tidy dependencies
cd x && go mod tidy
# 2. Build + vet the x module
cd x && go build ./... && go vet ./...
# 3. Build + vet the gost binary (verifies blank import)
cd gost && go build ./cmd/gost/... && go vet ./...
```
No unit tests needed — the existing codebase has no tests in `x/dialer/`.
+236
View File
@@ -0,0 +1,236 @@
# Issue #349: 通用 Cache 组件设计
## Context
Issue [#349](https://github.com/go-gost/gost/issues/349) 请求 HTTP 代理缓存层。用户要求先聚焦 cache 组件本身,设计要通用——不仅用于 HTTP,也要能统一替换现有的 DNS、Router、Traffic Limiter、TLS 等模块中的 ad-hoc 缓存。
## 现状:6 处独立缓存实现
| 位置 | 底层库 | Key | Value | TTL 模式 | 淘汰 | 最大容量 |
|------|--------|-----|-------|----------|------|----------|
| `x/internal/util/cache` | 自实现 | `string` | `interface{}` | 硬过期 | 无 | 无 |
| `x/internal/util/resolver` (DNS) | 自实现 | `CacheKey(string)` | `*dns.Msg` | 硬过期 + serve-stale | 最旧(O(n)) | 可选 |
| `x/limiter/traffic/cache` | #1 之上 | `string` | `traffic.Limiter` | 硬过期 + 刷新时滑动 | 无 | 无 |
| `x/limiter/traffic` | patrickmn/go-cache | `string` | `traffic.Limiter` | **滑动**(Get时重置) | 后台清理 | 无 |
| `x/internal/util/tls` | patrickmn/go-cache | `string` | `*x509.Certificate` | 硬过期(1周) | 后台清理 | 无 |
| `x/handler/router` | #1 之上 | `string` | `net.Addr` / `*Route` | 硬过期(1s) | 无 | 无 |
**问题**: 三种底层实现(自实现 ×2 + patrickmn/go-cache),功能重叠但接口不兼容,无通用配置入口。
---
## Core Interface
### `core/cache/cache.go` (新文件)
```go
package cache
var ErrNotFound = errors.New("cache: key not found")
// Entry 缓存条目
type Entry struct {
Data []byte
Expiration time.Time // zero = 永不过期
}
func (e *Entry) Expired() bool
func (e *Entry) TTL() time.Duration // 剩余 TTL,负数=已过期,-1=永不过期
// SetOptions 写入选项
type SetOptions struct {
TTL time.Duration // 逐条 TTL
}
type SetOption func(*SetOptions)
func WithTTL(d time.Duration) SetOption
// Cache 通用 KV 缓存接口
// 实现必须并发安全
type Cache interface {
Get(ctx context.Context, key string) (*Entry, error)
Set(ctx context.Context, key string, data []byte, opts ...SetOption) error
Delete(ctx context.Context, key string) error
Has(ctx context.Context, key string) bool
Clear(ctx context.Context) error
Len(ctx context.Context) int
RefreshTTL(ctx context.Context, key string, ttl time.Duration) error
}
```
**设计决策**:
- Key 固定为 `string`,Value 固定为 `[]byte` — 与 registry 模式兼容,所有后端统一
- `Get` 返回过期条目(`ErrNotFound` 仅对不存在的 key)— 支持 DNS serve-stale
- `RefreshTTL` 显式延长过期时间 — 支持 DNS 异步刷新防重复
- 调用方自行序列化/反序列化 — Cache 不绑定任何业务类型,真正通用
- 与 `recorder.Recorder` 接口复杂度相当(都是单方法 + 选项)
### Each handler wraps with typed methods
```go
// DNS handler 中:
func dnsCacheGet(c cache.Cache, key string) (*dns.Msg, time.Duration) {
entry, err := c.Get(ctx, key)
if err != nil { return nil, 0 }
msg := &dns.Msg{}
msg.Unpack(entry.Data)
return msg, entry.TTL()
}
// HTTP handler 中:
func httpCacheGet(c cache.Cache, key string) (*http.Response, error) { ... }
func httpCacheSet(c cache.Cache, key string, resp *http.Response, ttl time.Duration) error { ... }
```
---
## Config Structure
### `x/config/config.go` — 新增
```go
type CacheConfig struct {
Name string `json:"name"`
Memory *MemoryCache `yaml:",omitempty" json:"memory,omitempty"`
File *FileCache `yaml:",omitempty" json:"file,omitempty"`
Redis *RedisCache `yaml:",omitempty" json:"redis,omitempty"`
Plugin *PluginConfig `yaml:",omitempty" json:"plugin,omitempty"`
}
type MemoryCache struct {
TTL time.Duration `yaml:",omitempty" json:"ttl,omitempty"`
MaxSize int `yaml:",omitempty" json:"maxSize,omitempty"`
MaxBytes int64 `yaml:",omitempty" json:"maxBytes,omitempty"`
CleanupInterval time.Duration `yaml:",omitempty" json:"cleanupInterval,omitempty"`
Eviction string `yaml:",omitempty" json:"eviction,omitempty"` // "oldest" | "lru"
}
type FileCache struct {
Dir string `json:"dir"`
MaxBytes int64 `yaml:",omitempty" json:"maxBytes,omitempty"`
Levels int `yaml:",omitempty" json:"levels,omitempty"`
}
type RedisCache struct {
Addr string `json:"addr"`
DB int `yaml:",omitempty" json:"db,omitempty"`
Username string `yaml:",omitempty" json:"username,omitempty"`
Password string `yaml:",omitempty" json:"password,omitempty"`
KeyPrefix string `yaml:",omitempty" json:"keyPrefix,omitempty"`
}
// 在 Config struct 加:
// Caches []*CacheConfig `yaml:",omitempty" json:"caches,omitempty"`
// 在 ServiceConfig struct 加:
// Cache string `yaml:",omitempty" json:"cache,omitempty"`
```
### YAML 示例
```yaml
caches:
- name: cache-0
memory:
ttl: 60m
maxSize: 10000
maxBytes: 268435456 # 256MB
cleanupInterval: 5m
eviction: lru
services:
- name: service-0
addr: ":8080"
handler:
type: http
cache: cache-0 # 引用
forwarder:
nodes: [...]
```
---
## Memory Backend
### `x/cache/cache.go` — 内建实现
不依赖外部库,替代 `x/internal/util/cache/cache.go` 和 patrickmn/go-cache。
```go
type memoryCache struct {
mu sync.RWMutex
entries map[string]*entry
defaultTTL time.Duration
maxSize int
maxBytes int64
currentBytes int64
cleanupInterval time.Duration
evictionPolicy EvictionPolicy
stopCleanup chan struct{}
logger logger.Logger
}
type entry struct {
data []byte
expiration time.Time // zero = 永不过期
lastAccess time.Time // LRU 用
}
```
**功能矩阵**:
| 特性 | 说明 |
|------|------|
| 并发安全 | `sync.RWMutex` |
| 默认 TTL | 配置项,`Set` 的 `WithTTL` 可逐条覆盖 |
| 最大条目数 | 满时按 `evictionPolicy` 淘汰(oldest / lru) |
| 最大字节数 | 满时同样淘汰 |
| 后台清理 | 可选,按 `cleanupInterval` 定期删除过期条目 |
| serve-stale | `Get` 返回过期条目,`Entry.Expired()` 让调用方判断 |
| `RefreshTTL` | 延长过期时间,用于 DNS 异步刷新模式 |
| 注册 | `init()` 注册为 `registry.CacheRegistry().Register("memory", ...)` |
**淘汰策略**: 满时 → 先清过期 → 若仍满则按策略淘汰(oldest: 按插入时间; lru: 按 `lastAccess`)
---
## 实现步骤
| Step | 文件 | 动作 | 行数 |
|------|------|------|------|
| 1 | `core/cache/cache.go` | 新建 — `Entry`, `Cache` 接口, 选项 | ~70 |
| 2 | `x/config/config.go` | 修改 — `CacheConfig` + 子结构体 + Config 字段 | ~50 |
| 3 | `x/registry/registry.go` | 修改 — `cacheReg` + `CacheRegistry()` | ~15 |
| 4 | `x/registry/cache.go` | 新建 — 热重载包装器 | ~30 |
| 5 | `x/cache/cache.go` | 新建 — memory cache 实现 | ~200 |
| 6 | `x/config/parsing/cache/parse.go` | 新建 — `ParseCache()` 后端分派 | ~50 |
| 7 | `x/config/loader/loader.go` | 修改 — cache 注册块 | ~10 |
| 8 | `x/config/parsing/service/parse.go` | 修改 — 解析 cache 引用注入 handler option | ~25 |
| 9 | `core/handler/option.go` | 修改 — `CacheOption` | ~10 |
| 10 | `gost/cmd/gost/register.go` | 修改 — blank import `x/cache` | +1 |
**总计: ~460 行,集中在 `core/cache/` 和 `x/cache/`**
### 迁移路径 (后续 Phase)
| Phase | 内容 |
|-------|------|
| **Phase 1 (本次)** | `core/cache/` 接口 + memory 后端 + config/registry 全链路 |
| **Phase 2** | DNS handler 切换到新 cache;Router handler 切换;替换 `x/internal/util/cache` |
| **Phase 3** | Traffic limiter 切换;TLS cert pool 切换;移除 patrickmn/go-cache 依赖 |
| **Phase 4** | HTTP 代理缓存集成(handler 中查缓存/存缓存) |
| **Phase 5** | File 后端;Redis 后端;Plugin 后端 |
---
## 验证
```bash
# 编译
cd x && go build ./...
cd gost && go build ./cmd/gost/...
# 单元测试
cd x && go test ./cache/... -v
# 验证 registry 集成
cd x && go vet ./...
```
+131
View File
@@ -0,0 +1,131 @@
# Plan: Tor SOCKS5 Resolve Commands (go-gost/gost#524)
## Context
Issue [#524](https://github.com/go-gost/gost/issues/524) requests support for Tor-specific SOCKS5 extension commands:
- **`0xF0` RESOLVE** — resolve a hostname to an IP address via Tor's DNS
- **`0xF1` RESOLVE_PTR** — reverse DNS lookup (IP → hostname) via Tor's DNS
This allows tools like `torsocks` and `tor-resolve` to work through GOST when forwarding to a Tor SOCKS5 upstream proxy. Previously, GOST's SOCKS5 handler didn't recognize these commands (returning `CmdUnsupported`), so Tor-specific tools failed to work through GOST.
### Existing extension pattern
GOST already defines custom SOCKS5 extension commands in `x/internal/util/socks/socks.go`:
- `CmdMuxBind = 0xF2` — multiplex bind
- `CmdUDPTun = 0xF3` — UDP tunnel over TCP
We add `CmdResolve = 0xF0` and `CmdResolvePTR = 0xF1` to match the same pattern.
### Data flow
```
Client (torsocks) -> GOST SOCKS5 handler (CMD=0xF0) -> Chain -> SOCKS5 connector (CMD=0xF0 to Tor) -> Tor -> reply
|
Client <- GOST SOCKS5 handler (writes Reply with resolved IP) <- chain conn (carries resolved addr metadata)
```
Unlike CONNECT (continuous bidirectional pipe), RESOLVE/PTR is a single request-reply exchange. The connector sends the resolve command upstream, reads the reply containing the resolved address, and returns a connection carrying that address as metadata. The handler extracts it, writes the reply to the client, and closes.
## Files to modify
### 1. `x/internal/util/socks/socks.go` — Add Tor resolve command constants
Add after `CmdUDPTun`:
```go
CmdResolve uint8 = 0xF0 // Tor RESOLVE (hostname -> IP)
CmdResolvePTR uint8 = 0xF1 // Tor RESOLVE_PTR (IP -> hostname)
```
### 2. `x/ctx/value.go` — Add context key for SOCKS5 command override
Add a context key and helpers so the handler can signal a custom SOCKS5 command to the connector. Follows the existing pattern used by other context values in this file (Hash, ClientID, SrcAddr, etc.).
```go
type socks5CmdKey struct{}
func ContextWithSocks5Cmd(ctx context.Context, cmd uint8) context.Context {
return context.WithValue(ctx, socks5CmdKey{}, cmd)
}
func Socks5CmdFromContext(ctx context.Context) (uint8, bool) {
v, ok := ctx.Value(socks5CmdKey{}).(uint8)
return v, ok
}
```
### 3. `x/connector/socks/v5/connector.go` — Send custom SOCKS5 command when signaled
In `Connect()` at the point where `gosocks5.NewRequest(gosocks5.CmdConnect, &addr)` is created:
- Check `Socks5CmdFromContext(ctx)` for a custom command
- Use the context-specified command if present, otherwise default to `CmdConnect`
- After reading the reply, if the command was a resolve type, return a `resolveConn` wrapping the raw connection with the resolved address stored as metadata
### 4. `x/connector/socks/v5/resolve.go` (new file) — Define `resolveConn`
```go
type resolveConn struct {
net.Conn
resolvedAddr *gosocks5.Addr
}
func (c *resolveConn) Metadata() mdata.Metadata {
return mdata.NewMetadata(map[string]any{
"resolvedAddr": c.resolvedAddr,
})
}
```
Implements `net.Conn` (passing through to underlying connection) plus `metadata.Metadatable` to expose the resolved address.
### 5. `x/handler/socks/v5/handler.go` — Add resolve command cases
In `Handle()` switch on `req.Cmd`, add:
```go
case socks.CmdResolve:
return h.handleResolve(ctx, conn, address, ro, log)
case socks.CmdResolvePTR:
return h.handleResolvePTR(ctx, conn, address, ro, log)
```
### 6. `x/handler/socks/v5/resolve.go` (new file) — Implement resolve handlers
**`handleResolve`**:
1. If `enableTor` metadata is false, return `CmdUnsupported` reply
2. Set context with `socks.CmdResolve` via `ContextWithSocks5Cmd`
3. Call `h.options.Router.Dial(ctx, "tcp", address)` — chain routes to SOCKS5 connector, which sends CMD=0xF0
4. Type-assert returned conn to `md.Metadatable`, extract resolved addr
5. Write `gosocks5.Reply{Rep: Succeeded, Addr: resolvedAddr}` to client conn
6. Close chain connection
**`handleResolvePTR`**: Same structure but uses `socks.CmdResolvePTR`.
Both follow the existing error-handling patterns (log errors, write failure replies to client).
### 7. `x/handler/socks/v5/metadata.go` — Add `enableTor` metadata option
Add a boolean metadata field (keys: `"tor"`, `"enableTor"`, `"socks5.tor"`) that defaults to `false`. When `false`, Tor resolve commands are rejected with `CmdUnsupported`. When `true`, the handler accepts and forwards them.
## Verification
### Build check
```bash
cd /config/workspace/go-gost/x && go build ./... && go vet ./...
```
### Integration test
1. Start GOST with `-L "socks5://127.0.0.1:19055?tor=true" -F "socks5://127.0.0.1:9050"` (pointing at a Tor daemon)
2. Use a SOCKS5 client to send a RESOLVE (0xF0) request to port 19055
3. Verify the reply contains a valid resolved IP address
### Manual testing
```bash
gost -L "socks5://127.0.0.1:19055?tor=true" -F "socks5://127.0.0.1:9050"
tor-resolve example.com 127.0.0.1:19055
```
### Test areas
- `tor=false` (default) rejects 0xF0/0xF1 commands with `CmdUnsupported`
- `tor=true` accepts them and forwards through the chain
- Connector sends correct command byte (0xF0/0xF1) when context signals it
- Connector returns `resolveConn` with correct resolved address in metadata
@@ -0,0 +1,119 @@
# go-gost/gost#642 — UDP 目的端口范围跳跃: 可行性 & 复杂度分析
## 需求
客户端支持 UDP 目的端口范围跳跃 (destination port-range hopping),类似 Hysteria 的端口跳跃机制:
```
# 期望的客户端语法
gost3 -L=udp://:8000/mydomain.com:50000-50050/?hopInterval=x秒
gost3 -L=:1080 -F=kcp://127.0.0.1:8000
# 服务端用 iptables 将端口范围折叠到单一端口
iptables -t nat -A PREROUTING -i ens3 -p udp --dport 50000:50050 -j DNAT --to-destination :51000
gost3 -L=kcp://:51000
```
核心用例: 端口跳跃 evasion → iptables 服务端范围折叠 → KCP/QUIC 隧道 → 透明代理。
## 现有基础设施
### 1. `AddrPortRange` — 目的端口范围已可解析
[x/internal/net/addr.go:100](x/internal/net/addr.go#L100) 中 `AddrPortRange.Addrs()` 会将 `50000-50050` 展开为 51 个独立地址 `[":50000", ":50001", ..., ":50050"]`。
[x/config/cmd/cmd.go:440-499](x/config/cmd/cmd.go#L440) 中 `buildServiceConfig()` 将这些地址转化为 `ForwardNodeConfig` 条目。当只有 1 个 listener 但 N 个 forward node 时,所有 N 个 node 都会附加到同一个 service 的 `ForwarderConfig` 上 (一个 hop 持有 N 个 target)。
**关键缺漏**: selector 策略 (`RoundRobin`, `Random`, `Hash`, `FIFO`) 是 per-association / per-connection 的,**不支持基于时间的旋转**。所以要实现"每 X 秒跳跃端口",需要时间感知的选择逻辑。
### 2. UDP forwarder 的 relay 路径
UDP forwarding 由 `x/handler/forward/local/` 处理 (注册名为 `"udp"`, `"tcp"`, `"forward"`):
- **connected UDP** (`handleRawForwarding`): 一次 `dialTarget()` → 单次 `Router.Dial()` → 双向 pipe。目标地址由 hop.Selector 选出,在 association 生命周期内固定。
- **stateless UDP** (`handleRawDatagram`): 每个 datagram 做一次 `dialTarget()` → `ReadFrom` → `WriteTo` → `ReadFrom` → `WriteTo`。从技术上讲,dest port **可以 per-datagram 改变**,但当前 selector 不提供 per-datagram 旋转。
[x/handler/forward/local/forward.go:32-175](x/handler/forward/local/forward.go#L32)
### 3. KCP 会话身份绑定到 4-tuple
xtaci/kcp-go 库 (`v5.6.5`) 的会话模型:
| 组件 | 机制 | 端口跳跃的影响 |
|------|------|---------------|
| **服务端 `Listener`** | 会话存储在 `map[string]*UDPSession`,key 为 `remoteIP:Port` 字符串 | 若客户端源端口稳定 → 同一 session。symmetric NAT 下失效。 |
| **客户端 `output()`** | 固定 `msg.Addr = s.remote`(dial 时解析的地址) | 必须通过 PacketConn wrapper 重写目标端口才能跳跃。 |
| **客户端 `readLoop`** | 锁定到第一个看到的源地址;丢弃所有来自不同地址的包 | 服务端始终从 `:51000` 回复(真实端口,非跳跃范围端口),所以稳定。只有服务端回复时源端口变化才会出问题。 |
### 4. QUIC 支持连接迁移
GOST 的 QUIC 传输使用 quic-go,天然支持 QUIC Connection IDs。这些 ID 允许会话在客户端的 IP/端口变更后依然存活 (RFC 9000 连接迁移)。所以 **QUIC 对 symmetric NAT 也具有较高鲁棒性。**
但 QUIC listener 和 KCP listener 一样,只绑定到**一个端口**。跳跃仍然是面向**服务端的监听端口**的,因此需要 iptables 折叠范围,或者在 gost 内支持多端口 QUIC listener。
## 可行性: 三个方案层级
### 方案 A: 传输层跳跃 (推荐) — 中等工作量
**机制**: 用 PacketConn wrapper 包装 KCP/QUIC dialer 的 `net.PacketConn`,在 `WriteTo` 中根据当前跳跃端口重写目标端口。底层库 (kcp-go/quic-go) 不知道端口在跳跃。
**数据通路**:
```
客户端 KCP dialer 创建 PacketConn → hopWrapper.WriteTo(addr) 重写端口 → kcp-go output() 向 hopped 端口发包
服务端 iptables dport 范围 → 折叠到 :51000 → KCP listener 收到 → 按客户端源 IP:Port 关联会话
服务端从 :51000 回复 → 客户端 readLoop 收到稳定源地址 (一直是 :51000)
```
**需要修改的地方**:
1. **配置解析** (`x/config/cmd/cmd.go` / `x/config/config.go`): 为 `ForwardNodeConfig` 添加 `PortRange` + `HopInterval` 字段,或扩充 chain/hop 的 metadata。
2. **KCP dialer** (`x/dialer/kcp/dialer.go`): 若配置了跳跃参数,将 `kcp.NewConn` 传入的基础 `net.PacketConn` 用 `portHoppingConn` 包装。
3. **QUIC dialer** (`x/dialer/quic/dialer.go`): 同上,但这边的 `quic.DialEarly` 使用的是包装过的 PacketConn。
4. **PortHoppingConn** (新文件/复用 `x/internal/net/` 内现有工具): 一个实现了 `net.PacketConn` 的 wrapper:
- 持有 `currentIndex atomic.Int32` 用于在范围端口间轮转
- 每 `hopInterval` 秒递增 `currentIndex`
- `WriteTo(b, addr)` → 用 `currentPort()` 重写 `addr.Port`,委托到底层 `PacketConn.WriteTo`
- `ReadFrom` / `Close` / `LocalAddr` → 直透传到底层
**KCP 的注意事项**: symmetric NAT(为不同的目标端口分配不同的源端口)会导致服务端因为客户端的 `addr.String()` 变化而创建新 KCP 会话 → 连接中断。这是 KCP 的基本限制,无法在 gost 代码层面解决。**QUIC 没有这个问题**,因为连接 ID 的存在使得源端口变化也没问题。
**工作量**: 预估约 200-300 行新代码 / 约 5 个文件。
### 方案 B: Stateless UDP forwarder 跳跃 — 小工作量
**机制**: 只为 plain-UDP forwarding(非隧道)添加时间跳跃。已经在现有的多 node forwarder (方案 A 的步骤 1) 基础上完成了一半。
**数据通路**: `handleRawDatagram` 中的每次 `dialTarget()` → 当前端口 → `WriteTo`
**限制**: 仅适用于 plain-UDP relay,**不适用于 KCP/QUIC 隧道**,而隧道才是 evasion 真正需要的地方。
**工作量**: 约 50-100 行新代码 / 2-3 个文件。
### 方案 C: 完全协议重设计 — 超大工作量
机制: 不带 iptables 的原生会话迁移,fork / 修补 kcp-go 让会话按 Conversation ID 查找,多端口 listener,迁移信令。这正是 ginuerzh 在 [这条评论](https://github.com/go-gost/gost/issues/642#issuecomment-2568987717) 中提到 "需要新协议和流程" 时的设想。
**工作量**: 数周,跨多个仓库 (fork kcp-go,新认证流程,多 socket 监听)。
## 建议
**方案 A (传输层 PacketConn wrapper)** 是正确的中位数: 用约 200-300 行代码和简单的 PacketConn wrapper,为 KCP 和 QUIC 隧道实现真实的端口跳跃,服务端无需改动 (只需 iptables)。
回应 issue 时,需要解释以下几点:
1. 已实现用途: `AddrPortRange` 已经能将目标端口范围展开为多个 forward node。
2. 未实现部分: 时间感知跳跃,推荐用方案 A 实现。
3. KCP 限制: 服务端 symmetric NAT 会导致跳跃连接中断。建议用 QUIC 传输以获得完全的鲁棒性。
4. 服务端: iptables DNAT 是跨越端口范围的最轻量方案;不需要在 gost 服务端做任何改动。
## 验证
如果决定实现: 不具备 ci/macos 下自动化测试的大规模开销价值。手工验证:
```bash
# 服务端
iptables -t nat -A PREROUTING -p udp --dport 50000:50050 -j DNAT --to-destination :51000
gost -L=kcp://:51000
# 客户端 (运行需要方案 A 的二进制)
gost -L=:1080 -F=kcp://server:50000-50050?hopInterval=10s
# 通过代理发起 UDP 连接,验证 10 秒后目标端口变化 (tcpdump)
```
+942
View File
@@ -0,0 +1,942 @@
# Go-Gost #652: User Namespace 支持 — 技术调研
## Issue
[go-gost/gost#652](https://github.com/go-gost/gost/issues/652) — Feature request: Linux user namespace support.
作者: @Niek | 状态: OPEN
请求支持 Linux user namespaces,使任意程序可以被透明代理。参考项目:[wirez](https://github.com/v-byte-cpu/wirez)(Go 实现的 per-program 透明代理)。
## 核心发现:User NS vs Net NS
GOST 已有完善的**网络命名空间 (netns)** 支持,但 user namespace 与 network namespace 有本质差异:
| 属性 | Network NS | User NS |
|------|-----------|---------|
| 切换时机 | 可事后进入 (`setns()`) | **必须在进程创建时** (`clone(CLONE_NEWUSER)`) |
| 作用域 | per-thread | **per-process** |
| 权限要求 | 需要 `CAP_SYS_ADMIN` | **无需特权** (Linux 3.8+) |
| Go 实现 | `github.com/vishvananda/netns` | `syscall.SysProcAttr` + `os/exec` |
| 可事后 setns? | ✅ (如果有 fd) | ❌ per-process,不能 per-thread 切换 |
**关键结论:现有的 `switchNetns()` 模式([x/internal/net/dialer/dialer_netns.go])无法直接套用到 user namespace。** 需要完全不同的架构:进程创建时通过 `clone()` 建立 namespace,而不是事后 `setns()` 切换。
---
## 现有基础:GOST 已有的可复用组件
### 1. TUN Listener 框架
两个 TUN listener 实现:
- `x/listener/tun/` — 使用 `golang.zx2c4.com/wireguard/tun`,注册名 `"tun"`
- `x/listener/tungo/` — 同样的 wireguard tun driver,注册名 `"tungo"`
两者结构相同:`Init()` → `listenLoop()` → `Accept()` 返回 TUN `net.Conn`,handler 在 conn 上读写 IP 包。
### 2. gVisor Netstack 集成
`x/handler/tungo/` 已深度集成 gVisor netstack (`gvisor.dev/gvisor/pkg/tcpip/stack`)。handler 接收 TUN conn 上的 IP 包,通过 gVisor 做 TCP/UDP 协议栈处理,将流量路由到 GOST proxy chain。
**关键架构优势**:TUN listener 和 handler 在 `io.ReadWriteCloser` 接口处解耦。意味着新增一个 "在 user namespace 内创建 TUN 并通过 fd 传递传回" 的 listener,可以直接对接现有的 `tungo` handler。
### 3. Namespace 基础设施
- `x/internal/net/dialer/dialer_netns.go` — `switchNetns()` 函数,`runtime.LockOSThread()` + `netns.Get()` + `netns.Set()` 模式
- `x/internal/net/net.go` — `ListenConfig` wrapper,支持 namespace-aware 的 `Listen()`/`ListenPacket()`
- `x/config/parsing/parse.go` — `MDKeyNetns`, `MDKeyNetnsOut` metadata key 常量
- `x/config/parsing/service/parse.go` — Service 级别的 netns 解析和注入
### 4. Proxy Chain 基础设施
完全复用:路由 → chain → hop → dialer → connector 管道无需任何修改。
---
## 方案对比
### 方案 A:Wirez 风格透明代理 — 新增 `usertun` Listener ⭐ 推荐
新增 listener 类型,在子进程中 spawn 目标程序(`CLONE_NEWUSER | CLONE_NEWNET`),子进程创建 TUN 并通过 Unix socket + `SCM_RIGHTS` 将 fd 传回父进程。父进程用 gVisor netstack 在 TUN 上处理 TCP/UDP,路由到 GOST proxy chain。
```
┌─ Parent (GOST, unprivileged host user) ───────────┐
│ │
│ TUN fd ← SCM_RIGHTS ← Unix socket ← Child │
│ gVisor netstack ← TUN │
│ TCP/UDP → GOST chain → SOCKS5/HTTP proxy │
│ │
└─────────────────────────────────────────────────────┘
│ clone(CLONE_NEWUSER | CLONE_NEWNET)
▼
┌─ Child (new userns+netns, UID 0 inside) ───────────┐
│ │
│ Creates /dev/net/tun │
│ Configures routing (default gw → TUN) │
│ exec's target program │
│ │
└─────────────────────────────────────────────────────┘
```
**re-exec 核心代码示意:**
```go
// Parent: spawn child in new user+net namespace
socks, _ := syscall.Socketpair(syscall.AF_UNIX, syscall.SOCK_STREAM, 0)
cmd := exec.Command("/proc/self/exe", "--gost-usertun-child", ...)
cmd.SysProcAttr = &syscall.SysProcAttr{
Cloneflags: syscall.CLONE_NEWUSER | syscall.CLONE_NEWNET,
UidMappings: []syscall.SysProcIDMap{
{ContainerID: 0, HostID: os.Getuid(), Size: 1},
},
GidMappings: []syscall.SysProcIDMap{
{ContainerID: 0, HostID: os.Getgid(), Size: 1},
},
GidMappingsEnableSetgroups: false,
}
cmd.ExtraFiles = []*os.File{os.NewFile(uintptr(socks[1]), "tun-sock")}
cmd.Start()
// Receive TUN fd from child
oob := make([]byte, unix.CmsgSpace(4))
_, _, _, _, err := unix.Recvmsg(socks[0], nil, oob, 0)
// ... extract fd from SCM_RIGHTS control message
```
**可复用的已有代码:**
| 组件 | 文件 | 复用方式 |
|------|------|----------|
| TUN 设备创建 | `x/listener/tun/tun.go` (`createTunDevice()`) | 在子进程中调用 |
| gVisor netstack | `x/handler/tungo/` | 父进程中直接使用 |
| Proxy chain | `x/chain/`, `x/router/` | 无修改 |
| LockOSThread 模式 | `x/internal/net/dialer/dialer_netns.go` | 参考模式 |
| Netlink 路由配置 | `x/listener/tun/tun_linux.go` | 在子进程中调用 |
**需新增的代码(估算 ~400-600 行):**
| 工作项 | 估算行数 |
|--------|----------|
| re-exec 入口点 (child mode handler in `cmd/gost/`) | ~50 |
| Namespace + TUN 创建 + 路由 (child side) | ~120 |
| Parent/child Unix socket + SCM_RIGHTS fd 传递 | ~60 |
| usertun listener (parent side, gVisor attachment) | ~150 |
| Metadata 解析 + config 定义 | ~80 |
| 错误处理 + 优雅关闭 + cleanup | ~80 |
| 非 Linux 平台 stub | ~40 |
**依赖变化:零新增依赖。** 所有需要的原语已在依赖树中:
- `golang.org/x/sys/unix` — `CLONE_NEWUSER`, `CLONE_NEWNET`, `SCM_RIGHTS`, `Recvmsg`
- `golang.zx2c4.com/wireguard/tun` — TUN 设备创建
- `github.com/vishvananda/netlink` — 地址/路由配置
- `gvisor.dev/gvisor/pkg/tcpip/stack` — userspace 协议栈(已在 tungo handler 中使用)
**复杂度:中等。** 核心挑战是进程管理和 fd 传递,不涉及 GOST 核心转发逻辑改动。
**平台限制:Linux only。**
---
### 方案 B:简单 User NS 包装器 — 新增 `userexec` Listener
不集成 TUN/gVisor。让 GOST 启动外部命令在 user+net namespace 中运行,GOST 在 namespace 内暴露一个 SOCKS5 listener,子进程通过该 listener 代理。
```
gost -L "userexec://?cmd=curl&listen=127.0.0.1:0"
&proxy=socks5://127.0.0.1:1080"
```
GOST 在 namespace 内创建一个 SOCKS5 listener + handler,子进程通过环境变量 (`ALL_PROXY`) 或直接配置代理。
**改动量:~200-300 行**
**优点:** 实现简单,不影响现有架构
**缺点:**
- 需要目标程序支持 SOCKS5/HTTP 代理(或 LD_PRELOAD),比不上 Wirez 的透明性
- 对于不支持代理的程序(Go 静态链接、某些 CLI 工具)无效
- 本质上只是 `unshare -Urn` + `proxy exec` 的包装
---
### 方案 C:GOST 自身沙箱化
将 GOST 拆分为特权 controller(处理 listener bind、TUN 创建)+ 非特权 worker(在 user namespace 中处理流量转发)。
**不推荐:** 改动范围太大(涉及核心生命周期重构),投入产出比不合理。作为安全加固方向可以在未来考虑,但不是 #652 的合理解决方案。
---
## 关键技术风险
| 风险 | 严重程度 | 缓解措施 |
|------|----------|----------|
| **Go #29789**: `Unshareflags` + `UidMappings` 组合在 `exec.Cmd` 中可能 `EPERM` | 中 | 使用 `Cloneflags` 而非 `Unshareflags`;此 bug 在 `exec.Cmd` 中已有 workaround |
| **`/dev/net/tun` 访问权限** | 低 | 子进程在 user ns 中是 UID 0,有 `CAP_NET_ADMIN`;`/dev/net/tun` 在内核中不检查 host UID |
| **gVisor netstack 性能** | 低 | `tungo` handler 已验证 gVisor 性能可接受;对日常使用足够 |
| **`/etc/subuid` 依赖** | 无 | 单 UID 映射 (`host_uid → ns_uid 0`) 不需要 `/etc/subuid` |
| **`setgroups` 限制** | 低 | `GidMappingsEnableSetgroups: false` 是 Go 1.7+ 的标准写法 |
| **User NS 嵌套** | 低 | 如果 GOST 本身在容器中运行(已有 user ns),嵌套创建需要内核支持(最大 32 层);Linux 4.9+ 默认支持 |
| **re-exec 入口点** | 低 | 隐藏 CLI flag(`--gost-usertun-child`),不影响用户界面 |
---
## 不需要引入的库
以下库经过评估,**不建议引入**:
| 库 | 原因 |
|----|------|
| `github.com/opencontainers/runc/libcontainer` | 太重:~50+ 间接依赖(cgroups, seccomp, capabilities, apparmor, CRIU),只为 user+net ns 创建引入不划算 |
| `github.com/rootless-containers/rootlesskit` | 二进制导向架构(外部 `newuidmap`/`newgidmap`),不适合做 library |
| `github.com/v-byte-cpu/wirez` | 独立的命令行工具,不是 library |
---
## 实现路径建议
如果决定实现方案 A,分 3 个阶段:
### Phase 1: POC (~1-2 天)
- [ ] parent/child re-exec 框架(`cmd/gost/main.go` 中增加 `--gost-usertun-child` 隐藏 flag)
- [ ] 子进程中 user+net namespace 创建 + TUN 创建 + 路由配置
- [ ] Unix socket + SCM_RIGHTS fd 传递
- [ ] 父进程中 TUN fd → gVisor netstack → proxy chain 通路验证
- [ ] 非 root 用户端到端测试(如 `curl ifconfig.me` → SOCKS5 代理)
### Phase 2: 生产化 (~2-3 天)
- [ ] `usertun` listener 完整实现(metadata 解析、config 定义)
- [ ] 错误处理:子进程 crash、TUN 断开、namespace 清理
- [ ] 优雅关闭:SIGTERM → 子进程 cleanup → namespace 销毁
- [ ] DNS 在 namespace 内的处理(gVisor DNS resolver)
- [ ] 非 Linux 平台 stub(返回 "not supported")
### Phase 3: 测试/文档 (~1 天)
- [ ] E2E 测试(Docker-based,参考 `gost/tests/e2e/`)
- [ ] `docs/issues/` 中配置文档
- [ ] 上游 issue 回复
---
## 需要修改的文件清单
### 新文件
- `x/listener/usertun/listener.go` — usertun listener(父进程侧,接收 TUN fd)
- `x/listener/usertun/listener_linux.go` — Linux 实现
- `x/listener/usertun/listener_other.go` — 非 Linux stub
- `x/listener/usertun/metadata.go` — metadata 解析
- `x/listener/usertun/conn.go` — TUN conn 包装
- `x/internal/net/userns/userns_linux.go` — user+net namespace 创建 + re-exec 逻辑
- `x/internal/net/userns/userns_other.go` — 非 Linux stub
- `gost/cmd/gost/usertun_child.go` — re-exec 子进程入口点
### 需要修改的已有文件
- `x/config/parsing/parse.go` — 新增 `MDKeyUserTun` metadata key
- `x/config/parsing/service/parse.go` — usertun 的 service 解析
- `x/config/config.go` — UserTun 相关 config struct(如需要)
- `gost/cmd/gost/main.go` — re-exec 入口分发
- `gost/cmd/gost/register.go` — 注册 usertun listener
### 不需要修改的文件
- `x/handler/tungo/` — 不做任何修改,直接复用
- `x/chain/`, `x/router/` — 不做任何修改
- `x/internal/net/dialer/` — 不做任何修改
- `core/` — 接口不变
---
## 结论
**可行性:高。** GOST 已有的 TUN + gVisor + proxy chain 架构天然适合 Wirez 风格的透明代理集成。User namespace 逻辑是完全隔离的"胶水代码",不改动任何现有核心转发路径。
**集成复杂度:中等偏低。** 核心工作约 400-600 行新代码,零新增依赖,主要挑战是进程管理和 fd 传递。分为 3 个阶段,总计约 4-6 天工作量。
**建议:** 技术可行,值得列入 roadmap,但不急于实现。这个功能开启了"非 root 透明代理任意程序"的使用场景,对移动端/CI 环境/容器化部署有意义。可以等社区有更多需求信号时再推进实现。
---
## 附录:Windows 平台可行性研究
Linux user namespace 的核心价值是:**无需管理员权限,创建隔离的网络环境,使任意程序的流量可以被透明代理。** 在 Windows 上是否有等价机制?本节对此进行独立评估。
### Windows 与 Linux 的根本差异
| 概念 | Linux | Windows |
|------|-------|---------|
| Namespace 隔离 | 内核原生支持,非特权用户可创建 | 无等价概念 |
| 网络重定向 | `netns` + TUN + 路由表 | WFP / WinDivert / TUN(均需管理员) |
| 进程级代理 | `CLONE_NEWUSER` → 子进程隔离 | 无原生机制 |
| TUN 设备创建 | 非特权用户(在 user ns 内) | **始终需要管理员** |
**核心结论:Windows 不存在 Linux user namespace 的等价物。** 所有能实现"透明代理单个程序"的 Windows 机制都至少需要一次性管理员权限。
---
### Windows 透明代理技术全景
#### 方案 1:WinDivert(包级拦截 + 重定向)⭐ Windows 首选
WinDivert 是一个用户态包捕获/修改/重注入框架,内核驱动已签名预编译。
```
目标程序 → WinDivert.sys (内核) → Go 用户态程序 (godivert)
│
修改目标 IP → 127.0.0.1:proxy
│
GOST SOCKS5/HTTP 代理
│
GOST chain → 真实目标
```
| 属性 | 评估 |
|------|------|
| 管理员权限 | **需要**(安装驱动) |
| 单进程过滤 | ✅(`process.path = "C:\\...\\app.exe"`) |
| TCP 重定向 | ✅ |
| UDP 重定向 | ⚠️ 可能但复杂 |
| Go 库 | `github.com/yanlinLiu0424/godivert`, `github.com/imgk/divert-go` |
| 成熟度 | 高(ProxyBridge、Mullvad、SSTap 等均基于此) |
| 新增依赖 | `godivert` + `WinDivert64.sys`(预编译,~200KB) |
**GOST 集成方式:** 新增 `windivert` handler,监听 WinDivert 捕获的包,将 TCP SYN 目标改写为本机代理端口。参考实现:[ProxyBridge](https://github.com/InterceptSuite/ProxyBridge)(MIT,Go)。
**复杂度:中。** GOST 需要实现:WinDivert 会话管理 + 包解析/重写 + TCP 状态跟踪 + 代理转发。约 500-800 行新代码。
---
#### 方案 2:Wintun TUN + Route 重定向
GOST 已有完整的 Windows TUN 支持(`x/listener/tun/tun_windows.go`)。
| 属性 | 评估 |
|------|------|
| 管理员权限 | **需要**(创建网络适配器 + `netsh` 路由操作) |
| 单进程过滤 | ❌(路由表是系统级的) |
| TUN 创建 | ✅(已集成 `golang.zx2c4.com/wireguard/tun`) |
| gVisor netstack | ✅(`x/handler/tungo/` 已集成) |
| Go 库 | 全部已有 |
**限制:** Windows 没有 per-process 路由表。TUN 模式会将**整个系统**的流量(或在路由表中的特定 CIDR)导入代理,无法限制只代理单个进程。
**GOST 集成方式:** 完全不需新代码——`gost -L tun://` 在 Windows 上已经可以工作(以管理员权限运行)。
---
#### 方案 3:Wintun + WFP Callout(Mullvad 模式)— WinDivert 的主要竞争者
这是理论上最优、实践上最难的方案。核心思路:
**WFP callout 内核驱动在 `ALE_CONNECT_REDIRECT` / `ALE_BIND_REDIRECT` 层拦截特定进程的 socket 操作,将流量重定向到 Wintun 虚拟适配器。Wintun 上的 IP 包**保留原始目标地址**(因为重定向发生在 socket 层而非包层),GOST 已有的 TUN listener + gVisor netstack + proxy chain 直接处理,零新 handler 代码。**
```
目标进程调用 connect("example.com:443")
│
▼
┌─ WFP Callout 内核驱动 ──────────────────────────────┐
│ ALE_CONNECT_REDIRECT_V4/V6 │
│ ├─ 按 PID/进程名过滤 │
│ ├─ 重定向 socket → Wintun 接口 │
│ └─ 原始目标 IP 保持不变(socket 级操作) │
└──────────────────────┬──────────────────────────────┘
│ TCP SYN dst=example.com:443
▼
┌─ Wintun TUN 适配器 ─────────────────────────────────┐
│ IP 包: src=<app> dst=example.com:443 ← 目标完整保留 │
│ GOST TUN listener (已有) │
│ gVisor netstack (已有) │
│ GOST proxy chain (已有) │
└─────────────────────────────────────────────────────┘
```
**为什么这比 WinDivert 优雅:**
| 维度 | WFP Callout → Wintun | WinDivert |
|------|---------------------|-----------|
| **拦截层** | Socket 层(`connect()`/`bind()` 级别) | 包层(IP/TCP 头修改) |
| **原始目标** | 自然保留在 IP 头中 | 丢失,需映射表恢复 |
| **TCP 状态** | 内核处理,无需用户态跟踪 | 需用户态维护 TCP 状态机 |
| **校验和** | 无需修改 | 需重算 IP/TCP checksum |
| **UDP 支持** | ✅ 干净(BIND_REDIRECT + CONNECT_REDIRECT) | ⚠️ 复杂 |
| **GOST 代码复用** | ✅ 100%(TUN+gVisor+chain 不变) | ⚠️ ~50%(需新 handler) |
| **连接生命周期** | WFP 原生跟踪(connect/close 事件) | 用户态推断 |
| **递归重定向防护** | WFP 内置机制(`SIO_SET_WFP_CONNECTION_REDIRECT_RECORDS`) | 需手动维护排除列表 |
**但是——实现代价极其巨大:**
下面是 Mullvad [win-split-tunnel](https://github.com/mullvad/win-split-tunnel)(开源的 WFP callout 驱动,GPLv3/MPL-2.0)的工程复杂度分析:
| 维度 | 数据 |
|------|------|
| 语言 | C++ 94.2% + C 4.2%(非 C#/Rust/Go) |
| 类型 | 非 PnP KMDF 内核驱动 |
| 源文件 | 21 个根级文件 + 8 个 firewall/ 子系统文件 |
| 子系统 | 防火墙(WFP callout)、进程管理(完整进程树)、事件(用户态通信)、IOCTL 接口 |
| 驱动状态机 | STARTED → INITIALIZED → READY → ENGAGED(4 阶段 IOCTL 初始化) |
| WFP 层 | `ALE_CONNECT_REDIRECT_V4/V6`, `ALE_BIND_REDIRECT_V4/V6`, `ALE_AUTH_CONNECT_V4/V6`, `ALE_AUTH_RECV_ACCEPT_V4/V6` |
| 流量场景 | 9 种组合(IPv4/v6 × internet/tunnel 接口可用性) |
| 进程管理 | 系统范围进程创建/销毁通知 + 父子进程继承 |
| 事务系统 | 自定义并行事务系统(独立于 WFP 事务) |
| 构建环境 | Visual Studio 2022 + WDK v10.0.22621 + Windows SDK v10.0.22621(仅可在 Win10 2004+ 构建) |
| 代码签名 | EV 证书($300-500/年)+ Microsoft 认证签名 |
| 用户态依赖 | 需要 agent 进程持续提供配置、监控网络接口变化 |
| Git 提交 | 205 commits |
**对 GOST 来说,实现一个 WFP callout 驱动的实际工作量估算:**
| 工作项 | 估算 |
|--------|------|
| WFP callout 驱动开发(C++) | 3-5 周(全职) |
| 进程管理子系统(PID/进程名过滤 + 父子继承) | 1-2 周 |
| IOCTL 接口 + 用户态 Go 控制库(CGO) | 1 周 |
| EV 签名 + Microsoft 认证签名 | 持续成本 |
| 驱动安装/卸载/升级逻辑 | 1 周 |
| 跨 Windows 版本兼容性测试(10/11/Server) | 1-2 周 |
| 调试和稳定性(内核崩溃 = BSOD) | 2-4 周 |
| 用户态 agent(网络变化监控 + 配置推送) | 1 周 |
| **总计** | **10-16 周 + 持续的签名费用** |
**结论:Wintun+WFP 在技术架构上明显优于 WinDivert,但实现代价完全不匹配 GOST 的投入规模。** 这条路线只有在以下场景才合理:(a) 商业化产品有专职 Windows 内核开发团队;(b) 可以 fork Mullvad win-split-tunnel 直接复用(但 Mullvad 的驱动是"排除进程出 VPN"方向,需要反转逻辑 + 仍然需要签名)。
**GOST 若选择此路线,最现实的方案是直接复用 Mullvad 驱动 + 反转过滤逻辑(从"排除"改为"包含")**——但即使如此,仍需精通 Windows 内核开发的工程师,以及持续的代码签名费用。
---
#### 方案 4:API Hooking / DLL 注入
启动目标进程时注入 hook DLL,拦截 Winsock `connect()`/`WSAConnect()` 调用并重定向到本机代理。
| 属性 | 评估 |
|------|------|
| 管理员权限 | ⚠️ 不一定需要(注入自己启动的子进程通常可行) |
| 单进程过滤 | ✅(注入到目标进程内) |
| TCP 重定向 | ✅ |
| UDP 重定向 | ✅(hook `sendto()`/`recvfrom()`) |
| Go 实现 | 需要 CGO + 注入 DLL(C 编写) |
| 风险 | 杀软可能标记;32/64 位不匹配;对反注入保护的应用无效 |
参考:[ProxiGo](https://github.com/hypnguyen1209/proxigo)(Go 实现的 Detours 风格注入器)。
**复杂度:中高。** 需要维护注入 DLL(C 代码),且可靠性不如内核级方案。
---
#### 方案 5:AppContainer + 环境变量
Windows AppContainer 提供进程级安全隔离(UWP/IoT 使用),结合环境变量实现"受限代理"。
| 属性 | 评估 |
|------|------|
| 管理员权限 | **不需要** |
| 网络能力控制 | ✅(通过 Capability SID 限制 Internet/Private 访问) |
| 代理重定向 | ❌(只能 Allow/Deny,不能重定向) |
| 环境变量注入 | ✅(`HTTP_PROXY` 等) |
**限制:** AppContainer 不能重定向流量到代理。只能配合 `HTTP_PROXY` 环境变量使用(仅覆盖支持它的应用)。这不是透明代理方案。
---
#### 方案 6:Windows 容器 / HCS
Windows 容器(通过 `hcsshim` Go SDK)提供网络命名空间级别的隔离,等效于 Linux network namespace + user namespace。
| 属性 | 评估 |
|------|------|
| 管理员权限 | **需要**(启用 Containers 功能 + HCS 操作) |
| 网络隔离 | ✅(HCN/HNS 提供 network namespace 等效能力) |
| 代理策略 | ✅(`hcnproxyctrl` 支持 L4 WFP 代理策略) |
| Go SDK | ✅(`github.com/Microsoft/hcsshim`) |
| 开销 | 高(~200MB+ 基础镜像,秒级启动) |
**不推荐。** 开销太大,不适合"代理单个命令"的场景。
---
#### 方案 7:PAC / WinHTTP / WinINET 代理设置
| 属性 | 评估 |
|------|------|
| 管理员权限 | WinINET: 不需要(HKCU registry)。WinHTTP: 需要(HKLM)。 |
| 单进程过滤 | ❌(用户级或机器级全局设置) |
| 覆盖范围 | WinINET: IE + 使用 WinINET 的应用。WinHTTP: 服务/后台应用。均不覆盖直接使用 Winsock 的程序。 |
**不适合。** 既不能做到单进程,也不能覆盖所有应用。
---
### Windows 方案对比总表
| 方案 | 需要 Admin | 单进程代理 | 透明性 | TCP | UDP | 新代码 | 新依赖 |
|------|-----------|-----------|--------|-----|-----|--------|--------|
| **WinDivert** ⭐ | 一次性(驱动安装) | ✅ | ✅ | ✅ | ⚠️ | ~500-800行 | `godivert` |
| Wintun TUN | 每次运行 | ❌ | ✅ | ✅ | ✅ | 0 | 无 |
| Wintun + WFP | 每次运行 | ✅ | ✅ | ✅ | ✅ | ~1000行 | 内核驱动(C) |
| DLL 注入/Hooking | ⚠️ 不一定 | ✅ | ✅ | ✅ | ✅ | ~600行 | 注入DLL(C) |
| AppContainer | ❌ | ✅ | ❌ | N/A | N/A | ~200行 | 无 |
| Windows 容器 | 每次运行 | ✅ | ✅ | ✅ | ✅ | ~400行 | `hcsshim` (~10MB) |
| PAC/WinHTTP | ❌ | ❌ | ❌ | N/A | N/A | 0 | 无 |
---
### Windows 推荐方案:WinDivert
在 Windows 上实现 #652 的目标(透明代理单个程序),**WinDivert 是唯一在复杂度、可靠性、成熟度之间取得合理平衡的方案**。
**架构设计:**
```
gost -L "windivert://?process=chrome.exe&proxy=socks5://127.0.0.1:1080"
```
```
┌──────────────┐
│ GOST Process │
│ │
WinDivert64.sys ──→ │ godivert │──→ TCP 状态跟踪
(kernel driver) │ handler │──→ 包重写(目标→127.0.0.1:proxy)
↑ │ │──→ 代理 SOCKS5/HTTP
目标进程流量 │ local proxy │──→ GOST proxy chain
(chrome.exe) └──────────────┘
```
**关键实现要点:**
1. **进程过滤:** WinDivert filter string `outbound and tcp and process.path = "C:\\...\\target.exe"`
2. **包重写:** 修改 TCP SYN 的 destination IP/port 为 `127.0.0.1:<proxy_port>`,重算 checksum
3. **原目标记录:** 在代理端通过自定义协议或 TPROXY 等价机制获取原始目标地址
4. **连接跟踪:** 维护 src_ip:src_port → original_dst 映射表
5. **DNS 劫持:** 拦截 UDP 53 的 DNS 请求,防止 DNS 泄漏
**与 Linux 方案的对比:**
| 维度 | Linux (usertun) | Windows (windivert) |
|------|----------------|---------------------|
| 权限模型 | 零特权(user ns 内 root) | 需要一次性管理员安装驱动 |
| 架构 | re-exec 子进程 + TUN fd 传递 | 内核驱动拦截 + 用户态包重写 |
| 代码量 | ~400-600 行 | ~500-800 行 |
| 新增依赖 | 零 | `godivert` (~3KB Go 代码) + `WinDivert64.sys` (~200KB) |
| 成熟度参考 | wirez, tun2socks, gVisor | ProxyBridge, Mullvad, SSTap |
| 平台 | Linux 3.8+ | Windows 7+ |
---
### 业界参考:SocksCap64 与 Netch 的进程流量代理原理
为了更全面地评估 Windows 上的技术选型,本节分析两个成熟的 Windows 进程代理工具的内部机制。
#### SocksCap64:Inline API Hook + 远程 DLL 注入
**主页:** https://www.sockscap64.com/
**核心原理:** SocksCap64 是经典的 Permeo SocksCap 的 64 位继任者,采用**用户态 Inline API Hook + 远程 DLL 注入**技术。
**架构流程:**
```
SocksCap64.exe (启动器)
│
│ CreateProcess + 远程 DLL 注入
▼
目标进程 (已注入 Hook DLL)
│
│ Hook ws2_32.dll 函数:
│ connect(), send(), recv()
│ WSASend(), WSARecv()
│ sendto(), recvfrom()
│ bind(), GetAddrInfoW()
│
│ 每次 connect() 被截获时:
│ 1. 记录原始目标地址
│ 2. 将目标改写为 127.0.0.1:<proxy_port>
│ 3. 完成 SOCKS5 握手(认证 + CONNECT)
│ 4. 所有 send()/recv() → 通过 SOCKS5 隧道中转
▼
SOCKS5 代理服务器
```
**技术细节:**
| 维度 | 说明 |
|------|------|
| Hook 方式 | **Inline Hook**(函数开头写入 `JMP <HookHandler>`,5 字节) |
| Hook 库 | 未公开(可能基于 Detours 或自研) |
| 注入方式 | 远程线程注入(`CreateRemoteThread` + `LoadLibrary`) |
| 目标进程 | 由 SocksCap64 启动的任意 Win32 程序 |
| 管理员权限 | 理论上不需要(注入自己启动的子进程),安装时可能需要 |
| 源码 | 闭源 |
| Go 绑定 | 无 |
| 稳定性 | 中低 — 侵入式注入可能被反作弊/杀软拦截 |
| 覆盖范围 | 所有使用 Winsock 的 Win32 应用;**不覆盖 UWP/Metro 应用** |
**Hook 过程(5 步):**
1. `GetProcAddress` 获取 `ws2_32.dll` 中 `connect`/`send`/`recv` 等函数地址
2. 保存原始函数的前 N 字节指令(用于 trampoline 回调原始函数)
3. `VirtualProtect` 将函数地址页设为 `PAGE_EXECUTE_READWRITE`
4. 在函数开头写入 `JMP <HookHandler>`(E9 相对跳转,5 字节)
5. Hook Handler 中:判断是否代理 → 修改目标地址 → SOCKS5 握手 → 转发数据
**SocksCap64 vs Proxifier 差异:**
| | SocksCap64 | Proxifier |
|---|---|---|
| 技术 | Inline API Hook + DLL 注入 | Winsock LSP(Winsock SPI) |
| 机制 | 修改函数 prologue | 注册 Layered Service Provider |
| 稳定性 | 较低(侵入式注入) | 较高(使用官方 SPI 接口) |
| 反作弊对抗 | 容易被检测/拦截 | 兼容性更好 |
| 现状 | 已停止更新 | 持续维护 |
| 64 位支持 | 原生支持 | 原生支持 |
**对 GOST 的参考价值:**
- ✅ 验证了 "拦截 Winsock → 改写目标 → SOCKS5 转发" 模式的有效性
- ✅ 不需要内核驱动,不强制需要管理员权限
- ❌ DLL 注入本身可靠性低,对 GOST 这种网络工具不适用
- ❌ 注入 DLL 需要 C 编写,Go 无法直接实现
---
#### Netch ProcessMode:内核级包过滤 (netfilter2) + 用户态重定向
**主页:** https://github.com/BoyceLig/Netch | **许可:** GPLv3
**核心原理:** Netch 的 `ProcessMode` 使用 **ntkernel.com netfilter2 SDK**(商业内核驱动)+ **用户态 Redirector DLL**(C++),实现进程级 TCP/UDP 流量拦截和 SOCKS5 重定向。
**架构流程:**
```
┌─ Netch.exe (C# / .NET) ──────────────────────────┐
│ │
│ P/Invoke → Redirector.bin (C++ DLL) │
│ ├─ aio_init() 初始化 │
│ ├─ aio_register() 注册进程过滤规则 │
│ └─ aio_dial() 配置代理目标 │
│ │
│ Redirector DLL: │
│ ├─ nf_init() 注册 netfilter2 驱动 │
│ ├─ nf_registerDriver() │
│ ├─ nf_addRule() 安装过滤规则 │
│ └─ NF_EventHandler 回调: │
│ ├─ tcpConnectRequest ← 内核拦截 TCP 连接 │
│ ├─ tcpSend / tcpReceive │
│ ├─ udpCreated / udpSend / udpReceive │
│ └─ tcpClosed / udpClosed │
│ │
│ TCPHandler (用户态): │
│ ├─ 本地 TCP listener (127.0.0.1:<port>) │
│ ├─ CreateHandler(client, original_dst) 映射表 │
│ └─ Handle() → SocksHelper::TCP → SOCKS5 │
│ │
└─────────────────────────────────────────────────────┘
│ nf_registerDriver (ioctl)
▼
┌─ netfilter2.sys (内核驱动, ntkernel.com) ──────────┐
│ 拦截 TCP connect request (NDIS/WFP 层) │
│ 按 PID/进程名过滤 │
│ 将包信息回调到用户态 │
└─────────────────────────────────────────────────────┘
```
**关键源码文件(基于 GitHub 仓库分析):**
| 文件 | 作用 |
|------|------|
| `Redirector/Redirector.cpp` | DLL 入口,初始化 netfilter2 |
| `Redirector/EventHandler.cpp` | `tcpConnectRequest` 等回调实现 |
| `Redirector/TCPHandler.cpp` | 本地 TCP 监听 + 连接映射表 |
| `Redirector/SocksHelper.cpp` | SOCKS5 握手 + 认证 + CONNECT/UDP ASSOCIATE |
| `Redirector/DNSHandler.cpp` | DNS 劫持/重定向 |
| `Netch/` (C#) | 主程序,P/Invoke 调用 Redirector.bin |
| `RouteHelper/` | 辅助路由工具 |
**TCP `tcpConnectRequest` 重定向流程(基于 EventHandler.cpp 源码):**
```cpp
void tcpConnectRequest(ENDPOINT_ID id, PNF_TCP_CONN_INFO info) {
// 1. 自过滤:不拦截 redirector 自身的流量
if (CurrentID == info->processId) return;
// 2. 进程名过滤:检查 bypassList 和 handleList
if (checkBypassName(info->processName)) return; // 白名单跳过
if (!checkHandleName(info->processName)) return; // 不在代理列表中
// 3. 记录原始目标地址
SOCKADDR_IN6 client, remote;
memcpy(&client, &info->localAddress, sizeof(client));
memcpy(&remote, &info->remoteAddress, sizeof(remote));
// 4. 改写连接目标 → 本机 SOCKS 端口
// IPv4: info->remoteAddress = 127.0.0.1:tcpListen
// IPv6: info->remoteAddress = ::1:tcpListen
// 5. 建立映射表:client_addr → original_remote
TCPHandler::CreateHandler(client, remote);
}
```
**技术总结:**
| 维度 | 说明 |
|------|------|
| 拦截层 | **内核级**(netfilter2 NDIS/WFP 驱动) |
| 用户态组件 | Redirector.bin (C++ DLL),C# P/Invoke 调用 |
| 进程过滤 | 驱动层按 PID + 进程名匹配 |
| TCP 重定向 | 内核拦截 connect → 改写目标 → 本地 listener → SOCKS5 |
| DNS 处理 | DNSHandler 劫持 DNS 查询 |
| 管理员权限 | **必须**(内核驱动安装 + 运行) |
| 源码 | C++/C# 部分 GPLv3 开源;netfilter2 驱动是商业闭源 |
| Go 绑定 | **无**(netfilter2 SDK 提供 C API,可理论上通过 CGO 调用) |
| 稳定性 | 高(内核级拦截比用户态 DLL 注入可靠) |
**Netch 的 4 种模式对比:**
| 模式 | 技术 | 粒度 | Admin |
|------|------|------|-------|
| **ProcessMode** | netfilter2 内核驱动 | 单进程 | 必须 |
| **ShareMode** | WinPcap/Npcap | 系统级(共享) | 必须 |
| **TunMode** | WinTUN 驱动 | 系统级 | 必须 |
| **WebMode** | HTTP 代理 | 应用层(需应用支持) | 不需要 |
---
### Wintun+WFP vs WinDivert — 深度对比
#### 架构差异的本质
这两种方案的**根本区别**在于流量拦截的层级:
```
Wintun+WFP (socket级): WinDivert (包级):
connect("example.com") connect("example.com")
│ │
[WFP callout] [WinDivert driver]
改 socket 路由到 TUN 捕获 TCP SYN 包
│ │
TCP SYN: dst=example.com 改包: dst=127.0.0.1:proxy
原始目标在 IP 头中! │
│ TCP SYN: dst=127.0.0.1
[GOST TUN listener] 原始目标丢失!
│ │
[gVisor netstack] [自定义 handler]
TCP 协议栈自然处理 SOCKS5 握手 + dst 恢复
│ │
[GOST proxy chain] [GOST proxy chain]
│ │
→ example.com:443 → example.com:443
```
#### 逐项对比
| 评估维度 | Wintun+WFP Callout | WinDivert | 优势方 |
|----------|-------------------|-----------|--------|
| **原始目标保留** | ✅ IP 头自然保留 | ❌ 需映射表/编码恢复 | WFP |
| **TCP 处理** | ✅ gVisor 协议栈 | ⚠️ 手动 checksum + 状态跟踪 | WFP |
| **UDP 处理** | ✅ WFP bind/connect redirect | ⚠️ 无连接语义,复杂 | WFP |
| **连接生命周期** | ✅ WFP 原生 connect/close 事件 | ⚠️ 用户态推断 | WFP |
| **DNS 处理** | ✅ gVisor DNS resolver | ⚠️ 需独立 DNSHandler | WFP |
| **递归重定向防护** | ✅ WFP `SIO_SET_WFP_CONNECTION_REDIRECT_RECORDS` | ⚠️ 手动排除本地代理端口 | WFP |
| **GOST 代码复用** | ✅ 100%(TUN+gVisor+chain 零修改) | ⚠️ ~50%(需新 handler + 协议) | WFP |
| **驱动依赖** | ❌ 需自写内核驱动 (C++) | ✅ WinDivert64.sys 已存在 | **WinDivert** |
| **驱动签名** | ❌ EV 证书 + MS 认证 ($300-500/年) | ✅ 已签名 | **WinDivert** |
| **开发语言** | ❌ C++ (KMDF 内核驱动) | ✅ Go (`godivert`) | **WinDivert** |
| **开发工作量** | ❌ 10-16 周 + 内核调试 | ✅ 1-2 周 | **WinDivert** |
| **维护负担** | ❌ 跨 Win10/11/Server 兼容性 + BSOD 风险 | ✅ 上游维护驱动 | **WinDivert** |
| **Go 生态** | ❌ 无 Go 库,需 CGO 封装 | ✅ `godivert` 成熟 | **WinDivert** |
| **构建环境** | ❌ VS2022 + WDK + WinSDK(仅 Win10 2004+ 可构建) | ✅ `go build` | **WinDivert** |
| **开源参考** | Mullvad win-split-tunnel (C++, 205 commits) | ProxyBridge (Go, MIT) | **WinDivert** |
#### 结论
**Wintun+WFP 在技术架构上全面优于 WinDivert**(socket 级重定向 + 原始目标保留 + 100% 复用 GOST 已有 TUN 代码),但**实现代价完全不匹配 GOST 的投入规模**。
| | Wintun+WFP | WinDivert |
|---|---|---|
| 技术优雅度 | ⭐⭐⭐⭐⭐ | ⭐⭐⭐ |
| 实现可行性(对 GOST) | ⭐ | ⭐⭐⭐⭐⭐ |
| 投入产出比 | ⭐ | ⭐⭐⭐⭐ |
**WinDivert 仍然是 GOST 在 Windows 上唯一实际可行的单进程透明代理方案。** Wintun+WFP 在以下条件同时满足时才有意义:
1. 有专职 Windows 内核开发工程师
2. 愿意持续支付 EV 代码签名费用
3. 愿意承担跨 Windows 版本的内核兼容性维护
对于 GOST 这样以 Go 为主的社区驱动项目——WinDivert 是正确选择。
---
### Windows 三种技术路线对比
综合 SocksCap64、Netch、Wintun+WFP、WinDivert 的分析,Windows 上实现单进程透明代理的路径:
| | **SocksCap64** | **Netch** | **Wintun+WFP** | **WinDivert** ⭐ |
|---|---|---|---|---|
| **技术** | Inline Hook + DLL 注入 | netfilter2 商业驱动 | WFP Callout 驱动 + Wintun | WinDivert 开源驱动 |
| **拦截层** | 用户态(函数 hook) | 内核态(NDIS/WFP) | 内核态(WFP socket 重定向) | 内核态(WFP 包捕获) |
| **Go 集成** | ❌(需 C 注入 DLL) | ❌(商业驱动 + C API) | ❌(需写 C++ 内核驱动) | ✅(`godivert` Go 库) |
| **开源** | 闭源 | 半开源(驱动商业) | Mullvad 参考(GPLv3) | 开源(GPLv3+MIT) |
| **Admin** | ⚠️ 不必要 | 必须 | 必须 | 一次性驱动安装 |
| **可靠性** | 中低(注入易被拦截) | 高 | 高 | 高 |
| **原始目标保留** | ✅(hook 参数) | ✅(内核提供) | ✅(IP 头自然保留) | ❌(需映射表) |
| **GOST 代码复用** | 低 | 低 | ✅ 100%(TUN+gVisor 零改动) | 中(需新 handler) |
| **开发量** | 高(注入 DLL) | 中高(商业授权) | **极高(10-16周+签名费)** | 中(~500-800行Go) |
| **维护** | 高 | 中高 | **极高(内核跨版本)** | 低(上游维护) |
| **GOST** | 不推荐 | 不推荐(商业驱动) | 技术最优,实践不现实 | ⭐ 唯一可行 |
**结论:WinDivert 仍然是 GOST 在 Windows 上的唯一实际可行方案。** Wintun+WFP 在技术架构上更优(socket 级重定向保留原始目标、100% 复用 GOST TUN+gVisor 代码),但实现代价(自写 C++ 内核驱动 + EV 签名费 + 跨版本兼容维护)与 GOST 的投入规模完全不匹配。
- SocksCap64 路线:Go 不能写注入 DLL,排除
- Netch 路线:netfilter2 是商业闭源驱动,排除
- **Wintun+WFP 路线:技术架构最优,但需 10-16 周 C++ 内核开发 + 持续签名费,对社区项目不现实**
- **WinDivert 路线:技术上不如 WFP 优雅,但是唯一开源+Go 可集成的方案**
---
### WinDivert UDP 可靠性深度分析
WinDivert 对 TCP 的支持相对成熟(有连接的语义、可以可靠地重写 SYN 目标地址、通过 ACK/FIN 跟踪生命周期),但 **UDP 的支持有一系列架构层面的限制**,直接影响 GOST 集成的可行性。
#### 核心矛盾:SOCKET 层 vs NETWORK 层
这是 WinDivert 最根本的问题,源于 WFP 内核框架的设计:
| 层 | 进程 ID 可用 | 包数据正确 | 用途 |
|-----|------------|-----------|------|
| `WINDIVERT_LAYER_SOCKET` | ✅ `processId` 可用 | ❌ 包数据**损坏/丢失** | 仅能获取进程关联,不能用于数据捕获 |
| `WINDIVERT_LAYER_FLOW` | ✅ `processId` 可用 | ❌ 包数据**损坏/丢失** | 同上 |
| `WINDIVERT_LAYER_NETWORK` | ❌ `processId = 0` | ✅ 完整正确的 IP 包 | 可用于数据捕获,但**不能按进程过滤** |
**根因**(来自 WinDivert 作者):WFP 的 NETWORK/IPPACKET 层在内核中**不携带 ProcessId**——这不是 WinDivert 的 bug,而是 WFP 框架的设计限制。SOCKET/FLOW 层有进程信息,但 WinDivert 在这两层捕获的包数据不包含可解析的 IP/TCP/UDP 头。
来源:[basil00/WinDivert#382](https://github.com/basil00/WinDivert/issues/382), [basil00/WinDivert#71](https://github.com/basil00/WinDivert/issues/71)
#### 唯一可行的 workaround:双 Handle + 5-tuple 映射
所有基于 WinDivert 的透明代理工具(Tallow、zapret/winws、ProxyBridge)的通用做法:
```
┌─ Handle 1: SOCKET 层 ────────────────────────┐
│ Filter: "processId != 0" │
│ 目的: 仅用于建立 5-tuple → PID 映射表 │
│ (数据损坏不关心,只要事件到达即可) │
│ │
│ g_pidMap[(proto, srcIP, srcPort, │
│ dstIP, dstPort)] = processId │
└────────────────────────────────────────────────┘
│ 共享映射表 (mutex + cv)
▼
┌─ Handle 2: NETWORK 层 ───────────────────────┐
│ Filter: "outbound && udp" │
│ 目的: 捕获正确的 UDP 包 │
│ 每个包: 提取 5-tuple → 查映射表 → 决定放行/拦截│
└────────────────────────────────────────────────┘
```
**但这个架构有三个不可消除的固有问题:**
**问题 1:竞态条件 (Race Condition)**
NETWORK 层包可能**先于** SOCKET 层事件到达。没有顺序保证。
缓解方案:
- 对未命中映射的包做**排队等待**(典型超时 2-5 秒)
- 超时后如果仍无映射,**丢弃该包**(首包丢失)
- 这意味着 UDP 应用的**第一个数据包可能被静默丢弃**
**问题 2:UDP 是无连接的 — 映射表管理远比 TCP 复杂**
| | TCP | UDP |
|---|---|---|
| 生命周期事件 | SYN → 已连接 → FIN/RST | **无** |
| 映射何时建立 | connect 前即可获得 | 第一个 sendto() 后 |
| 映射何时释放 | close/fin 事件明确可知 | **无法确定**(socket 可能一直复用) |
| 端口复用 | 简单(TIME_WAIT 后不可复用) | 同一个 socket sendto 到**不同目标**可能复用端口 |
| 映射表增长 | 可控(一个连接一条记录) | 不可控(一个 socket 可能有 N 条记录) |
对于 UDP,如果程序创建了一个 socket 并 sendto 到 100 个不同目标,映射表需要 100 条记录。且无法知道程序何时不再向某个目标发送数据。
缓解方案(ProxyBridge 的做法):
- 通过 `GetExtendedUdpTable` 轮询当前 UDP 绑定关系,作为补充
- 设置较短的 TTL(~30s),超时未使用则清除映射
**问题 3:SOCKET 层可能不触发 UDP 事件**
WinDivert SOCKET 层的 `WINDIVERT_EVENT_SOCKET_BIND` 和 `WINDIVERT_EVENT_SOCKET_CONNECT` 事件主要针对 TCP 设计。UDP 的 `bind()`/`connect()`(若存在)触发的事件覆盖率不可靠。对于无连接的 `sendto()`,SOCKET 层可能**完全不产生事件**。
这意味着某些 UDP 应用的 5-tuple → PID 映射**永远无法建立**。
#### WinDivert UDP 已知问题清单
| 问题 | 严重度 | 缓解措施 | 来源 |
|------|--------|----------|------|
| **SOCKET 层包数据损坏** | 🔴 致命 | 双 handle 架构 | [#382](https://github.com/basil00/WinDivert/issues/382) |
| **NETWORK 层无 processId** | 🔴 致命 | 双 handle + 5-tuple 映射 | [#71](https://github.com/basil00/WinDivert/issues/71) |
| **NETWORK 与 SOCKET 层竞态** | 🟠 高 | 包排队 + 超时(首包可能丢) | 通用架构问题 |
| **DHCP/广播 UDP 丢失** | 🟠 高 | Filter 排除广播 IP | [mitmproxy#6902](https://github.com/mitmproxy/mitmproxy/issues/6902) |
| **多播包重复** | 🟡 中 | 跳过 Loopback=1 的包 | [#364](https://github.com/basil00/WinDivert/issues/364) |
| **校验和 offload 冲突(VM)** | 🟡 中 | 禁用网卡 checksum offload | Xen ML |
| **DontFragment 标志干扰** | 🟡 中 | 无已知 fix | [#278](https://github.com/basil00/WinDivert/issues/278) |
| **高负载下队列断开** | 🟡 中 | 保护性队列大小限制 | 文档 |
| **包重排序** | 🟡 中 | 无(WinDivert 本身会引入) | 文档 |
| **PID 过期/无效** | 🟡 中 | TTL 缓存 | SO |
#### 业界工具的实际 UDP 支持情况
| 工具 | UDP 支持 | 实际策略 | 可靠性 |
|------|----------|----------|--------|
| **ProxyBridge** | 声称支持 | 双 handle + GetExtendedUdpTable 轮询 | ⚠️ DNS 单独处理,警告"大多数 SOCKS5 代理不支持 UDP" |
| **Netch ProcessMode** | 声称支持 | netfilter2 商业驱动(内核级 udpCreated/udpSend 回调) | ✅ 高(商业驱动,socket 级拦截) |
| **SocksCap64** | 声称支持 | Hook sendto/recvfrom | ⚠️ 中(用户态 hook,覆盖不全) |
| **Clash Verge/Mihomo** | 部分支持 | TUN 模式(系统级,不走 WinDivert) | ✅ 高(TUN 模式绕过 WinDivert) |
| **Tallow (Transparent Tor)** | ❌ TCP only | 明确只支持 TCP | N/A |
#### 对 GOST 集成的具体影响
如果 GOST 采用 WinDivert 实现 Windows 透明代理,UDP 的可靠性边界如下:
**可以可靠支持的场景:**
- ✅ TCP 所有流量(有连接 + 生命周期清晰 + 映射表可靠)
- ✅ DNS UDP(端口 53,单独处理,不改写目标,仅劫持响应)
- ⚠️ "典型" UDP 应用(使用 connected UDP socket 的应用,如 QUIC/HTTP3)
**不可靠/不支持的场景:**
- ⚠️ 无连接 UDP (sendto 到多目标,映射表爆炸)
- ⚠️ 高频 UDP(游戏、VoIP,包重排序和延迟不可预测)
- ⚠️ 广播/多播 UDP(DHCP、mDNS、SSDP)
- ❌ 依赖精确 `processId` 的 UDP 过滤(5-tuple 映射可能永远无法建立)
**实用建议:** GOST 的 WinDivert handler 应该默认只处理 TCP,UDP 支持作为**实验性功能**(opt-in flag),并明确文档化限制。
#### 与 Linux usertun 方案的对比
| | Linux usertun (user ns) | Windows WinDivert |
|---|---|---|
| TCP | ✅ 内核协议栈 (gVisor) | ✅ 可接受 |
| UDP | ✅ 内核协议栈 (gVisor) | ⚠️ 有限制 |
| 原始目标 | ✅ IP 头保留 | ❌ 需映射(TCP 还算可靠,UDP 不可靠) |
| 进程隔离 | ✅ 完全(独立 netns) | ⚠️ 基于 5-tuple 推断 |
| 实现复杂度 | TUN+gVisor (已有) | 自定义 handler (~500 行) |
这也又一次说明:**Linux usertun 方案在架构上远优于 Windows 上的任何方案**(包括 WinDivert)。
---
### Windows 额外发现:GOST 现有能力
GOST 在 Windows 上已有以下可用的透明代理能力(均需管理员):
1. **`gost -L tun://`** — TUN 模式,通过 Wintun 创建虚拟网卡,系统级透明代理。**无需新代码,已可用。**
2. **`gost -L tap://`** — TAP 模式,通过 OpenVPN TAP 驱动。同上。
3. **`gost -L tun://` + `gost -L tungo://`** — TUN + gVisor netstack,更完整的协议栈支持。
限制:这些是**系统级**透明代理,不是 Linux user namespace 的**单进程**透明代理。
---
### Windows 结论
**可行性:有条件可行。** 在 Windows 上不存在 Linux user namespace 的非特权等价物。实现"透明代理单个程序"需要内核级拦截(WinDivert 或 WFP callout),均需要一次性管理员权限安装驱动。
**推荐路径:**
1. **如果需要管理员权限可接受:** 实现 `windivert` handler(新代码 ~500-800 行),基于 `godivert` + 已有的 proxy chain。**TCP 支持可靠,UDP 应作为实验性 opt-in 功能。**
2. **如果要求零管理员权限:** 不可行。退而求其次的方案是环境变量注入(`HTTP_PROXY`)+ AppContainer 网络限制,仅覆盖尊重代理环境变量的应用。
3. **如果只需系统级透明代理:** GOST 现有 `tun://` listener 在 Windows 上已可用(需管理员运行)。TUN 模式走 gVisor netstack,**TCP/UDP 均可靠**——缺点是系统级而非单进程。
**建议:** Windows 的 WinDivert 方案可以作为 Linux usertun 方案的后续补充(Phase 2),但不作为 #652 的核心交付。Issue #652 应以 Linux user namespace 方案为主。
+206
View File
@@ -0,0 +1,206 @@
# Fix go-gost #754 — SIGHUP reload `bind: address already in use`
## Context
On SIGHUP hot-reload, every service that keeps its address logs
`init: listen tcp :PORT: bind: address already in use` and the reload fails
(issue go-gost/gost#754). Root cause: in `x/config/loader/loader.go`,
`register()` parses+binds **all** new services in a loop (each
`service_parser.ParseService(c)` binds its port at `ln.Init()`) **before**
`registerGroup` closes the old services. So new listeners bind while old
listeners are still open → `EADDRINUSE`. This is a regression from commit
`82e7e50`, which reversed the order (previously close-old-first, then parse)
to preserve old services on a *parse* error.
Only **services** bind ports; chains/hops/limiters/etc. don't, so the bug is
services-only.
**Goal (Fix #1):** separate listener **construction** from **binding** so the
reload can: (1) construct+validate all new services **without** binding
(any error here leaves old services running — preserves `82e7e50`'s property),
(2) close all old services (free ports), (3) bind+create+register the new ones.
## Design
Split `ParseService` into `PrepareService` (construct, no bind) + a
`(*PreparedService).Build()` method (bind + `NewService`). Keep `ParseService`
as a thin wrapper so its two other callers (`x/api/config_service.go`
`createService`/`updateService`) are unaffected — both already avoid
self-collision.
### Critical correctness detail — netns
The tcp listener binds with a **plain** `net.ListenConfig{}` (no self-switch,
`listener/tcp/listener.go:54,59`). For `metadata.netns` services, the bind
currently relies on the caller having switched namespaces first via the
`netns.Set` block at `parse.go:206-231`. Since that block wraps **both**
construction (line 235) **and** the bind (line 244) today, moving `ln.Init()`
into `Build()` would otherwise bind in the wrong (origin) namespace.
Resolution: **move the netns switch into `Build()`**, wrapping `ln.Init()`.
Construction (`rf(listenOpts...)`) only stores options — it is namespace-
independent, so `PrepareService` drops the netns block. Listeners that
self-switch (`xnet.ListenConfig`, `internal/net/net.go:51`) are unaffected
(a redundant same-namespace re-switch is harmless). Capture `netnsIn` on the
`PreparedService`.
## File-by-file changes
### 1. `x/config/parsing/service/parse.go`
- Add exported type:
```go
// PreparedService is a service fully constructed (listener+handler built,
// options assembled, handler.Init'd) but whose listener is NOT yet bound.
// Build binds the listener and returns a runnable service.Service.
type PreparedService struct {
name string
ln listener.Listener
listenerMD metadata.Metadata
h handler.Handler
netns string
log logger.Logger
options []xservice.Option
}
```
- Add `func PrepareService(cfg *config.ServiceConfig) (*PreparedService, error)`:
the current `ParseService` body with these changes:
- **Remove** the netns block (current lines 206-231) — no longer needed at
construction.
- Construct listener (factory lookup + `rf(listenOpts...)`) as today.
- Capture `listenerMD := metadata.NewMetadata(cfg.Listener.Metadata)` instead
of calling `ln.Init`.
- **Reorder** handler construction (current lines 249-348: handler TLS/auth/
recorders/routerOpts/handler factory/forwarder/`h.Init()`) to run **before**
the bind. Safe: none of it depends on a bound listener. This also fixes a
latent leak where a handler error after `ln.Init` orphans a bound socket.
- Assemble the `[]xservice.Option` slice (current lines 350-362) here while
locals are in scope.
- Return `&PreparedService{...}` (store `netnsIn`).
- Add `func (p *PreparedService) Build() (service.Service, error)`:
- If `p.netns != ""`: `runtime.LockOSThread()` + `netns.Get`/`GetFromPath`|
`GetFromName`/`Set` with `defer netns.Set(originNs)` (mirror old 206-231).
- `p.ln.Init(p.listenerMD)`; on error log `"init: "`, then
`if c, ok := p.h.(io.Closer); ok { c.Close() }` (avoid orphaned handler
goroutines), return `nil, err`.
- `s := xservice.NewService(p.name, p.ln, p.h, p.options...)`; return `s`.
- Refactor `ParseService` to:
```go
func ParseService(cfg *config.ServiceConfig) (service.Service, error) {
ps, err := PrepareService(cfg)
if err != nil { return nil, err }
if ps == nil { return nil, nil }
return ps.Build()
}
```
- Add `"io"` import (used in `Build` error path). `netns`, `runtime`,
`strings`, `metadata`, `xservice` already imported.
### 2. `x/config/loader/loader.go`
Replace the services block in `register()` (lines 304-318) with a three-phase
sequence (services are now special-cased away from the generic
`registerGroup` full-swap):
```go
// --- services (special: only group whose Init binds a port) ---
// Phase 1: construct+validate all new services WITHOUT binding; on any
// error return early (old services untouched).
// Phase 2: close all old services (free ports).
// Phase 3: bind+create+register each prepared service.
{
type preparedEntry struct {
name string
ps *service_parser.PreparedService
}
var prepared []preparedEntry
for _, c := range cfg.Services {
ps, err := service_parser.PrepareService(c)
if err != nil {
return err
}
if ps != nil {
prepared = append(prepared, preparedEntry{c.Name, ps})
}
}
registerGroup[service.Service](nil, registry.ServiceRegistry()) // Phase 2: close-all
for _, e := range prepared { // Phase 3
svc, err := e.ps.Build()
if err != nil {
logger.Default().Errorf("service %s: %v", e.name, err) // log+continue
continue
}
if err := registry.ServiceRegistry().Register(e.name, svc); err != nil {
svc.Close()
logger.Default().Errorf("service %s: %v", e.name, err)
}
}
}
```
Phase 3 partial-failure policy is **log+continue** (aborting would leave all
not-yet-built services down with old already closed). `registerGroup(nil, r)`
is the existing "clear the group" idiom (covered by
`TestRegisterGroup_EmptyEntries_ClearsExisting`). No new imports needed.
### 3. `x/config/loader/loader_test.go`
Add a port-binding stub + reload test that reproduces EADDRINUSE pre-fix:
- `bindingStubListener` (distinct from inert `stubListener`): `Init` does
`net.Listen("tcp", addr)`, stores the `net.Listener`; `Addr`/`Close` delegate.
- `freeTCPPort(t)` helper: `net.Listen("tcp","127.0.0.1:0")`, read port, close.
- Register the binding factory under a test-only name (e.g. `"tcp-binding-test"`)
and the existing inert `"auto"` handler stub, with save/restore `t.Cleanup`.
- `TestRegister_ServiceReloadNoCollision`: build a config with one service on a
fixed free port; call `register()` twice; assert the **second** call returns
no error and the service stays registered. Before the fix the second call
returns `EADDRINUSE`. Assert via `strings.Contains(err.Error(), "address
already in use")` on failure.
- Add `"fmt"`, `"strings"` imports.
## Reused / referenced (no edits)
- `x/registry/registry.go:83` `Unregister` → calls `Close()` (frees ports in
Phase 2). `registerGroup` (loader.go:101) used as close-all idiom.
- `x/service/service.go:144` `NewService` (inert re: sockets; runs pre-up,
sets state) and `:344` `Close()` (closes listener/handler/observer).
- `internal/net/net.go:51` `xnet.ListenConfig` self-switches netns (context
for why moving the netns block to `Build` is safe).
## Verification
```bash
cd /config/workspace/go-gost/x
go build ./... # refactor compiles
go vet ./...
go test ./config/loader/... # existing + new test
CGO_ENABLED=1 go test -race ./config/loader/... # race-clean (per CLAUDE.md)
cd /config/workspace/go-gost && go build ./... # gost binary still builds
```
The new test must pass after the change and would fail (`EADDRINUSE` on the
second `register()`) if the construct/bind split were reverted.
Optional manual end-to-end: run gost with one TCP service on a fixed port,
`kill -HUP <pid>`, confirm no `address already in use` in the log and the
service keeps serving.
## Risks / notes
- **netns**: resolved by moving the netns switch into `Build` (see Design).
During implementation, grep to confirm no listener binds in its *constructor*
(factories should only store options); if one does, it would need separate
handling. The 4 checked (tcp/udp/tls/ws) bind in `Init`.
- **Bind failure = one service down**: after close-all, a single bind failure
takes only that service down (log+continue); unavoidable given the fix's
premise. A joined-error return is a possible follow-up.
- **Pre-existing, orthogonal**: `x/api/config_reload.go` spawns fresh `Serve`
goroutines for every registered service on each reload — not caused or
changed by this fix; flagged so it isn't mistaken for a regression.
## Post-implementation (out of plan scope)
After the code lands + builds + tests pass: optionally post the drafted
explanatory comment to issue #754 and/or open a PR (English comment; show to
user before posting per global GitHub-comments rule).
+203
View File
@@ -0,0 +1,203 @@
# Plan: QUIC/HTTP3 SNI sniffer (issue go-gost/gost#761, Scope A)
## Context
Issue go-gost/gost#761 requests a QUIC/HTTP3 protocol sniffer, referencing the v2fly and Xray
implementations. GOST already sniffs **TCP** traffic (TLS/HTTP/SSH) via
`x/internal/util/sniffing/` to extract the SNI/Host for **bypass decisions and traffic recording**.
QUIC (HTTP/3) runs over **UDP** and is encrypted from the first packet, so it cannot flow through
the existing TCP `Sniff(*bufio.Reader)` path — there is currently **no UDP sniffing anywhere** in
the codebase (`grep udp` in `sniffing/`+`forwarder/` → nothing).
This plan adds **Scope A only**: sniff the QUIC Initial packet to extract the SNI (server_name) for
SNI-based bypass and recording, then forward datagrams transparently. **No MITM/termination** of
QUIC (that is Scope B — a far larger effort, intentionally excluded). The goal matches exactly what
Xray/v2fly do and what GOST's existing TLS sniffer achieves for TCP.
Outcome: a transparent UDP redirect (`redu`) service can apply bypass rules and record QUIC flows by
hostname, closing the HTTP/3 gap.
## Approach
The QUIC Initial parser is extracted into a **separate repo `quic-dissector/`** (parallel to
[`tls-dissector/`](../../../tls-dissector/)), following the same pattern: zero/minimal dependencies,
clean I/O boundary (bytes in → parsed info out), independently versioned, reusable by projects
outside GOST. The `x/` module consumes it as a standard Go dependency, just like it does
`tls-dissector`.
Three layers, lowest-risk-first.
### Phase 1 — `quic-dissector/` repo (new, parallel to `tls-dissector/`)
New repo `github.com/go-gost/quic-dissector` at workspace root `quic-dissector/`. Structure:
```
quic-dissector/
├── go.mod module github.com/go-gost/quic-dissector
│ deps: tls-dissector + golang.org/x/crypto
├── dissector.go SniffQUIC — public API
├── dissector_test.go golden-packet tests + fuzz
├── internal/quic/
│ ├── varint.go QUIC varint (~5 lines, avoids quic-go dep)
│ ├── hkdf.go HKDF-Expand-Label per RFC 8446 §7.1 (~30 lines)
│ └── initial.go Header protection removal + AEAD decrypt + CRYPTO reassembly
├── CLAUDE.md
├── README.md
├── LICENSE
└── .github/workflows/ci.yml
```
**Why separate repo** (same rationale as `tls-dissector`):
- Clean boundary: `[]byte` in, `*dissector.ClientHelloInfo` out, no I/O, no state
- Reusable: any Go project doing QUIC inspection needs this (Xray, v2fly, network monitors)
- Independently versionable: QUIC v2 (RFC 9369) has different salt + inverted type bits
- Nearly zero-dependency: only `x/crypto` (ubiquitous) + `tls-dissector` (same org, zero-dep)
- Precedent: `tls-dissector/` already established this pattern in the go-gost ecosystem
**Public API** (`dissector.go`):
```go
package quicdissector
import "github.com/go-gost/tls-dissector"
// ErrNotQUIC is returned when the datagram is not a parsable QUIC Initial packet.
var ErrNotQUIC = errors.New("not a QUIC Initial packet")
// SniffQUIC decrypts a QUIC Initial datagram and parses the embedded TLS ClientHello.
// Returns the ClientHello info (ServerName=SNI, SupportedProtos=ALPN),
// or ErrNotQUIC when the datagram cannot be parsed.
// Only the first datagram is parsed; cross-datagram ClientHello reassembly is out of scope.
func SniffQUIC(datagram []byte) (*dissector.ClientHelloInfo, error)
```
Returning `*dissector.ClientHelloInfo` directly (rather than separate `sni, alpn` strings)
gives callers access to all ClientHello fields (SupportedVersions, CipherSuites, etc.) at
zero extra cost. The `x/` handler extracts `.ServerName` and `.SupportedProtos[0]` for
bypass/recording — same fields the TCP TLS sniffer uses today.
Algorithm (port of Xray `common/protocol/quic/sniff.go`, confirmed against quic-go internals):
1. **Long-Header validation** — bit 7 set (`0x80`), fixed bit set (`0x40`); read 4-byte version;
accept `0x00000001` (QUIC v1) and `0xff00001d` (draft-29). Reject short headers (1-RTT) — not
sniffable.
2. **Packet type** — `(typeByte & 0x30) >> 4 == 0x0` = Initial; skip non-Initial long-header packets.
3. **DCID** (8–20 bytes) — the key material. Add an explicit `dcidLen` range guard Xray omits.
4. **Token** (varint-length, Initial-only) — skip.
5. **Packet length + header boundary** via QUIC varint (self-contained, ~5 lines — avoids pulling in
`github.com/quic-go/quic-go/quicvarint`).
6. **Derive Initial keys** (HMAC-SHA256 HKDF, `golang.org/x/crypto/hkdf` + stdlib `crypto/aes`,
`crypto/cipher`):
- `initialSecret = HKDF-Extract(salt=QUIC_SALT, IKM=DCID)` — v1 salt
`38762cf7f55934b34d179ae6a4c80cadccbb7f0a` (RFC 9001 §5.2).
- `clientSecret = HKDF-Expand-Label(initialSecret, "client in", 32)`.
- `hpKey = HKDF-Expand-Label(clientSecret, "quic hp", 16)`,
`key = …("quic key", 16)`, `iv = …("quic iv", 12)`.
- `HKDF-Expand-Label` per RFC 8446 §7.1 (`"tls13 "+label`, 2B length + 1B label-len + 1B ctx-len).
Self-contained implementation in `internal/quic/hkdf.go`.
7. **Remove header protection** — AES-ECB single-block of the 16-byte sample at offset 4 into the
protected region; XOR low nibble of type byte (→ packet-number length) + up to 4 PN bytes.
8. **AEAD decrypt** — AES-128-GCM (`key`/`iv`), nonce = `iv` XOR truncated packet number, AAD = the
full unmasked header. QUIC Initial protection is always AES-128-GCM regardless of negotiated cipher.
9. **CRYPTO frame reassembly** — within the decrypted payload, iterate frames (PADDING/PING/ACK/
CRYPTO/CONNECTION_CLOSE); assemble `CRYPTO` (type `0x06`) data by offset into a buffer. Handle
coalesced packets within the single datagram (Xray's outer loop).
10. **ClientHello SNI** — prepend a synthetic TLS record header (`0x16,0x03,0x03,<len>`) to the
reassembled handshake bytes and call `dissector.ParseClientHello` (reuse — same parser the TCP
TLS sniffer uses at `sniffer_tls.go:34`), returning the full `*ClientHelloInfo`.
Reuses: `dissector.ParseClientHello`. New deps: `golang.org/x/crypto` (hkdf). **No quic-go
dependency** — varint and HKDF-Expand-Label are self-contained.
**Limitation (documented):** parses only the ClientHello within the **first datagram** (handles
coalescing inside it). Cross-datagram ClientHello reassembly needs connection-state tracking and is
out of scope — note in a code comment. This covers the overwhelming majority of real traffic.
### Phase 2 — redirect/udp (`redu`) handler integration
`x/handler/redirect/udp/` is the clean mirror of `redirect/tcp` (both use `conn.LocalAddr()` as the
TPROXY destination and `xnet.Pipe`). The per-client `net.Conn` from `internal/net/udp/listener.go`
already buffers the first datagram by the time `Handle()` runs (the stateful listener Accepts on the
first datagram), so peeking will not block.
- **`metadata.go`** — add `sniffing bool` and `sniffingTimeout time.Duration`. Keys
`mdutil.GetBool(md, "sniffing")`, `mdutil.GetDuration(md, "sniffing.timeout")` (mirror
`redirect/tcp/metadata.go`). No `mitm.*` fields — Scope A does no MITM.
- **`handler.go` `Handle()`** — after `dstAddr := conn.LocalAddr()`, if `h.md.sniffing`:
1. `conn.SetReadDeadline(now + sniffingTimeout)` (default from `sniffing.DefaultReadTimeout`).
2. Read the first datagram into a buffer (one `Read` returns exactly the buffered datagram).
3. `info, err := quicdissector.SniffQUIC(buf)`. Clear the read deadline.
4. On success: set `ro.Host = net.JoinHostPort(info.ServerName, port)`, `ro.TLS = &xrecorder.TLSRecorderObject{ServerName: info.ServerName, Proto: info.SupportedProtos[0]}` (ALPN, may be empty). Apply SNI-based bypass via `h.options.Bypass.Contains(ctx, "udp", ro.Host, ...)` → `xbypass.ErrBypass` (mirrors `sniffer_tls.go:59`).
5. Dial upstream to `dstAddr` (transparent — destination stays the original IP, SNI is for bypass/recording only), then forward the buffered first datagram + remaining via `xnet.Pipe`, prepending the buffered bytes with `xnet.NewReadWriteConn(io.MultiReader(bytes.NewReader(buf), conn), conn, conn)` (exact pattern from `redirect/tcp/handler.go:207`).
6. On `quicdissector.ErrNotQUIC` / decode failure: fall through to the existing transparent-forward path, prepending the already-read datagram the same way so no data is lost.
`forward/local` UDP sniffing is a **secondary, optional** extension (its sniffing is currently gated
`network == "tcp"`). Defer unless requested — `redirect/udp` is the primary, cleanest target.
### Phase 3 — protocol constant + integration
- **`sniff.go`** — add `ProtoQUIC = "quic"` constant for recording consistency (the TCP `Sniff()`
function itself is unchanged — it is TCP-only by design; QUIC has its own datagram entry point).
- **`handler/redirect/udp`** — build is the verification gate (no tests in this module per
`x/CLAUDE.md`); a `play/` YAML for manual end-to-end verification (below).
- **`quic-dissector/` tests** — golden-packet table tests (`SniffQUIC` against captured QUIC Initial
datagrams as hex literals with known SNI), plus negative cases (short header, non-Initial, truncated,
bogus version). Fuzz test (`FuzzSniffQUIC`) for panic/DoS resistance. Mirror the fixture style in
`tls-dissector/dissector_test.go`. Capture golden packets at implementation time via a one-off
quic-go client or `openssl` to a test endpoint; commit as hex.
## Files to add / modify
### New repo: `quic-dissector/`
| File | Change |
|------|--------|
| `quic-dissector/go.mod` | **NEW** — module `github.com/go-gost/quic-dissector`, deps: `tls-dissector` + `golang.org/x/crypto` |
| `quic-dissector/dissector.go` | **NEW** — `SniffQUIC` public API |
| `quic-dissector/dissector_test.go` | **NEW** — golden-packet unit tests + fuzz |
| `quic-dissector/internal/quic/varint.go` | **NEW** — QUIC varint (~5 lines) |
| `quic-dissector/internal/quic/hkdf.go` | **NEW** — HKDF-Expand-Label per RFC 8446 §7.1 (~30 lines) |
| `quic-dissector/internal/quic/initial.go` | **NEW** — header protection removal + AEAD decrypt + CRYPTO reassembly |
| `quic-dissector/CLAUDE.md` | **NEW** — build/test instructions |
| `quic-dissector/.github/workflows/ci.yml` | **NEW** — CI (vet + test + fuzz loop, mirror tls-dissector) |
### Existing repo: `x/`
| File | Change |
|------|--------|
| `x/go.mod` | add `github.com/go-gost/quic-dissector v0.1.0` |
| `x/internal/util/sniffing/sniff.go` | add `ProtoQUIC` constant |
| `x/handler/redirect/udp/metadata.go` | add `sniffing`, `sniffingTimeout` fields + parse keys |
| `x/handler/redirect/udp/handler.go` | sniff-first-datagram branch (bypass + recording + prepend) |
| `play/quic-sniff.yaml` (or similar) | manual e2e config (TPROXY UDP → QUIC upstream w/ bypass rule) |
Reused, existing code (do not duplicate): `dissector.ParseClientHello` (`tls-dissector/dissector.go`),
`sniffing.DefaultReadTimeout`, `xnet.NewReadWriteConn`/`Pipe`,
`xrecorder.TLSRecorderObject`, `xbypass.ErrBypass`, `mdutil.GetBool/GetDuration`.
## Verification
1. **`quic-dissector/` build + test** (own CI, mirror `tls-dissector`):
`cd quic-dissector && go build ./... && go vet ./... && go test -v -cover ./...`
Golden packets must round-trip the expected SNI; negative cases must return `ErrNotQUIC` without
false positives. Fuzz test runs 30s in CI.
2. **`x/` build + vet** (per `x/CLAUDE.md`, the module's verification path):
`cd x && go build ./... && go vet ./...`
Confirms the `quic-dissector` import compiles and the handler integration is correct.
3. **Manual e2e** (transparent UDP redirect): run the `play/` config with `sniffing: true` and a
bypass rule on a known QUIC hostname; `curl --http3 https://<host>` through the redirect and
confirm (a) the bypass rule fires for the SNI, (b) non-bypassed QUIC still reaches the upstream,
(c) recorder logs show `ro.TLS.ServerName` populated.
## Notes / scope boundaries
- **Scope A only** — sniff SNI + transparent forward. No QUIC MITM/termination (Scope B).
- **QUIC v1 + draft-29** initially (matches proven Xray code). QUIC **v2** (`0x6b3343cf`, different
salt + inverted type bits, RFC 9369) is a small follow-up enhancement — noted in a code comment.
- **Single-datagram** ClientHello parsing; cross-datagram reassembly out of scope (documented).
- **Separate repo** (`quic-dissector/`) — follows the `tls-dissector/` pattern. The parser is
self-contained, independently versioned, and reusable by projects outside GOST. The `x/` module
consumes it as a standard Go dependency.
- **No quic-go dependency** — varint and HKDF-Expand-Label are self-contained implementations
(~35 lines total). Only external dep is `golang.org/x/crypto` (hkdf).
- Comments in **English only** (project convention). Doc comments on all exported symbols.
@@ -0,0 +1,129 @@
# go-gost/gost#765 — IPv6 TPROXY UDP fails through SOCKS5 with `relay=udp`
**Status**: Open, `NeedsInvestigation`
**Reported**: 2025-08-19
**Reporter**: @przemyslaw0
## Summary
When GOST uses TPROXY (`redu` listener) to intercept IPv6 UDP traffic and forwards it through a SOCKS5 proxy chain with `relay=udp`, responses never come back — `outputBytes: 0` for all sessions, DNS queries time out.
## Reproduction
```
gost -L "redu://:1111?netns=ns-a" -F "socks5://127.0.0.1:1080?relay=udp"
```
TPROXY-intercepted IPv6 DNS packets (`[2606:4700:4700::1001]:53`) forwarded through 3proxy SOCKS5 server. DNS queries time out with no response data.
## Root Cause
The primary bug is in `relayUDP()` at `x/connector/socks/v5/connector.go:216`. When `relay=udp` is used (standard SOCKS5 UDP ASSOCIATE per RFC 1928), GOST does not handle the server's relay bind address correctly.
### Bug: Wildcard relay address not substituted
When 3proxy returns `0.0.0.0:0` as the relay endpoint (visible in 3proxy logs: `UDPMAP 0.0.0.0:0`), GOST dials directly to this literal address:
```go
// connector.go:216
cc, err := opts.Dialer.Dial(ctx, "udp", reply.Addr.String())
```
This creates a UDP socket "connected" to `0.0.0.0:0`, which is not a usable destination. Per RFC 1928 practice, when the SOCKS5 server returns a wildcard address (`0.0.0.0` or `[::]`), the client should substitute it with the TCP connection's remote address (the SOCKS5 server's actual address).
**Compare with GOST's own SOCKS5 server** in `x/handler/socks/v5/udp.go:63`, which correctly substitutes the wildcard:
```go
saddr.Host, _, _ = net.SplitHostPort(conn.LocalAddr().String())
```
The client-side `relayUDP()` does NOT perform this substitution — it uses `reply.Addr` verbatim.
### Additional concern: earlier IPv6 address corruption
In an earlier test (2024-07-11, [3proxy/3proxy#560](https://github.com/3proxy/3proxy/issues/560#issuecomment-2224053601)), the user noted "The IPv6 address received by 3proxy from GOST is received incomplete." The address `2606:4700:4700::1001` arrived as `4700::1001:0:0:53` — a 2-byte left shift of the raw bytes, suggesting possible SOCKS5 UDP header encoding/parsing issues.
In the later test (2025-08-19, the go-gost issue), the address in the 3proxy log appears correct (`2606:4700:4700::1001:53`), though it's ambiguous whether the `:53` is the port or part of the address. This may have been fixed in a GOST version update, or the corruption was a 3proxy log formatting issue.
## Evidence from Logs
**GOST log** (session lifecycle):
- Receives TPROXY UDP from `[fd00::2]:59871` → `[2606:4700:4700::1001]:53`
- `src: 127.0.0.1:42587` — the outbound relay socket's local address (IPv4, expected)
- `inputBytes: 40, outputBytes: 0` — sent 40-byte DNS query, received 0 bytes back
- `duration: 30s` — timed out
**3proxy log**:
- `127.0.0.1:53888 2606:4700:4700::1001:53 40 0 0 UDPMAP 0.0.0.0:0`
- Received 40 bytes from client, returned 0 bytes to client
- Relay bound to `0.0.0.0:0` (wildcard)
## Data Flow Trace
```
dig @2606:4700:4700::1001
│
▼ (TPROXY intercepts IPv6 UDP)
redu listener (::1:1111)
│ conn.LocalAddr() = [2606:4700:4700::1001]:53
│ conn.RemoteAddr() = [fd00::2]:59871
▼
handler.redirect/udp → Router.Dial(ctx, "udp", "[2606:4700:4700::1001]:53")
│
▼ chain: socks5://127.0.0.1:1080?relay=udp
TCP connect to 127.0.0.1:1080
│ SOCKS5 handshake
▼
socks5Connector.relayUDP()
│ sends CmdUdp (0x03) with nil address
│ receives reply: bind = 0.0.0.0:0 ← WILDCARD
▼
opts.Dialer.Dial(ctx, "udp", "0.0.0.0:0") ← BUG: dials to wildcard
│ creates IPv4 UDP socket connected to 0.0.0.0:0
▼
udpRelayConn.WriteTo(data, [2606:4700:4700::1001]:53)
│ wraps in SOCKS5 UDP header (correct IPv6 ATYP=4)
│ sends via c.udpConn.Write() → to 0.0.0.0:0 ← goes nowhere useful
▼
outputBytes: 0 — no response received
```
## Key Files
| File | Role |
|------|------|
| `x/connector/socks/v5/connector.go` | `relayUDP()` — the buggy method (line 195-234) |
| `x/connector/socks/v5/conn.go` | `udpRelayConn` — SOCKS5 UDP encapsulation (ReadFrom/WriteTo) |
| `x/handler/socks/v5/udp.go` | GOST's own SOCKS5 server `handleUDP()` — has correct wildcard substitution |
| `gosocks5/socks5.go` | SOCKS5 protocol: `Addr`, `UDPHeader`, `UDPDatagram` encoding |
| `x/handler/redirect/udp/handler.go` | Redu handler — TPROXY UDP forwarding |
| `x/listener/redirect/udp/listener_linux.go` | Redu listener — Linux TPROXY socket handling |
## Proposed Fix
In `relayUDP()` at `x/connector/socks/v5/connector.go`, after receiving the SOCKS5 UDP ASSOCIATE reply (line 213), substitute wildcard addresses:
```go
// After reply is read and verified
bindAddr := reply.Addr
// Per RFC 1928: if server returns 0.0.0.0 or [::], substitute
// with the TCP connection's remote address (the SOCKS5 server).
if bindAddr.Host == "0.0.0.0" || bindAddr.Host == "::" {
host, _, _ := net.SplitHostPort(conn.RemoteAddr().String())
bindAddr.Host = host
}
log.Debugf("bind on: %v", bindAddr)
cc, err := opts.Dialer.Dial(ctx, "udp", bindAddr.String())
```
## Workaround
Use GOST's proprietary UDP tunnel mode (default, without `relay=udp`), which tunnels SOCKS5-encapsulated UDP datagrams over the existing TCP connection. This avoids the separate UDP relay socket entirely:
```
gost -L "redu://:1111?netns=ns-a" -F "socks5://127.0.0.1:1080"
```
**Caveat**: This uses GOST's proprietary `CmdUDPTun` (0xF3), which 3proxy does NOT support. For 3proxy specifically, the `relay=udp` fix is required.
+156
View File
@@ -0,0 +1,156 @@
# Fix: go-gost/gost#785 — IP Alias Interface Binding (Revised)
## Context
The initial fix (commit `caa82f2`) skipped `SO_BINDTODEVICE` only for loopback interfaces. The user's follow-up comment reveals this is **insufficient** — they also use a custom `ip-aliases` **dummy interface** (not loopback) for IP aliases:
```
3: ip-aliases: <BROADCAST,NOARP,UP,LOWER_UP> mtu 1500
inet 10.2.2.2/32 scope global ip-aliases
inet 10.1.1.1/32 scope global ip-aliases
inet6 fc00::beef/128 scope global
```
When `interface: "10.1.1.1"` is specified:
1. `findInterfaceByIP(10.1.1.1)` resolves to `"ip-aliases"` (not `"lo"`)
2. `SO_BINDTODEVICE("ip-aliases")` is called → forces traffic through a dummy interface with no route to external hosts → **fails**
The user also tested with IPs on `lo` (`10.3.3.3`, `fc33::1`) and got `dial tcp: address [fc33::1]:0: no suitable address found` — this is a separate IPv6→IPv4 mismatch issue (expected), not a `SO_BINDTODEVICE` problem.
## Root Cause
When the `interface` config is specified as an **IP address**, GOST resolves it to an interface name and calls `SO_BINDTODEVICE` on that interface. But the user's intent is source IP binding for policy routing, not device binding. `SO_BINDTODEVICE` on a dummy/loopback interface breaks outbound connectivity.
`SO_BINDTODEVICE` is only meaningful when the user specifies an actual interface **name** (like `eth0`) to force traffic through a specific physical NIC.
## Fix: Skip `SO_BINDTODEVICE` when interface was specified as IP
Track whether `ParseInterfaceAddr`'s input was an IP address or an interface name. When it was an IP, skip `SO_BINDTODEVICE` — source IP binding via `LocalAddr` is sufficient for policy routing.
### Files to modify
#### 1. `internal/net/addr.go` — `ParseInterfaceAddr`
Add a boolean `isIP` return value:
```go
func ParseInterfaceAddr(ifceName, network string) (ifce string, addr []net.Addr, isIP bool, err error) {
if ifceName == "" {
addr = append(addr, nil)
return
}
ip := net.ParseIP(ifceName)
if ip == nil {
// Interface NAME — SO_BINDTODEVICE is meaningful
isIP = false
var ife *net.Interface
ife, err = net.InterfaceByName(ifceName)
if err != nil {
return
}
var addrs []net.Addr
addrs, err = ife.Addrs()
if err != nil {
return
}
if len(addrs) == 0 {
err = fmt.Errorf("addr not found for interface %s", ifceName)
return
}
ifce = ifceName
for _, addr_ := range addrs {
if ipNet, ok := addr_.(*net.IPNet); ok {
addr = append(addr, ipToAddr(ipNet.IP, network))
}
}
} else {
// IP ADDRESS — skip SO_BINDTODEVICE, use LocalAddr binding only
isIP = true
ifce, err = findInterfaceByIP(ip)
if err != nil {
return
}
addr = []net.Addr{ipToAddr(ip, network)}
}
return
}
```
#### 2. `internal/net/dialer/dialer.go` — `Dial` and `dialOnce`
Pass `bindToDevice` flag through:
In `Dial()` (line ~93-118):
```go
ifces := strings.Split(d.Interface, ",")
for _, ifce := range ifces {
strict := strings.HasSuffix(ifce, "!")
ifce = strings.TrimSuffix(ifce, "!")
var ifceName string
var ifAddrs []net.Addr
ifceName, ifAddrs, isIP, err = xnet.ParseInterfaceAddr(ifce, network)
if err != nil && strict {
return
}
for _, ifAddr := range ifAddrs {
conn, err = d.dialOnce(ctx, network, addr, ifceName, ifAddr, !isIP, log)
if err == nil {
return
}
log.Debugf("dial %s/%s via interface %s@%v failed: %s", addr, network, ifceName, ifAddr, err)
if strict &&
!strings.Contains(err.Error(), "no suitable address found") &&
!strings.Contains(err.Error(), "mismatched local address type") {
return
}
}
}
```
In `dialOnce()` — add `bindToDevice bool` parameter, guard `bindDevice` calls:
```go
func (d *Dialer) dialOnce(ctx context.Context, network, addr, ifceName string, ifAddr net.Addr, bindToDevice bool, log logger.Logger) (net.Conn, error) {
// ... existing code ...
// In both UDP and TCP branches, change:
// if ifceName != "" {
// to:
// if ifceName != "" && bindToDevice {
}
```
#### 3. `internal/net/dialer/dialer_linux.go` — Revert loopback-specific check
Remove the loopback check added in `caa82f2` — the broader `isIP`-based fix supersedes it:
```go
func bindDevice(network, address string, fd uintptr, ifceName string) error {
if ifceName == "" {
return nil
}
host, _, _ := net.SplitHostPort(address)
if ip := net.ParseIP(host); ip != nil && !ip.IsGlobalUnicast() {
return nil
}
return unix.BindToDevice(int(fd), ifceName)
}
```
#### 4. `internal/net/addr_test.go` — Update `TestParseInterfaceAddr`
Update test calls to use new 4-return-value signature.
#### 5. `internal/net/dialer/dialer_linux_test.go` — Add test for loopback skip
Add a test verifying `bindDevice` is not called for IP-resolved interfaces (the loopback check is no longer in `bindDevice` itself, but the caller now skips it).
### Verification
```bash
cd x && go build ./... && go vet ./...
cd x && go test ./internal/net/... -v
cd x && go test ./internal/net/dialer/... -v
```
+166
View File
@@ -0,0 +1,166 @@
# Fix gost#803: Chain silently bypasses failed hops
## Context
Issue: [go-gost/gost#803](https://github.com/go-gost/gost/issues/803) — When all nodes in a chain hop are unreachable, GOST silently truncates the chain and routes traffic through only the remaining hops or falls back to direct dial. This is a **security concern**: users think traffic is going through the full proxy chain but it's being silently bypassed.
Maintainer response (Oct 2025): "后面会增加控制参数" (will add a control parameter later). No fix implemented yet.
### Root Cause Chain (4 interacting bugs)
1. **`Chain.Route()` returns a truncated route** ([x/chain/chain.go:134-136](x/chain/chain.go#L134-L136)): When `h.Select()` returns nil (no usable node in a hop), the method returns the partial route with only nodes from previous hops — not nil. This means the caller can't detect that hops are missing.
2. **`Router.dial()` only falls back when route is nil** ([x/chain/router.go:142-144](x/chain/router.go#L142-L144)): The check `if route == nil { route = DefaultRoute }` fires ONLY for nil routes. Since `Chain.Route()` never returns nil for a partial route, the DirectRoute fallback is only triggered for the all-hops-failed case.
3. **`FailFilter` has `len(vs) <= 1` guard** ([x/selector/filter.go:28-29](x/selector/filter.go#L28-L29)): A single failed node passes through the filter unchanged. This means a hop with 1 node that has been marked as failed still gets selected.
4. **`chainHop.Select()` returns single node directly** ([x/hop/hop.go:202-203](x/hop/hop.go#L202-L203)): When `len(nodes) == 1`, the node is returned immediately — bypassing the selector and its FailFilter entirely.
### User's Configuration (reporter)
```
Service (socks5 :9000) → hop-0 (local Clash socks5 :9001) → hop-1 (2 remote grpc servers, round-robin + failover)
```
When both remote servers in hop-1 are unreachable: hop-0 is selected, hop-1 returns nil → `Chain.Route()` returns route with hop-0 only → traffic flows through Clash only, never reaching the remote servers. **Silent security degradation.**
## Proposed Fix
### Change 1: `Chain.Route()` — return nil on hop failure
**File**: `x/chain/chain.go`
When `h.Select()` returns nil (no eligible node found for a hop), return `nil` instead of the partial `rt`. This signals "no usable route" to the caller.
```go
// Before (line 134-136):
if node == nil {
return rt
}
// After:
if node == nil {
return nil
}
```
A nil return from `Chain.Route()` matches the documented contract: "Route builds a route by selecting one node from each hop." If a hop can't provide a node, the route is not built.
### Change 2: `Router.dial()` — error on chain failure instead of direct-dial fallback
**File**: `x/chain/router.go`
When a chain is configured (`r.options.Chain != nil`) but returns nil route, the router should return an error rather than falling back to `DefaultRoute` (direct dial). This prevents the silent bypass.
```go
// Current (line 142-144):
if route == nil {
route = DefaultRoute
}
// Replace with:
if route == nil {
if r.options.Chain != nil {
return nil, fmt.Errorf("no available route: all hops failed")
}
route = DefaultRoute
}
```
### Change 3: `FailFilter` — remove `len(vs) <= 1` guard
**File**: `x/selector/filter.go`
Remove the early return that passes through single nodes. A failed node should be filtered regardless of how many candidates exist.
```go
// Before (line 28-30):
if len(vs) <= 1 {
return vs
}
// Remove this guard entirely.
```
The guard was likely a defensive optimization, but it creates a correctness issue: a single-node hop with that node failed will never be filtered, making failover ineffective for single-node hops.
**Interaction with Change 4**: After removing this guard, `chainHop.Select()` must always go through the selector for the FailFilter to see single-node hops. Change 4 ensures this.
### Change 4: `chainHop.Select()` — always use selector when configured
**File**: `x/hop/hop.go`
The single-node fast path (line 202-203) and the priority short-circuit (lines 210-219) both bypass the selector entirely. When a selector is configured, it should always be called — even for single-node or priority-short-circuit cases — so that FailFilter gets a chance to filter dead nodes.
```go
// Current (line 199-204):
if len(nodes) == 0 {
return nil
}
if len(nodes) == 1 {
return nodes[0]
}
// New behavior: if selector exists, always use it.
// Only use the fast path when there's no selector.
if len(nodes) == 0 {
return nil
}
if s := p.options.selector; s != nil {
// Always go through selector (applies FailFilter, BackupFilter, strategy)
return s.Select(ctx, nodes...)
}
if len(nodes) == 1 {
return nodes[0]
}
```
Additionally, the priority short-circuit (lines 210-219) should also yield to the selector when one is configured.
### Change 5 (Optional): Add `required` field to HopConfig
**Files**: `x/config/config.go`, `x/config/parsing/hop/parse.go`
If backward compatibility with the old "skip failed hops" behavior is desired, add a `Required` field to `HopConfig` and `hop.Options`. When `required: false`, a hop returning nil would be skipped (old behavior). When `required: true` (default), the chain fails if the hop has no nodes.
```go
// HopConfig addition:
type HopConfig struct {
// ... existing fields ...
Required *bool `yaml:"required,omitempty" json:"required,omitempty"`
}
```
Default: `required: true` (fail if hop has no nodes). Setting `required: false` restores the old "skip" behavior.
Then in `Chain.Route()`:
```go
if node == nil {
if h.Options().Required {
return nil // fail the chain
}
continue // skip this hop (old behavior)
}
```
However, this adds complexity. For the initial fix, I recommend **Changes 1-4 only** — they fix the core bug for all users. If the maintainer wants the old behavior as an opt-in, Change 5 can be added later.
## Files Modified
| File | Change |
|------|--------|
| `x/chain/chain.go` | `Chain.Route()`: return nil when hop fails |
| `x/chain/router.go` | `Router.dial()`: error when chain fails, don't fall back to direct |
| `x/selector/filter.go` | `FailFilter`: remove `len(vs) <= 1` guard |
| `x/hop/hop.go` | `chainHop.Select()`: always use selector when configured; remove single-node and priority fast paths |
## Verification
1. **Build**: `cd x && go build ./... && go vet ./...`
2. **Existing tests**: `cd x && CGO_ENABLED=1 go test -race ./...`
3. **Specific package tests**: `go test -v ./chain/... ./hop/... ./selector/...`
4. **Manual verification**: Add a test case for the multi-hop chain with failed middle hop scenario:
- Create a 2-hop chain where hop-1 has a single node that always fails
- Verify `Chain.Route()` returns nil
- Verify the router returns an error (not a direct connection)
5. **E2E**: Run relevant e2e tests if available: `go test ./gost/tests/e2e/ -v -timeout 10m`
+305
View File
@@ -0,0 +1,305 @@
# go-gost/gost#842: Per-App Transparent Proxy via cgroup + nftables + Policy Routing
## Context
Issue [#842](https://github.com/go-gost/gost/issues/842) requests per-application transparent proxying. Instead of the originally proposed `gost run` subcommand with namespace isolation, this plan enhances the existing `tungo` listener with cgroup-based traffic filtering — achieving the same goal with **far less code** (estimated ~180 lines vs. ~460+ lines for namespace approach) and **zero changes to the handler layer**.
## Core Insight
The `tungo` handler already handles all packets that arrive on the TUN device through gVisor netstack → GOST chain forwarding. The problem is not *how to process packets* — it's *how to get only the target process's packets onto the TUN*. The answer: **OS-level traffic marking + policy routing**.
```
所有进程 (除目标进程外的所有流量) ──→ main routing table ──→ eth0 ──→ 正常上网
目标进程 (在 cgroup /gost-tunnel 中)
└─→ nftables OUTPUT: socket cgroupv2 → meta mark set <mark>
└─→ ip rule fwmark <mark> → table <n>
└─→ default dev <tun> → TUN 设备 → tungo handler → 代理链
```
The `tungo` handler receives only the marked traffic. Zero changes to handler, gVisor, or chain code.
## Mechanism: nftables `socket cgroupv2` Match
Kernel 5.x+ and nftables support matching on the **cgroup v2 path** of the sending socket:
```bash
nft add rule inet gost output socket cgroupv2 level 2 "gost-tunnel" meta mark set 0x1
```
This marks every packet originating from processes in the cgroup `/sys/fs/cgroup/gost-tunnel`. The mark is then used by policy routing to divert those packets to the TUN device instead of the default route.
**Why this works:**
- Socket-level, not per-packet — applies to all packets from matching processes
- Works for TCP and UDP (OUTPUT chain)
- Child processes automatically inherit cgroup membership
- No LD_PRELOAD, no namespaces, no forked child processes
- The target process has zero knowledge it's being proxied (true transparent proxy)
## Design: Enhance `tungo` Listener
### New metadata fields
| Key | Type | Default | Description |
|-----|------|---------|-------------|
| `cgroup` / `tun.cgroup` | string | `""` (disabled) | cgroup v2 path relative to `/sys/fs/cgroup/` (e.g. `"gost-tunnel"`) |
| `cgroup.create` / `tun.cgroup.create` | bool | `false` | Auto-create the cgroup directory if it doesn't exist |
| `cgroup.cleanup` / `tun.cgroup.cleanup` | bool | `false` | Remove cgroup + nftables rules + ip rules on listener Close() |
When `cgroup` is empty (default), behavior is **completely unchanged** — no nftables rules, no policy routing.
### Modified files
#### 1. `x/listener/tungo/metadata.go` — parse new fields (~20 lines)
```go
type metadata struct {
config *tun_util.Config
guid string
cgroup struct {
path string // e.g. "gost-tunnel"
create bool // auto-create cgroup dir
cleanup bool // remove rules on close
}
}
```
Parse `cgroup`, `cgroup.create`, `cgroup.cleanup` from metadata.
#### 2. `x/listener/tungo/tun_linux.go` — add `setupCgroupFilter()` (~120 lines)
New function called from `createTun()` after TUN is up:
```go
func (l *tunListener) setupCgroupFilter() error {
if l.md.cgroup.path == "" {
return nil // feature not enabled
}
// 1. Verify cgroup exists or create it
cgroupPath := filepath.Join("/sys/fs/cgroup", l.md.cgroup.path)
if l.md.cgroup.create {
os.MkdirAll(cgroupPath, 0755)
}
if _, err := os.Stat(cgroupPath); err != nil {
return fmt.Errorf("cgroup %s does not exist: %v", cgroupPath, err)
}
// 2. Generate a unique table name and mark value
tableName := fmt.Sprintf("gost-%s", l.md.config.Name)
fwmark := l.md.cgroup.mark
if fwmark == 0 {
fwmark = 0x1 // default mark
}
tableID := l.md.cgroup.table
if tableID == 0 {
tableID = 100 // default routing table
}
// 3. Add nftables rule:
// nft add table inet <tableName>
// nft add chain inet <tableName> output "{ type filter hook output priority 0; }"
// nft add rule inet <tableName> output socket cgroupv2 level 2 "<path>" meta mark set <fwmark>
for _, cmd := range [][]string{
{"nft", "add", "table", "inet", tableName},
{"nft", "add", "chain", "inet", tableName, "output",
"{", "type", "filter", "hook", "output", "priority", "0", ";", "}"},
{"nft", "add", "rule", "inet", tableName, "output",
"socket", "cgroupv2", "level", "2",
strconv.Quote(l.md.cgroup.path),
"meta", "mark", "set", strconv.FormatUint(uint64(fwmark), 10)},
} {
if err := exec.Command(cmd[0], cmd[1:]...).Run(); err != nil {
return fmt.Errorf("nftables setup: %v", err)
}
}
// 4. Add policy routing:
// ip rule add fwmark <fwmark> table <tableID>
rule := netlink.NewRule()
rule.Mark = int(fwmark)
rule.Table = tableID
if err := netlink.RuleAdd(rule); err != nil && !os.IsExist(err) {
return fmt.Errorf("ip rule add fwmark %d table %d: %v", fwmark, tableID, err)
}
// 5. Add default route in the table pointing to TUN:
// ip route add default dev <tun-name> table <tableID>
// (this is already handled by existing addRoutes() — just add route to the table)
// 6. Store for cleanup
l.md.cgroup.tableName = tableName
l.md.cgroup.fwmark = fwmark
l.md.cgroup.tableID = tableID
return nil
}
```
#### 3. `x/listener/tungo/listener.go` — cleanup in Close() (~30 lines)
```go
func (l *tunListener) Close() error {
// ... existing close logic ...
// Cleanup nftables rules and policy routing
if l.md.cgroup.cleanup && l.md.cgroup.tableName != "" {
exec.Command("nft", "delete", "table", "inet", l.md.cgroup.tableName).Run()
rule := netlink.NewRule()
rule.Mark = int(l.md.cgroup.fwmark)
rule.Table = l.md.cgroup.tableID
netlink.RuleDel(rule)
}
return nil
}
```
#### 4. `x/config/parsing/parse.go` — new metadata key constants (~6 lines)
```go
const (
// MDKeyCgroup sets the cgroup v2 path for per-process traffic filtering
// on TUN/TAP/TUNGO listeners.
MDKeyCgroup = "cgroup"
// MDKeyCgroupCreate auto-creates the cgroup directory.
MDKeyCgroupCreate = "cgroup.create"
// MDKeyCgroupCleanup removes nftables rules and policy routing on listener close.
MDKeyCgroupCleanup = "cgroup.cleanup"
// MDKeyCgroupMark sets the fwmark value used for cgroup traffic (default 0x1).
MDKeyCgroupMark = "cgroup.mark"
)
```
### New dependency
`netlink.RuleAdd` / `netlink.RuleDel` from `github.com/vishvananda/netlink` — **already vendored** at `v1.1.1-0.20211118161826-650dca95af54`. No new Go dependency needed.
`nft` command — **already included** in Dockerfile (line 31: `apk add nftables`). No new system dependency needed.
## Usage
### YAML config
```yaml
services:
- name: per-app-proxy
addr: :0
handler:
type: tungo
chain: chain-0
listener:
type: tungo
metadata:
net: 10.0.0.1/24
mtu: 1420
cgroup: gost-tunnel # cgroup path (relative to /sys/fs/cgroup/)
cgroup.create: true # auto-create the cgroup
cgroup.cleanup: true # remove rules on exit
cgroup.mark: 0x1 # fwmark value (optional, default 0x1)
chains:
- name: chain-0
hops:
- nodes:
- addr: socks5://proxy:1080
connector:
type: socks5
dialer:
type: tcp
```
Then:
```bash
# Start GOST
sudo gost -C gost-run.yaml
# In another terminal, start target process in the cgroup
sudo cgexec -g cpu,memory,pids:gost-tunnel firefox
# OR on cgroup v2 systems:
echo $$ > /sys/fs/cgroup/gost-tunnel/cgroup.procs # current shell
firefox # all its traffic goes through TUN → proxy chain
```
### Inline CLI (via existing -L flag)
```bash
gost -L "tungo://:0?net=10.0.0.1/24&cgroup=gost-tunnel&cgroup.create=true&cgroup.cleanup=true" \
-F "socks5://proxy:1080"
```
### preUp/postDown alternative (zero code, already works)
GOST already supports `preUp`/`postDown` hooks that run shell commands. Users can achieve the same result today:
```yaml
services:
- name: per-app-proxy
addr: :0
handler:
type: tungo
chain: chain-0
listener:
type: tungo
metadata:
net: 10.0.0.1/24
preUp: |
nft add table inet gost-tun
nft add chain inet gost-tun output '{ type filter hook output priority 0; }'
nft add rule inet gost-tun output socket cgroupv2 level 2 "gost-tunnel" meta mark set 0x1
ip rule add fwmark 0x1 table 100
ip route add default dev tungo table 100
postDown: |
nft delete table inet gost-tun
ip rule del fwmark 0x1 table 100
ip route del default table 100
```
This already works with **zero code changes**. The code change proposed above simply makes it a first-class configuration option instead of requiring shell commands.
## Verification
### Unit-level verification
- `go build ./...` — entire workspace
- `go vet ./...` — x/ module
- Verify `nftables` command is available in test environment
- Verify `netlink.RuleAdd` works (already tested by existing `netlink.RouteReplace` usage)
### Integration test (manual)
```bash
# Start echo server on host
python3 -m http.server 8080 &
# Start GOST
sudo ./gost -C test-cgroup.yaml
# Put test process in cgroup and make a request
sudo cgexec -g cpu,memory:gost-tunnel curl http://example.com
# Verify traffic goes through proxy chain, not directly
```
### Existing tests unaffected
- No handler changes → all existing tungo handler tests pass
- No listener code path changes when `cgroup` is empty → all existing listener tests pass
## Summary
| Metric | Value |
|--------|-------|
| New code | ~180 lines |
| Modified files | 4 |
| New files | 0 |
| New dependencies | 0 (netlink already vendored, nftables already in Docker) |
| Handler changes | **0** |
| gVisor changes | **0** |
| Chain/Router changes | **0** |
| Service model changes | **0** |
| CLI conflict with `--` | **None** |
| Privilege required | CAP_NET_ADMIN (existing TUN requirement) + rw access to cgroup dir |
| Kernel requirement | 5.x+ (cgroup v2 + nftables `socket cgroupv2` match) |
| Child process tracking | Automatic (cgroup containment) |
| TCP support | Yes |
| UDP support | Yes (OUTPUT path; UDP replies match via conntrack) |
| Cleanup on exit | Configurable (`.cleanup=true`) |
+142
View File
@@ -0,0 +1,142 @@
# Issue #853 — 无状态 UDP 包转发(`stateless` metadata)
## 背景
Issue [#853](https://github.com/go-gost/gost/issues/853):用户需要一个类似 NGINX Stream Module 的
简单 UDP 包转发器。当前 GOST 的 `udp` 类型通过 `udp.NewListener()` 创建每客户端虚拟连接,用
`xnet.Pipe()` 做双向流拷贝——面向会话的模型与无状态"一包一响应"的语义不兼容,干扰了后端应用。
根本原因:
- `x/internal/net/udp/listener.go` 的 `udp.NewListener()` 按 remoteAddr 做 per-client 解复用,每个
客户端一个虚拟 `net.Conn`,带接收队列 (`rc chan`)
- `x/handler/forward/local/forward.go` 的 `handleRawForwarding()` 用 `xnet.Pipe()` 做双向流拷贝,
面向 stream 而非 datagram
- 即使 `keepalive=false`,虚拟 conn 在 TTL 内仍被 connPool 持有,同客户端快速发包可能丢失
## 方案:在现有 `udp` 代码上加 `stateless=true` metadata,辅以轻量重构
不新建服务类型,在现有 `udp` listener/handler 中增加无状态模式。通过重构 `udp.NewListener()` 消除
后台 goroutine 和 connPool 的空转开销,通过 handler 分支实现单包请求-响应路径。
### 修改的文件(6 个文件,~110 行改动,0 个新文件)
#### 1. `x/internal/net/udp/listener.go` — 核心重构(~50 行)
- **`ListenConfig`** 加 `Stateless bool` 字段
- **`NewListener()`** 加 stateless 分支:
- 不启动 `listenLoop()` goroutine
- 不创建 `connPool`
- 不创建 `cqueue` 通道
- **`Accept()`** stateless 模式下直接调用 `conn.ReadFrom(buf)` 阻塞读包,读到后创建轻量
`datagramConn` 返回(见下方),而非从 `cqueue` 通道读取
- **新增 `datagramConn` 结构体**(~15行,存于同文件):
```go
type datagramConn struct {
pc net.PacketConn // 底层套接字,用于 Write 回复
data []byte // 缓冲的单个数据包
offset int // Read 游标
localAddr net.Addr
remoteAddr net.Addr
}
```
实现 `net.Conn`:`Read` 从 data 拷贝,`Write` 调用 `pc.WriteTo(b, remoteAddr)`,
`Close` 无操作(无状态)。无 channel、无 mutex、无池逻辑。
#### 2. `x/listener/udp/metadata.go`(~5 行)
- `metadata` 结构体加 `stateless bool` 字段
- `parseMetadata()` 中加 `l.md.stateless = mdutil.GetBool(md, "stateless")`
#### 3. `x/listener/udp/listener.go`(~2 行)
- `Init()` 中 `ListenConfig` 传入 `Stateless: l.md.stateless`
#### 4. `x/handler/forward/local/metadata.go`(~10 行)
- `metadata` 结构体加 `stateless bool`、`bufferSize int` 字段
- `parseMetadata()` 中解析
#### 5. `x/handler/forward/local/forward.go`(~40 行)
- **新增 `handleRawDatagram()` 方法**:读一个包 → 转发 → 读回复 → 写回
```go
func (h *forwardHandler) handleRawDatagram(ctx context.Context, conn net.Conn, ...) error {
// 1. hop 选择目标节点(复用现有逻辑)
// 2. Router.Dial("udp", targetAddr)
// 3. buf := make([]byte, h.md.bufferSize)
// n, _ := conn.Read(buf)
// cc.Write(buf[:n])
// 4. n, _ = cc.Read(buf) // 带 readTimeout
// conn.Write(buf[:n])
// 5. cc.Close()
// return nil
}
```
复用 `handleRawForwarding` 的 hop 选择、Router.Dial、proxyproto、recorder、日志逻辑。
#### 6. `x/handler/forward/local/handler.go`(~5 行)
- `Handle()` 中加分支:
```go
if h.md.stateless {
return h.handleRawDatagram(ctx, conn, ro, log, network, proto)
}
```
### 不改动的文件
- `gost/cmd/gost/register.go` — 不需要新空白导入
- `x/config/cmd/cmd.go` — 现有 `udp` 类型自动匹配,无需修改
- `x/internal/net/udp/pool.go` — 不动,stateless 模式根本不会调用
### 关键设计决策
- **stateless 模式下零后台 goroutine**:`listenLoop` 和 `idleCheck` 都不启动,资源开销和新建类型
完全一致
- **`datagramConn` 极简**:只有 data + addr + offset,15 行,满足 `net.Conn` 接口但不携带任何
session 状态
- **handler 分支点只有一个**:`Handle()` 中一行 if 决定走 datagram 路径还是 stream 路径,之后
完全独立,互不污染
- **hop/forwarder 复用**:stateless 模式同样支持节点列表和选择器
- **`Router.Dial` 每次新建 outbound socket**:每个包一个新源端口,与 NGINX stream 行为一致;
如需持久源端口可以后续加连接池
### 配置示例
**CLI:**
```bash
gost -L "udp://:10000/127.0.0.1:2000?stateless=true"
```
**YAML:**
```yaml
services:
- name: udp-proxy
addr: ":10000"
listener:
type: udp
metadata:
stateless: true
handler:
type: udp
metadata:
stateless: true
forwarder:
nodes:
- name: target-0
addr: 127.0.0.1:2000
```
### 不纳入范围
- Outbound UDP socket 池化(持久源端口)
- ICMP 错误传播
- UDP 分片处理(>MTU 数据包)
## 验证
1. **Build**:`cd gost && go build ./cmd/gost/...` — 编译通过
2. **Vet**:`go vet ./x/internal/net/udp/... ./x/listener/udp/... ./x/handler/forward/local/...`
3. **手动测试**:启动 UDP echo server → 启动 gost `udp://...?stateless=true` → `nc -u` 发包验证往返正常
4. **多客户端**:两个不同端口同时发包,验证各自独立收到回复,无串扰
5. **全量编译**:`go build ./...` workspace 级别确认无回归
+55
View File
@@ -0,0 +1,55 @@
# Skip Authentication for Whitelisted Client IPs
## Context
Issue go-gost/gost#881 requests a `skipauth` metadata option that allows client IPs/networks (CIDR) to bypass authentication. Currently users needing this must run two services (one with auth, one without + admission + iptables DNAT), which is needlessly complex.
## Approach
Wrap the `auth.Authenticator` at service-parse time with a `WhitelistedAuthenticator` that checks the client IP (from context's `SrcAddr`) against the skipauth CIDR/IP list before delegating to the real auther. This covers ALL handlers transparently — HTTP, SOCKS4/5, relay, http2, SSH — since the wrapping happens in `ParseService` before the auther is injected.
### Files to create
**`x/auth/whitelist.go`** — `WhitelistedAuthenticator` wrapper type
- `NewWhitelistedAuthenticator(auther auth.Authenticator, patterns []string)` — parses patterns as IPs and CIDRs, builds two matchers
- `Authenticate(ctx, user, password, opts...)` — extracts `SrcAddr` from context, checks IP against matchers; if matched returns `("", true)` to skip auth; otherwise delegates to real auther
### Files to modify
**`x/config/parsing/service/parse.go`** — parse `skipauth` from handler metadata, wrap auther
- After line 303 (after auther group is built), before handler construction (line 349)
- Read `skipauth` from `cfg.Handler.Metadata`: try `GetStrings` first (YAML list), then fall back to comma-split `GetString` (URL query format)
- If non-empty and auther is set, wrap with `WhitelistedAuthenticator`
### Key details
| Aspect | Details |
|--------|---------|
| Matchers | Reuse `matcher.IPMatcher` + `matcher.CIDRMatcher` from `x/internal/matcher/` (same infrastructure as admission control) |
| Context | `xctx.SrcAddrFromContext(ctx)` provides the client IP; established in service loop before handler.Handle() |
| Config formats | YAML: `handler.metadata.skipauth: ["10.0.0.0/8", "192.168.1.1"]`; URL: `?skipauth=10.0.0.0/8,192.168.1.1` |
| Handlers covered | All handlers using `handler.AutherOption(auther)` — HTTP, SOCKS4/5, HTTP2, relay, SSH |
| Nil auther edge case | If no auther is configured but skipauth is set, whitelisted clients pass auth; non-whitelisted fail auth (returns false) — user mistake, but consistent behavior |
### Wire format for comma-separated URL values
The URL query `?skipauth=10.0.0.0/8,192.168.1.1` is stored in metadata as a single string. `mdutil.GetStrings` does not handle plain strings, so add explicit comma-splitting in ParseService:
```go
if ss := mdutil.GetStrings(hmd, "skipauth"); len(ss) > 0 {
skipauth = append(skipauth, ss...)
} else if s := mdutil.GetString(hmd, "skipauth"); s != "" {
for _, p := range strings.Split(s, ",") {
if p = strings.TrimSpace(p); p != "" {
skipauth = append(skipauth, p)
}
}
}
```
## Verification
```bash
cd /config/workspace/go-gost/x
go build ./... && go vet ./...
go test ./...
```
+197
View File
@@ -0,0 +1,197 @@
# Fix SOCKS5 UDP DNS Leak & Hostname Preservation (gost#882)
**Issue**: [go-gost/gost#882](https://github.com/go-gost/gost/issues/882) — two related problems in the SOCKS5 UDP ASSOCIATE path:
1. **DNS leak**: When a SOCKS5 UDP datagram carries ATYP=DOMAINNAME (e.g., `dns.google:53`), `socks.udpConn.ReadFrom()` calls `net.ResolveUDPAddr("udp", ...)` at [x/internal/util/socks/conn.go:131](x/internal/util/socks/conn.go#L131) — always uses system DNS (127.0.0.53), even when `resolver=1.1.1.1` is configured in the handler.
2. **Hostname not passed to relay chain**: Because `ReadFrom` resolves domain→IP immediately, the relay chain receives only the IP. For a setup like `-F relay://...`, the exit relay cannot resolve DNS using its own configured resolver.
## Context
### Affected code path
Client → SOCKS5 handler → `handleUDP()` → `socks.UDPConn(pc1)` → `udp.NewRelay(pc1, pc)` → relay loop:
```
Goroutine 1 (client→upstream):
pc1.ReadFrom() → parsed SOCKS5 UDP header, net.ResolveUDPAddr → IP
pc.WriteTo(data, IP)
Goroutine 2 (upstream→client):
pc.ReadFrom() → upstream response
pc1.WriteTo(data, addr)
```
### Key files
| File | Role |
|------|------|
| [x/internal/util/socks/conn.go](x/internal/util/socks/conn.go) | `udpConn.ReadFrom()` at L131 calls `net.ResolveUDPAddr` — **root cause of DNS leak** |
| [x/handler/socks/v5/udp.go](x/handler/socks/v5/udp.go) | `handleUDP()` at L31 — creates `pc1` (client-facing) and `pc` (upstream), sets up relay |
| [x/handler/socks/v5/metadata.go](x/internal/util/socks/conn.go) | Metadata already has `udpResolveDomain` at L29, used for outbound path only |
| [x/internal/util/relay/conn.go](x/internal/util/relay/conn.go) | `udpTunConn.ReadFrom()` at L57 also resolves via `net.ResolveUDPAddr` — mirror fix |
| [x/internal/net/udp/relay.go](x/internal/net/udp/relay.go) | `Relay.Run()` copies between pc1/pc2 — address from ReadFrom goes to WriteTo as-is |
### Key insight about relay chain vs direct
- **Relay chain** (e.g., `-F relay://...`): `pc` is a `UDPTunClientConn` whose `WriteTo()` calls `gosocks5.Addr.ParseFrom(addr.String())` — naturally preserves domain names as ATYP=Domain in the SOCKS5 UDP header, and the relay server handles resolution.
- **Direct** (no chain): `pc` is `*net.UDPConn`. Writing a domain address to it fails because `net.UDPConn.WriteTo` requires `*net.UDPAddr`.
So the fix is **NOT** to resolve in `ReadFrom` at all, but to let domains pass through naturally for the relay case. For the direct case, wrap `pc` with a resolver.
## Plan
### Step 1: Add custom `domainAddr` type
**File**: [x/internal/util/socks/conn.go](x/internal/util/socks/conn.go)
Add a `domainAddr` type that preserves a hostname as a `net.Addr`:
```go
// domainAddr is a net.Addr that preserves a domain name without resolving it.
// Used when a SOCKS5 UDP datagram carries ATYP=DOMAINNAME to avoid triggering
// DNS resolution via net.ResolveUDPAddr.
type domainAddr struct {
network string
host string
port int
}
func (a *domainAddr) Network() string { return a.network }
func (a *domainAddr) String() string { return net.JoinHostPort(a.host, strconv.Itoa(a.port)) }
```
This type ensures that `addr.String()` returns `"dns.google:53"` instead of `"8.8.8.8:53"`, preserving the hostname for downstream consumers like `UDPTunClientConn.WriteTo`.
### Step 2: Modify `udpConn.ReadFrom()` to not resolve domains
**File**: [x/internal/util/socks/conn.go](x/internal/util/socks/conn.go) — `udpConn.ReadFrom()` (L131), the **client-facing reader** where the leak is.
Replace:
```go
addr, err = net.ResolveUDPAddr("udp", socksAddr.String())
```
With:
```go
if net.ParseIP(socksAddr.Host) != nil {
addr, err = net.ResolveUDPAddr("udp", socksAddr.String())
} else {
addr = &domainAddr{network: "udp", host: socksAddr.Host, port: int(socksAddr.Port)}
}
```
If the SOCKS5 address is already an IP (ATYP IPv4/IPv6), return `*net.UDPAddr` as before. If it's a domain, return a `domainAddr` without resolution.
**Do NOT touch `udpTunConn.ReadFrom()`** at [x/internal/util/relay/conn.go:75](x/internal/util/relay/conn.go#L75). That is the *upstream* side — it reads relay-exit response *sources*, which are always resolved IPs. No leak there. Changing it risks reintroducing ATYP=Domain into client-bound responses, breaking the `udpResolveDomain` invariant for clients like tun2proxy/Surge. Add `strconv` to conn.go's imports for `domainAddr.String()`.
### Step 3: Add resolving wrapper for direct connections
**File**: [x/handler/socks/v5/udp.go](x/handler/socks/v5/udp.go)
Add a `resolvePacketConn` wrapper that resolves domain addresses to IPs before writing to the raw socket:
```go
// resolvePacketConn wraps a net.PacketConn and resolves domain addresses to
// IP addresses before writing. Used for direct UDP connections (no relay chain)
// where the underlying socket requires *net.UDPAddr.
type resolvePacketConn struct {
net.PacketConn
resolver resolver.Resolver
hostMapper hosts.HostMapper
}
func (c *resolvePacketConn) WriteTo(b []byte, addr net.Addr) (int, error) {
host, portStr, err := net.SplitHostPort(addr.String())
if err != nil {
return c.PacketConn.WriteTo(b, addr)
}
if net.ParseIP(host) != nil {
return c.PacketConn.WriteTo(b, addr)
}
var ips []net.IP
if c.hostMapper != nil {
ips, _ = c.hostMapper.Lookup(context.Background(), "ip", host)
}
if len(ips) == 0 && c.resolver != nil {
ips, _ = c.resolver.Resolve(context.Background(), "ip", host)
}
if len(ips) == 0 {
ips, _ = net.LookupIP(host)
}
if len(ips) == 0 {
return 0, fmt.Errorf("socks5 udp: cannot resolve %s", host)
}
// Prefer IPv4
ip := ips[0]
for _, candidate := range ips {
if candidate.To4() != nil {
ip = candidate
break
}
}
port, _ := strconv.Atoi(portStr)
return c.PacketConn.WriteTo(b, &net.UDPAddr{IP: ip, Port: port})
}
```
This mirrors the existing `domainResolvePacketConn.WriteTo()` (same file, L217-266) but wraps the *upstream* side.
### Step 4: Wire it up in `handleUDP()`
**File**: [x/handler/socks/v5/udp.go](x/handler/socks/v5/udp.go)
**Critical placement:** the `*net.UDPConn` detection MUST happen at the raw cast (line 86), BEFORE `pc = metrics.WrapPacketConn(...)` at line 112 — after the metrics wrap `pc` is always a metrics wrapper, so a late type assertion would always fail.
Insert immediately after the `pc, ok := c.(net.PacketConn)` block (after line 94, before line 112):
```go
// Wrap upstream PC with domain resolution for direct (no-chain) connections.
// Relay-chain connections (udpTunConn) encode domain addrs as ATYP=Domain
// in the SOCKS5 UDP header and let the exit resolve; raw *net.UDPConn.WriteTo
// needs *net.UDPAddr, so domains must be resolved here.
// Always wrap direct conns — udpConn.ReadFrom now returns domainAddr, which
// *net.UDPConn cannot consume; the resolver→system fallback lives inside.
if _, isDirect := pc.(*net.UDPConn); isDirect {
pc = &resolvePacketConn{
PacketConn: pc,
resolver: h.options.Router.Options().Resolver,
hostMapper: h.options.Router.Options().HostMapper,
}
}
```
Detection rationale:
- Direct `Router.Dial("udp", "")` returns `*net.UDPConn` (already a PacketConn, returned as-is by the router)
- Any chain (relay/socks5/ss) returns an address-encoding PacketConn (`udpTunConn`, `udpRelayConn`, ...) that preserves domains via `ParseFrom` — not `*net.UDPConn`, so correctly skipped
### Step 5: Backward compatibility notes
- `socks.UDPConn()` signature unchanged — callers unaffected
- Existing `domainResolvePacketConn` still works for the outbound path (upstream→client) when `udpResolveDomain=true`
- The new `resolvePacketConn` handles the inbound path (client→upstream) for direct connections
## Verification
1. **Build check**: `cd x && go build ./... && go vet ./...`
2. **Unit test**: `cd x && go test ./internal/util/socks/` (if tests exist)
3. **Manual e2e scenario**: Setup gost with relay chain:
```bash
# Relay server
gost -L 'relay://:1111?bind=true&resolver=1.1.1.1'
# SOCKS5 entry
gost -L 'socks5://:5999?udp=true&resolver=1.1.1.1' -F relay://127.0.0.1:1111
# Send UDP datagram to dns.google:53 via SOCKS5
# Before: strace shows connect(127.0.0.53) on entry
# After: no DNS query on entry, resolved at relay exit
```
4. **Direct (no chain)**: Verify resolver=1.1.1.1 is used instead of system DNS
```bash
gost -L 'socks5://:5999?udp=true&resolver=1.1.1.1'
```
+180
View File
@@ -0,0 +1,180 @@
# Pass verified mTLS client cert identity to auther plugins
**Issue:** [go-gost/gost#884](https://github.com/go-gost/gost/issues/884) — When a TLS listener with mTLS (`caFile`) is used, pass the verified client certificate's CN, SANs, and fingerprint to the auther plugin (gRPC and HTTP) so external authorization can use certificate-based identity.
## Context
Currently, the `AuthenticateRequest` proto/JSON sent to auth plugins only carries `{username, password, client, service}`. With an mTLS listener, the handler receives a `*tls.Conn` (embedded in wrapper layers), but never extracts or forwards the verified client certificate. The cert identity is invisible to auth plugins.
## Approach
Use **context propagation** — the simplest path: extract TLS peer cert from the connection after the TLS handshake completes (during `http.ReadRequest`), store in `context.Context`, and have auth plugin implementations extract it when building their requests.
This requires zero changes to the `auth.Authenticator` interface, zero changes to the HTTP handler's `Authenticate.Authenticate()` method in auth.go, and zero changes to call sites.
## Files to modify
### 1. `x/ctx/value.go` — Add PeerCert context type
Add a `PeerCert` struct and context get/set helpers, following the existing pattern (`Sid`, `ClientID`, etc.):
```go
type peerCertKey struct{}
type PeerCert struct {
CN string
SANs []string // DNSNames + EmailAddresses + URI strings
Fingerprint string // SHA-256 hex of cert.Raw
}
func ContextWithPeerCert(ctx context.Context, cert *PeerCert) context.Context {
return context.WithValue(ctx, peerCertKey{}, cert)
}
func PeerCertFromContext(ctx context.Context) *PeerCert {
v, _ := ctx.Value(peerCertKey{}).(*PeerCert)
return v
}
```
### 2. `x/ctx/value_test.go` — Add PeerCert tests
Add tests following the existing test patterns (set/get, empty, wrong-type, nil).
### 3. `x/handler/http/handler.go` — Extract TLS cert info in Handle()
In `Handle()`, save the raw conn before `conn = stats_wrapper.WrapConn(conn, &pStats)` (line 357). After `http.ReadRequest(br)` succeeds (line 382; TLS handshake is now complete), call a private unwrap+extract helper. If a verified peer cert is found, store in context:
```go
// At the top of Handle(), before stats wrapping (currently line 357):
rawConn := conn
// ... (existing code: stats_wrapper.WrapConn, checkRateLimit, bufio, ReadRequest) ...
// After http.ReadRequest succeeds (currently after line 382), TLS handshake is done:
if peerCert := getTLSPeerCert(rawConn); peerCert != nil {
ctx = xctx.ContextWithPeerCert(ctx, peerCert)
}
```
Add at the bottom of the file (a private function, not exported):
```go
// getTLSPeerCert extracts the verified mTLS client certificate identity from
// a connection by walking wrapper layers (traffic limiter, etc.) to reach the
// underlying *tls.Conn, then reading ConnectionState().VerifiedChains.
// It returns nil if the connection has no TLS peer certificate.
func getTLSPeerCert(conn net.Conn) *xctx.PeerCert {
for {
if tc, ok := conn.(interface{ ConnectionState() tls.ConnectionState }); ok {
cs := tc.ConnectionState()
if cs.HandshakeComplete && len(cs.VerifiedChains) > 0 && len(cs.VerifiedChains[0]) > 0 {
cert := cs.VerifiedChains[0][0]
fpr := sha256.Sum256(cert.Raw)
sans := make([]string, 0, len(cert.DNSNames)+len(cert.EmailAddresses)+len(cert.URIs))
sans = append(sans, cert.DNSNames...)
sans = append(sans, cert.EmailAddresses...)
for _, u := range cert.URIs {
sans = append(sans, u.String())
}
return &xctx.PeerCert{
CN: cert.Subject.CommonName,
SANs: sans,
Fingerprint: hex.EncodeToString(fpr[:]),
}
}
return nil
}
if uw, ok := conn.(interface{ UnwrapConn() net.Conn }); ok {
conn = uw.UnwrapConn()
continue
}
return nil
}
}
```
This handles both cases from the TLS listener's `Accept()`:
- **No traffic limiter**: `rawConn` is `tlsConn{*tls.Conn}` → `ConnectionState()` found on first iteration.
- **Traffic limiter enabled**: `rawConn` is `limitConn{tlsConn{*tls.Conn}}` → the unwrap loop peels `limitConn`, then finds `ConnectionState()`.
New imports: `crypto/sha256`, `crypto/tls`, `encoding/hex`. `crypto/tls` is already imported in many handler packages for sniffing; add it here.
### 4. `x/auth/plugin/grpc.go` — Forward PeerCert in AuthenticateRequest
In `Authenticate()`, extract `PeerCert` from context before building the proto request:
```go
if v := xctx.PeerCertFromContext(ctx); v != nil {
req.ClientCn = v.CN
req.ClientSan = v.SANs
req.ClientCertFingerprint = v.Fingerprint
}
```
Add the proto fields to the `AuthenticateRequest` literal.
### 5. `x/auth/plugin/http.go` — Forward PeerCert in JSON request
Same pattern as gRPC — extract from context, populate additional fields in `httpPluginRequest`.
Add to `httpPluginRequest`:
```go
ClientCn string `json:"clientCn,omitempty"`
ClientSan []string `json:"clientSan,omitempty"`
ClientCertFingerprint string `json:"clientCertFingerprint,omitempty"`
```
### 6. `plugin/auth/proto/auth.proto` — Add new fields
```protobuf
message AuthenticateRequest {
string username = 1;
string password = 2;
string client = 3;
string service = 4;
string client_cn = 5;
repeated string client_san = 6;
string client_cert_fingerprint = 7;
}
```
### 7. `plugin/auth/proto/auth.pb.go` — Regenerate
Run protoc from the `plugin/auth/proto/` directory:
```
protoc --go_out=. --go_opt=paths=source_relative \
--go-grpc_out=. --go-grpc_opt=paths=source_relative \
auth.proto
```
## What is NOT changed
- `core/auth/auth.go` — interface unchanged
- `x/handler/http/auth.go` — `Authenticate.Authenticate()` method unchanged
- Other handlers (SOCKS5, relay, etc.) — extracting PeerCert is opt-in per handler; only HTTP for now
- Other handlers (SOCKS5, relay, etc.) — extracting PeerCert is opt-in per handler; only HTTP for now
- The `getTLSPeerCert` unwrap loop handles all known wrapper layers generically: `limitConn` (traffic limiter), `serverConn` (conn limiter), `quotaConn` (quota limiter) — all implement `UnwrapConn() net.Conn`. If a future wrapper layer doesn't implement `UnwrapConn()`, the loop returns nil (no cert info), which is the safe default.
- `stats_wrapper.conn` and `xnet.readWriteConn` wrap the conn further but we save `rawConn` *before* these wrappings, so the unwrap loop doesn't need to handle them.
- The `tlsConn` type from `x/internal/util/tls/listener.go` embeds `*tls.Conn`, so `ConnectionState()` is promoted and found by the interface check.
## Review findings (已修复)
1. **`ConnectionState()` interface check fails through `limitConn`** — The TLS listener's `Accept()` wraps `tlsConn` in `limitConn` when a traffic limiter is configured. `limitConn` implements `UnwrapConn()` but NOT `ConnectionState()`. Fixed by replacing the direct interface check with an unwrap loop in `getTLSPeerCert()`.
2. **No new file needed** — The Plan agent proposed `x/internal/util/tls/cert.go` as a reusable helper. Since only the HTTP handler needs this for now, a private function in handler.go is the smaller diff. Can extract later if other handlers adopt it.
3. **`VerifiedChains` vs `PeerCertificates`** — Using `VerifiedChains[0][0]` (Go's verified chain leaf) rather than `PeerCertificates[0]` (all presented certs, potentially unverified). Correct: `VerifiedChains` only contains certs that passed `tls.RequireAndVerifyClientCert` validation.
## Verification
1. **Build check**: `cd x && go build ./...` — verify compilation of all modified packages
2. **Vet check**: `cd x && go vet ./...`
3. **Proto build**: `cd plugin && go build ./...` — verify proto changes compile
4. **Proto regeneration**: Run protoc, verify `.pb.go` generated correctly
5. **Context tests**: `cd x && go test ./ctx/ -v -run PeerCert` — new tests pass
6. **Full workspace**: `cd /config/workspace/go-gost && go build ./...` — no breakage
### Manual verification (config-driven)
Create a test config with an mTLS TLS listener + HTTP handler + auth plugin, and verify that the auth request includes `client_cn`, `client_san`, and `client_cert_fingerprint`. This would use the existing e2e test infrastructure in `gost/tests/e2e/`.
@@ -0,0 +1,150 @@
# Add SSE Streaming Rewrite Support to HTTP Sniffer
## Context
The HTTP sniffer forwarder (`x/internal/util/forwarder/`) needs to support
**streaming rewrite** for SSE (Server-Sent Events, `Content-Type: text/event-stream`)
responses. The use case is protocol conversion (e.g., Anthropic API ↔ OpenAI API)
via the rewriter plugin (`core/rewriter.Rewriter`), where each SSE event's data
payload must be transformed before reaching the client.
### Current behavior
- **SSE without rewrite/record:** `resp.Write(rw)` uses `io.Copy` internally to
stream the body — works correctly for basic proxying.
- **SSE + record:** `NewBody` wraps the body for recording, `resp.Write` streams,
partial capture is stored. Works, but misleading (incomplete capture).
- **SSE + rewrite:** `rewriteRespBody` exits early (`ContentLength <= 0` guard at
`sniffer_rewrite.go:45`) — the rewriter is never called. This is the gap.
- **Request body rewrite:** `rewriteReqBody` also exits early on `ContentLength <= 0`.
Non-streaming API requests (OpenAI → Anthropic) have a finite body and work fine.
### Approach
**SSE response body:** A new `io.ReadCloser` wrapper that reads SSE events from
the upstream body, applies the rewrite chain per-event, and returns rewritten
bytes via `Read()`. SSE is line-delimited: events separated by `\n\n`. Each event
is a self-contained message that can be independently rewritten.
**Non-streaming response/request body:** The existing `rewriteRespBody` /
`rewriteReqBody` already work for finite JSON bodies. No changes needed.
**Rewriter plugin server:** A standalone process (separate from sniffer) that
implements the `rewriter.Rewriter` interface via HTTP/gRPC. It handles:
- OpenAI request JSON → Anthropic request JSON
- Anthropic response JSON → OpenAI response JSON
- Anthropic SSE event bytes → OpenAI SSE event bytes (each event is independent)
## Changes
### 1. New file: `x/internal/util/forwarder/sniffer_sse.go`
**`isStreamingResponse(resp *http.Response) bool`** — returns `true` when
`Content-Type` starts with `"text/event-stream"`.
**`newSSERewriteBody(src io.ReadCloser, rewrites []chain.HTTPBodyRewriteSettings, contentType string) io.ReadCloser`** —
returns an `io.ReadCloser` that:
1. Uses `bufio.Scanner` with a custom `SplitFunc` that splits on `\n\n`
(handles both `\n\n` and `\r\n\r\n`)
2. For each scanned SSE event token, applies the rewrite chain (same logic as
`rewriteRespBody` in `sniffer_rewrite.go`):
- Content-Type filter from `rewrite.Type`
- Plugin rewriter: `rewrite.Rewriter.Rewrite(ctx, event)` if pattern matches
or no pattern set
- Regex replacement: `rewrite.Pattern.ReplaceAll(event, replacement)` if
no plugin rewriter
3. Rewritten event + `\n\n` terminator written to internal `bytes.Buffer`
4. `Read()` returns from this buffer
5. On upstream EOF, any remaining buffered data (incomplete final event) is
flushed as-is
**Edge cases:**
- Empty events (`\n\n\n\n`): pass through for each `\n\n`, no rewrite call
- Incomplete final event (no trailing `\n\n`): forwarded as-is when upstream closes
- No rewrites configured: `newSSERewriteBody` not called (guard in httpRoundTrip)
- `Content-Encoding` set: skip (same guard as `rewriteRespBody`)
- Plugin returns `error` or `nil` data: log warning, pass original event through
### 2. Modify `x/internal/util/forwarder/sniffer_http.go` — `httpRoundTrip()`
Insert **two guards** before `rewriteRespBody` call (line 449):
```go
// If the response is text/event-stream with body rewrites, switch to streaming
// rewrite: wrap resp.Body to rewrite each SSE event individually.
if isStreamingResponse(resp) && len(respBodyRewrites) > 0 {
resp.Body = newSSERewriteBody(resp.Body, respBodyRewrites, resp.Header.Get("Content-Type"))
}
```
The existing flow after this is untouched:
- `rewriteRespBody` at line 449: returns nil immediately due to `ContentLength <= 0`
- `NewBody` wrapping at line 454: wraps the SSE rewrite wrapper for recording
- `resp.Write(rw)` at line 457/461: streams rewritten events to client
### 3. No changes needed (rationale)
| File | Reason |
|------|--------|
| `x/internal/util/forwarder/sniffer_h2.go` | H2 handler uses `http.ResponseWriter` + `io.Copy` but has no rewrite support at all. SSE rewrite here is out of scope for this implementation. |
| `x/internal/util/forwarder/sniffer_rewrite.go` | `rewriteRespBody`/`rewriteReqBody` unchanged — they handle finite bodies only. `ContentLength <= 0` guard correct. |
| `core/rewriter/rewriter.go` | Interface `Rewriter.Rewrite(ctx, []byte) → ([]byte, error)` fits both non-streaming and per-event SSE rewrite. |
| `x/rewriter/plugin/grpc.go` / `http.go` | Plugin clients are agnostic to body content. No changes. |
| `core/chain/node.go` | `HTTPBodyRewriteSettings` / `HTTPNodeSettings` — all fields sufficient. |
| `x/config/parsing/` | Config parsing already wires rewriter plugins into nodes. |
### 4. Tests — `x/internal/util/forwarder/sniffer_test.go`
**`TestNewSSERewriteBody`** — unit test:
- SSE events rewritten by plugin rewriter (data: line replacement)
- SSE events with regex replacement
- Multiple rewrite rules applied sequentially
- Empty events pass through
- Incomplete final event flushed on close
- No rewrites (pass-through)
**`TestRewriteRespBody_SSE_ContentType`** — verify `rewriteRespBody` skips
`text/event-stream` (already tested by `TestRewriteRespBody_NegativeContentLength`
but add explicit test with proper content type).
### 5. Rewriter Plugin Server (separate from sniffer)
Located in `llm-api-converter/` (already exists):
- `cmd/root.go` — CLI flag parsing (`--addr`, `--model`, `--max-tokens`)
- `main.go` — entry point calling `rewriter.ListenAndServe()`
- `convert/convert.go` — `Convert(data []byte, opts) → ([]byte, error)`:
- Parse content type or detect format from JSON structure
- OpenAI request → Anthropic request conversion
- Anthropic response (non-streaming) → OpenAI response
- Anthropic SSE event bytes → OpenAI SSE event bytes
- `convert/types.go` — request/response types for both APIs
- `rewriter/server.go` — HTTP plugin server (accepts POST /rewrite)
- `rewriter/server_test.go` — validates all conversion paths
The plugin server is an external process started separately from gost. The sniffer
connects to it via the rewriter plugin client.
**Config example** (in gost YAML):
```yaml
rewriters:
- name: api-converter
plugin:
type: http # or grpc
addr: "127.0.0.1:8000"
```
## Files modified
| File | Change |
|------|--------|
| `x/internal/util/forwarder/sniffer_sse.go` | **NEW** — `isStreamingResponse()`, `newSSERewriteBody()` |
| `x/internal/util/forwarder/sniffer_http.go` | Add SSE body wrapping guard in `httpRoundTrip()` |
| `x/internal/util/forwarder/sniffer_test.go` | SSE rewrite tests |
## Verification
```bash
cd x && go build ./internal/util/forwarder/
cd x && go vet ./internal/util/forwarder/
cd x && go test ./internal/util/forwarder/ -v -run 'TestNewSSERewriteBody|TestRewriteRespBody' -count=1
```
@@ -0,0 +1,152 @@
# Generic Cache Component + HTTP Response Cache (issue #349)
## Context
Issue [#349](https://github.com/go-gost/gost/issues/349) requests an HTTP response cache so GOST can mirror static sites / CDNs (e.g. gh-pages) via its proxy. GOST currently re-forwards every request to upstream — no caching.
Rather than a one-off HTTP cache, we build a **generic cache component** (interface + memory backend + registry/config wiring), then make the HTTP proxy its first consumer. This is justified — not speculative — because two more consumers are committed roadmap:
- **DNS** ([resolver/cache.go](../../internal/util/resolver/cache.go)) already stores `[]byte`-serializable data with near-identical TTL + serve-stale + async-refresh semantics → migratable to the new interface later.
- **Redis / other backends** are planned beyond in-memory.
Scope decisions carried from prior review rounds:
- Interface trimmed to **3 methods** (`Get`/`Set`/`Delete`). 349's 7-method draft dropped `Has`/`Clear`/`Len` (awkward/expensive on Redis); `RefreshTTL` also dropped — its only consumer is DNS async-refresh ([handler.go:343](../../handler/dns/handler.go#L343)), which is not in this PR, and it is expressible as `Get`+`Set(WithTTL)`. Add it in the DNS-migration PR where it has a real caller and a native Redis `EXPIRE` mapping. The HTTP consumer uses `singleflight`, not `RefreshTTL`, for anti-stampede.
- **HTTP consumer lands in this same PR** as the validating consumer (349 had put it in "Phase 4" — wrong; it's the issue's actual ask). DNS migration and Redis backend are explicit follow-ups, not this PR.
- **Named shared store** referenced by name (`services[].cache: <name>`), not the per-chain-node config from the earlier `http-response-cache-for-reverse-proxy.md` draft (which was per-node, ambiguous with a shared store, and modified `core/chain/node.go`).
- Interface lives in **`core/cache/`** (first-class component, consistent with recorder/limiter — maintainer-approved core addition). HTTP policy configured via **handler metadata**, keeping the generic cache as dumb bytes.
## Architecture
```
caches: (named, registry) ──► registry.CacheRegistry().Get(name) ──► handler.CacheOption
│
services[].cache: cache-0 ▼
handler metadata: cache.ttl / cache.status.* / cache.serveStale ─► httpcache.Cache{Store, Policy, sf}
│ held by SnifferBuilder (built once in Init)
▼ Build() copies ref per connection (shared store)
HandleHTTP ──(before dial)── cache hit? ──► serve from cache
└─(miss)── singleflight ─► dial+roundtrip ─► tee response ─► Store
```
**Key correctness points** (from round-1 review of the sniffer):
- The `Sniffer` is built **per connection** (`SnifferBuilder.Build()`), so the cache/singleflight must live on the **shared** `SnifferBuilder` (built once in `Init`), never on `*Sniffer`. The store itself is a registry singleton, so all handlers/connections referencing the same name share it.
- Cache **hit check goes in `HandleHTTP` before `h.dial`** (and before keep-alive reuse), not inside `httpRoundTrip` — otherwise the upstream dial is already paid and nothing is saved.
- **SNI caching only works with MITM TLS termination.** Non-MITM `HandleTLS` pipes ciphertext ([sniffer_tls.go:116](../../internal/util/sniffing/sniffer_tls.go#L116)) and cannot cache. With `Certificate`+`PrivateKey` set, `terminateTLS` → `HandleHTTP`, so the HTTP cache glue covers SNI-with-MITM automatically — no extra code, just document the requirement.
## Files
### New — the generic component
**`core/cache/cache.go`** (~60 lines) — interface only. First-class component (maintainer-approved core addition). Mirrors `core/recorder/recorder.go` complexity.
```go
package cache
var ErrNotFound = errors.New("cache: key not found")
type Entry struct {
Data []byte
Expiration time.Time // zero = never expires
}
func (e *Entry) Expired() bool
func (e *Entry) TTL() time.Duration // remaining; <=0 expired, -1 never
type SetOptions struct{ TTL time.Duration }
type SetOption func(*SetOptions)
func WithTTL(d time.Duration) SetOption
// Cache is a concurrency-safe []byte KV store. Get returns ErrNotFound ONLY
// when the key is absent; expired entries are returned (with Expired()==true)
// to support serve-stale. Callers serialize/deserialize their own types.
type Cache interface {
Get(ctx context.Context, key string) (*Entry, error)
Set(ctx context.Context, key string, data []byte, opts ...SetOption) error
Delete(ctx context.Context, key string) error
}
```
Three methods only. `Get`/`Set` are load-bearing for HTTP; `Delete` is kept as a fundamental KV primitive (needed for a future PURGE API, unambiguous on every backend). `RefreshTTL` deliberately omitted — see scope decisions above.
**`core/handler/option.go`** (+~6 lines) — add `Cache cache.Cache` field + `func CacheOption(c cache.Cache) Option`, exactly like `RecordersOption` (option.go:40,115).
**`x/cache/memory.go`** (~180 lines) — memory backend implementing `cache.Cache`. Self-contained (no external lib), supersedes the trivial [x/internal/util/cache](../../internal/util/cache/cache.go) and the intent of patrickmn/go-cache. Features: `sync.RWMutex`, per-entry TTL (default from config, `WithTTL` override), `maxSize` + `maxBytes` bounds, eviction `oldest|lru`, optional background cleanup goroutine (stopped via `io.Closer` — registry's `Unregister` calls Close, see [registry.go Unregister](../../registry/registry.go)). No `init()` self-registration (recorder-style; instances are registered by the loader under their config name).
**`x/config/parsing/cache/parse.go`** (~40 lines) — `ParseCache(cfg *config.CacheConfig) cache.Cache`, dispatches on backend field. Only `Memory` now; `Redis`/`File`/`Plugin` are follow-ups (do NOT ship un-parseable empty backend structs). Mirrors [ParseRecorder](../../config/parsing/recorder/parse.go).
**`x/registry/cache.go`** (~40 lines) — `cacheRegistry` embedding `registry[cache.Cache]` + hot-reload `cacheWrapper` delegating each method to the current instance. Copy [x/registry/recorder.go](../../registry/recorder.go) exactly.
**`x/internal/util/httpcache/httpcache.go`** (~150 lines) — HTTP-specific glue over `cache.Cache`, shared by both sniffer packages. Holds all HTTP policy so the generic cache stays dumb bytes:
```go
type Policy struct {
DefaultTTL time.Duration
StatusTTL map[int]time.Duration // per-status (200/302→60m, 404→1m)
ServeStale bool
Methods map[string]bool // default GET, HEAD
MaxBodyBytes int
}
type Cache struct {
Store cache.Cache
Policy Policy
sf singleflight.Group // shared thundering-herd guard
}
func Key(method, host, uri string) string // "GET example.com/api"
func (c *Cache) Lookup(req) (resp *http.Response, stale bool, ok bool)
func (c *Cache) Cacheable(req, resp) bool // method + 2xx/3xx + configured statuses
func (c *Cache) TTLFor(status int) time.Duration
func (c *Cache) TeeWriter(w io.Writer) *teeWriter // captures up to MaxBodyBytes, discards if exceeded
func (c *Cache) Store(req, data []byte, ttl)
```
Serialization uses stdlib `resp.Write` / `http.ReadResponse` (pass the request method so HEAD parses body-less).
### Modified — config & wiring (copy recorder pattern verbatim)
- **`x/config/config.go`** — add `CacheConfig{Name, Memory *MemoryCache}` + `MemoryCache{TTL, MaxSize, MaxBytes, CleanupInterval, Eviction}`; `Config.Caches []*CacheConfig`; `ServiceConfig.Cache string` (name ref, like `Limiter string` at config.go:514).
- **`x/registry/registry.go`** — add `cacheReg` singleton (registry.go:49 style) + `CacheRegistry()` accessor (registry.go:187).
- **`x/config/loader/loader.go`** — add a `registerGroup` block iterating `cfg.Caches` → `ParseCache` → `CacheRegistry()` (copy recorder block loader.go:239-247).
- **`x/config/parsing/service/parse.go`** — look up `registry.CacheRegistry().Get(cfg.Cache)`, read HTTP cache policy from handler metadata, build `httpcache.Cache`, inject via `handler.CacheOption(...)` (alongside `RecordersOption`, parse.go:393). New metadata keys in [parsing/parse.go](../../config/parsing/parse.go): `cache.ttl`, `cache.status.<code>` (via `mdutil.GetStringMapString(md,"cache.status")`), `cache.serveStale`, `cache.methods`, `cache.maxBodyBytes`.
### Modified — HTTP consumers (the feature)
Both sniffer packages get the same edit. `SnifferBuilder` gains a `Cache *httpcache.Cache` field, `Build()` copies it onto `Sniffer`:
- **`x/internal/util/forwarder/`** (`sniffer.go`, `sniffer_http.go`) — serves forward/local + forward/remote.
- **`x/internal/util/sniffing/`** (`sniffer.go`, `sniffer_http.go`) — serves sni + http/connect (+ redirect).
In each `HandleHTTP`: **before `h.dial`** (and before keep-alive reuse at the re-dial branch), if `Cache != nil` and method is cacheable, `Lookup`; on fresh hit write to client (`resp.Write(rw)`), record, and continue the keep-alive loop with correct `shouldClose` (do **not** close the client conn). In `httpRoundTrip`: on miss, run dial+roundtrip inside `sf.Do(key,...)`; if `Cacheable`, wrap `rw` with `TeeWriter` so `resp.Write(tee)` captures bytes, then `Store` after a successful write. On upstream error with a stale entry and `ServeStale`, write the stale response instead of erroring.
The four dispatch sites that call `Build()` are unchanged in shape — they already build a fresh sniffer per connection; the cache reference rides along on the builder:
[forward/local/sniffing.go:79](../../handler/forward/local/sniffing.go#L79), [forward/remote/sniffing.go:79](../../handler/forward/remote/sniffing.go#L79), [sni/sniffing.go:104](../../handler/sni/sniffing.go#L104), [http/connect.go:126](../../handler/http/connect.go#L126). Each handler's `SnifferBuilder` is populated from `h.options.Cache` in `Init`.
## What is NOT in this PR (deferred, interface already accommodates)
- Redis / File / Plugin backends (Redis struct + impl land together in a follow-up).
- DNS / router / limiter / TLS-certpool migration to the new interface (Phase 2+).
- PURGE API, Vary, RFC 7234 conditional requests, disk persistence.
## Verification
```bash
cd x && go build ./... && go vet ./...
cd ../gost && go build ./cmd/gost/...
cd ../core && go build ./... && go vet ./... # interface + handler option
```
Unit tests:
- `x/cache/memory_test.go` — set/get/expire, `WithTTL` override, oldest & lru eviction, maxBytes bound, cleanup goroutine stop on Close, `Get` returns expired entry (serve-stale contract). Run with `CGO_ENABLED=1 go test -race`.
- `x/internal/util/httpcache/httpcache_test.go` — key format, per-status TTL, `Cacheable` (method/status matrix), teeWriter cutoff at `MaxBodyBytes`, response round-trip via `resp.Write`/`http.ReadResponse` incl. HEAD.
End-to-end (memory backend, HTTP proxy):
```yaml
caches:
- name: c0
memory: { maxSize: 1000, maxBytes: 268435456, eviction: lru }
services:
- name: web
addr: ":8080"
handler:
type: http
metadata: { cache: c0, cache.ttl: 60m, cache.status.404: 1m, cache.serveStale: true }
forwarder:
nodes: [{ name: up, addr: "backend:80" }]
```
Two identical GETs: first logs `<-> upstream`, second serves from cache (no upstream dial in trace). Kill the backend after warming the cache → with `serveStale: true`, the stale entry is still served.
```bash
cd gost && go run ./cmd/gost/... -C cache.yml -D # -D trace shows hit/miss/dial
```
For SNI: same works only when the service also has MITM certs configured (otherwise TLS is piped and uncacheable — documented behavior).
+102
View File
@@ -0,0 +1,102 @@
# Plan: Client-Controlled HTTP/WS Recording with Three Levels
## Context
The GOST server unconditionally records HTTP/WS traffic. The client needs per-request control
with three granularity levels.
## Design
**Header**: `Gost-Record` with three values (case-insensitive) (matches `Gost-Sid`, `Gost-Forwarded-Node` convention):
| Value | Behavior |
|-------|----------|
| *(absent)* or `full` | Full recording: metadata + body (existing behavior) |
| `headers` | Metadata only: method, URL, status, headers. Skip body/payload capture. |
| `off` | No recording at all. |
**Field**: `RecordMode string` on `HandlerRecorderObject` (`json:"-"`).
**Central guard** (`Record()`): `RecordMode == "off"` → skip entirely.
**Body guard** (all capture sites): `RecordMode == "headers" || RecordMode == "off"` → skip body.
**Per-request**: Set `ro.RecordMode` after cloning `ro` from the request header.
Each request independently controls its own recording level.
**Header stripped** before forwarding.
## Files to Modify (8 files)
### 1. `x/recorder/recorder.go` — RecordMode field + guard
Add field:
```go
RecordMode string `json:"-"` // "" = full, "headers" = metadata only, "off" = no recording
```
Update `Record()`:
```go
func (p *HandlerRecorderObject) Record(ctx context.Context, r recorder.Recorder) error {
if p == nil || r == nil || p.Time.IsZero() || p.RecordMode == "off" {
return nil
}
// ... existing logic unchanged
}
```
### 2. `x/handler/tunnel/entrypoint/ephttp.go` — httpRoundTrip
After ro clone, parse header:
```go
if v := req.Header.Get("Gost-Record"); v != "" {
ro.RecordMode = strings.ToLower(v)
}
req.Header.Del("Gost-Record")
```
Body capture: `opts.HTTPBody && ro.RecordMode != "headers" && ro.RecordMode != "off"` (lines 199, 252)
### 3. `x/handler/tunnel/entrypoint/epwebsocket.go` — copyWebsocketFrame
Body capture: `opts.HTTPBody && ro.RecordMode != "headers" && ro.RecordMode != "off"` (line 123)
### 4. `x/handler/http/proxy.go` — proxyRoundTrip
Same header parse as #2. Strip alongside existing proxy header stripping.
Body capture: `!ro.NoRecord` → `ro.RecordMode != "headers" && ro.RecordMode != "off"` (lines 180, 229)
### 5. `x/handler/http/websocket.go` — copyWebsocketFrame
Body capture: same pattern (line 105)
### 6. `x/internal/util/sniffing/sniffer_http.go` — httpRoundTrip
Same header parse + strip before `req.Write(cc)`.
Body capture: same pattern (lines 204, 264)
### 7. `x/internal/util/sniffing/sniffer_ws.go` — copyWebsocketFrame
Body capture: same pattern (line 108)
### 8. `x/internal/util/sniffing/sniffer_h2.go` — h2Handler.ServeHTTP
Same header parse + strip.
Body capture: same pattern (lines 142, 172)
## Propagation (unchanged from original design)
```
httpRoundTrip/proxyRoundTrip (clone ro, set RecordMode)
→ handleUpgradeResponse (receives cloned ro)
→ sniffingWebsocketFrame (clones: *ro2 = *ro)
→ copyWebsocketFrame (RecordMode propagates via clone)
→ ro.Record() → guarded by RecordMode
```
## Edge Cases
- **TLS passthrough / relay protocol**: No HTTP request. No header to check. Unaffected.
- **Handler `Handle()` outer deferred**: `ro.Time` zeroed before dispatch for HTTP paths. Unaffected.
## Verification
```bash
cd x && go build ./... && go vet ./...
```
@@ -0,0 +1,157 @@
# HTTP Response Cache for Reverse Proxy (Port Forwarding with HTTP Sniffer)
## Context
为 GOST 的反向代理(local port forward + HTTP sniffer)增加类似 nginx `proxy_cache` 最简子集的缓存功能。
GOST 目前做 HTTP 转发时不缓存任何响应——每个请求都完整转发到 upstream。对于可缓存的静态资源或 API 响应,这会浪费 upstream 带宽和延迟。
## Requirements
- **缓存粒度**:Method + Host + URI 作为缓存键(`GET example.com/api/users`)
- **过期策略**:TTL(可配置),忽略标准 HTTP Cache-Control 语义
- **存储**:纯内存,进程重启丢失
- **配置位置**:chain node 级别,与现有 `httpSettings`(rewriteURL, requestHeader 等)同级
- **范围**:最简可用,不实现 RFC 7234 完整语义
## Design
### Architecture
在 `httpRoundTrip` 的请求-响应周期中插入缓存层:
```
客户端请求 → HandleHTTP → httpRoundTrip
├─ 缓存命中 (未过期) → 直接写回客户端
└─ 缓存未命中/已过期 → dial upstream → tee writer 同时写客户端和捕获字节 → 写完后将捕获的完整响应存入缓存
```
**缓存键**:`Method + " " + Host + " " + URI`,例如 `GET example.com/api/users`。不同 Host 的相同 URI 可能返回不同内容,Host 不可省略。
**只缓存幂等请求**:GET/HEAD 缓存,POST/PATCH/DELETE 等非幂等方法不缓存。
**缓存实例**:存储在 `*Sniffer` 结构体上,首次遇到启用缓存的 node 时惰性初始化。一个 Sniffer 服务于一个 service,这是合理的缓存共享范围。
### Files to modify
#### 1. `core/chain/node.go` — 新增配置结构体
在 `HTTPNodeSettings` 中增加 `Cache` 字段:
```go
type HTTPNodeSettings struct {
// ... existing fields ...
Cache *HTTPCacheSettings `json:"cache,omitempty" yaml:"cache,omitempty"`
}
type HTTPCacheSettings struct {
Enabled bool `json:"enabled" yaml:"enabled"`
TTL time.Duration `json:"ttl" yaml:"ttl"` // 缓存过期时间,默认 60s
MaxEntries int `json:"maxEntries" yaml:"maxEntries"` // 最大条目数,默认 1000
MaxBodyBytes int `json:"maxBodyBytes" yaml:"maxBodyBytes"` // 缓存的最大响应体大小,默认 1MB
}
```
#### 2. `x/config/parsing/node/parse.go` — 配置解析
在 `parseHTTP` 函数中解析缓存配置到 `httpSettings.Cache`。
YAML 配置示例:
```yaml
nodes:
- name: cache-node
addr: upstream:8080
http:
cache:
enabled: true
ttl: 300s
maxEntries: 500
maxBodyBytes: 2097152 # 2MB
```
#### 3. `x/internal/util/forwarder/cache.go` — 新增缓存实现
```go
type httpCache struct {
mu sync.RWMutex
entries map[string]*cacheEntry
maxEntries int
}
type cacheEntry struct {
data []byte // resp.Write() 输出的完整 HTTP 响应(status line + headers + body)
expiresAt time.Time
}
```
- `get(key string) (*http.Response, error)` — 检查过期,返回克隆的响应
- `set(key string, data []byte, ttl time.Duration)` — 按 `[]byte` 存(已经 tee 捕获好了),满时淘汰最老条目
- `delete(key string)`
- `cleanup()` — 后台 goroutine 定期清理过期条目
- `teeWriter(w io.Writer, maxBytes int) *cacheTeeWriter` — 条件化 tee writer:同时写客户端和捕获字节,超过 `maxBytes` 时停止捕获(丢弃整个缓冲),避免为大响应体浪费内存
序列化/反序列化使用 `net/http` 标准 `resp.Write` / `http.ReadResponse`。
#### 4. `x/internal/util/forwarder/sniffer_http.go` — 集成缓存
在 `httpRoundTrip` 中两处插入:
- **函数开头**(~第 320 行,`dial` 之前):检查 `node.HTTP.Cache.Enabled`,若启用且方法为 GET/HEAD,查缓存。命中则 `cachedResp.Write(rw)` → 直接返回。此时 `cc` 为 nil,`defer cc.Close()` 需 guard nil。
- **响应写入处**(替换 ~第 566-571 行的 `resp.Write(rw)`):若缓存启用且响应满足条件(2xx/3xx,体大小未超标),用 `cache.teeWriter` 包装 `rw`,使得 `resp.Write(tee)` 同时写入客户端和捕获字节。写完后若捕获成功则同步存入缓存。不满足条件则直接用原来的 `resp.Write(rw)`。
**Singleflight 优化**:使用 `golang.org/x/sync/singleflight`(已在依赖中)防止同一缓存键的并发 miss 都去打 upstream——只有第一个请求真正转发,其他等待共享结果。
**缓存命中时的 Date 头**:回放的响应保留捕获时的原始 `Date`。这与 nginx `proxy_cache` 默认行为一致,不算 bug。如需覆盖可后续加配置项。
### Safety guards
| 机制 | 说明 |
|------|------|
| `maxEntries` | 条目数上限,超过则淘汰最老条目 |
| `maxBodyBytes` | 单响应体上限,超过不缓存 |
| 只缓存 GET/HEAD | POST/PATCH 等非幂等方法不缓存 |
| 只缓存 2xx/3xx | 不缓存错误响应 |
| 过期条目后台清理 | 独立 goroutine,60s 默认间隔 |
| `singleflight` | 防止 thundering herd |
### What is NOT included (and why)
- **磁盘持久化** — 用户选择纯内存
- **Cache-Control/ETag/条件请求** — 最简方案,不实现 RFC 7234
- **Vary 头处理** — 最简方案忽略
- **PURGE/缓存清理 API** — 后续需要再加
- **缓存统计/指标** — 后续集成到 metrics 体系
- **分布式缓存(Redis 等)** — 单进程场景不需要
## Complexity and Effort
| 维度 | 评估 |
|------|------|
| 新增文件 | 1 个(`cache.go` ~120 行) |
| 修改文件 | 3 个(`node.go` +5 行, `parse.go` +10 行, `sniffer_http.go` +20 行) |
| 总代码量 | ~155 行 |
| 新依赖 | 0(所有依赖已在 go.mod 中) |
| 复杂度 | 低 |
| 估时 | 半天开发 + 半天测试 |
## Risk Assessment
| 风险 | 等级 | 缓解 |
|------|------|------|
| 内存膨胀 | 中 | `maxEntries` + 最老淘汰 |
| 缓存错误响应 | 低 | 只缓存 2xx/3xx |
| 并发 miss 惊群 | 低 | `singleflight` 合并并发请求 |
| 大响应体占内存 | 中 | `maxBodyBytes` 上限 |
| 与 rewrite 管道交互 | 低 | 缓存在 rewrite **之后**存储,在 rewrite **之前**返回——即缓存的是最终改写后的响应 |
## Verification
1. **单元测试**:`cache_test.go` — 测试 set/get/expire/eviction/cleanup
2. **集成测试**:修改 `sniffer_test.go` — 模拟 upstream 返回可缓存响应,验证第二次请求命中缓存
3. **手动验证**:
```bash
# 启动 gost 作为反向代理,配置缓存
go run ./cmd/gost/... -L "tcp://:8080" -F "tcp://upstream:9090?http.cache.enabled=true&http.cache.ttl=30s"
# 两次相同的请求应该都在日志中显示 first request → upstream, second → cache hit
```
+125
View File
@@ -0,0 +1,125 @@
# Cmd Probe for Node Liveness
## Context
The routing matcher system (`Host`, `Path`, `ClientIP`, etc.) is entirely
static — no way to make dynamic decisions like "route to node A only during
business hours" or "check external conditions before selecting this node."
Original idea: add a `Cmd` matcher to the routing rule engine, running a shell
command per-request. That fails because `Match()` is on the hot path (every
connection, every node candidate) — per-request `fork + exec` is a non-starter.
The right home is the existing [node-liveness-probe plan](node-liveness-probe.md):
add a `cmd` probe type. Periodic control-plane check, not per-request
data-plane check.
## Design
Add a `"cmd"` probe type to the node liveness probe system. The probe runs a
user-provided command on the configured interval; exit 0 = healthy, non-zero
(including timeout) = mark dead.
```
probeNode → cmd probe → exec sh -c <command>
│
exit 0 → node.Marker().Reset() + SetProbeResult(ok)
exit ≠0 → node.Marker().Mark() + SetProbeResult(fail)
│
FailFilter skips marked nodes in hop selector
```
This reuses the **entire existing pipeline** — `FailFilter`,
`LowestLatencyStrategy`, `LatencyFilter`, goroutine lifecycle (`Close()`
cascade on reload) — with zero new infrastructure.
## Config
```yaml
hops:
- name: hop-1
nodes:
- name: biz-hours
addr: proxy1:1080
probe:
type: cmd
command: 'test $(date +%H) -ge 9 -a $(date +%H) -lt 18' # 9am-6pm only
interval: 30s
timeout: 5s
- name: with-check
addr: proxy2:1080
probe:
type: cmd
command: './scripts/health-check.sh'
interval: 60s
timeout: 10s
```
`ProbeConfig` gains a `Command` field. When `type: cmd` is specified, the
probe goroutine shells out to `sh -c <command>` instead of doing TCP/HTTP
dial.
## CmdProber
```go
package probe
import (
"context"
"fmt"
"os/exec"
"time"
)
type CmdProber struct {
Command string
Timeout time.Duration
}
func (p *CmdProber) Probe() error {
ctx, cancel := context.WithTimeout(context.Background(), p.Timeout)
defer cancel()
cmd := exec.CommandContext(ctx, "sh", "-c", p.Command)
if err := cmd.Run(); err != nil {
return fmt.Errorf("cmd probe: %w", err)
}
return nil
}
```
No env vars needed — the command runs periodically, not per-request.
## Implementation
| File | Change |
|------|--------|
| `core/chain/probe.go` | `ProbeConfig` + `Command` field, `ProbeTypeCmd` constant |
| `x/internal/probe/cmd.go` | **NEW** — `CmdProber` struct + `Probe()` method |
| `x/chain/node.go` (`probeNode` func) | Add `case chain.ProbeTypeCmd:` dispatch |
| `x/config/config.go` | `ProbeConfig` + `Command` field |
All other changes (goroutine lifecycle, marker integration, `FailFilter`,
`LowestLatencyStrategy`, `LatencyFilter`, `Close()` cascade) come from the
liveness probe plan with zero modification.
## Security
- Commands run as the same user as gost — no privilege escalation
- Config files are already trusted input (they can specify arbitrary
listeners, dialers, plugin binaries)
- Users who don't want this simply don't use `type: cmd`
## Skipped
- Per-request `Cmd` matcher in routing rules — wrong hot-path fit,
`exec` belongs on the control plane
## Verification
```bash
cd x && go build ./... && go vet ./...
# Manual: write a play/ config with cmd probe, run gost, observe logs
```
Sits inside the liveness probe plan's verification framework — node probe
tests cover cmd probe alongside tcp/http.
+145
View File
@@ -0,0 +1,145 @@
# failCodes — HTTP Status-Based Node Fail Marking
## Context
When GOST acts as HTTP reverse proxy (e.g. LLM API gateway), the upstream
backend can return HTTP 5xx/429 errors while the node itself is healthy at
the transport level. The existing Marker + FailFilter only catches dial-time
failures — there is no way to mark a node unhealthy based on HTTP response
codes.
`failCodes` adds a node-level HTTP setting defining which response status
codes trigger node marking. Cooldown is handled by the existing
`failTimeout`/`maxFails` selector config. No Marker interface changes needed.
No retry — the response is always relayed to the client as-is; marking only
affects subsequent node selection.
## Design
### 1. `core/chain/node.go` — `FailCodes` type
```go
// FailCodes lists response status codes that mark the node as failed.
// Codes < 100 are wildcard hundred-level prefixes (e.g. 5 → 5xx).
type FailCodes []int
func (fc FailCodes) Match(statusCode int) bool {
for _, code := range fc {
if (code < 100 && statusCode/100 == code) || statusCode == code {
return true
}
}
return false
}
```
`HTTPNodeSettings` gets one new field:
```go
FailCodes FailCodes
```
### 2. `x/config/config.go` — config struct
```go
// in HTTPNodeConfig:
FailCodes string `yaml:"failCodes,omitempty" json:"failCodes,omitempty"`
```
Plain comma-separated string — simpler than the original array-of-objects
approach, consistent with the "single string value" metadata convention.
### 3. `x/config/parsing/node/parse.go` — parsing
```go
func parseFailCodes(s string, log logger.Logger) chain.FailCodes {
var codes chain.FailCodes
for _, part := range strings.Split(s, ",") {
part = strings.TrimSpace(part)
if len(part) == 3 && strings.HasSuffix(part, "xx") {
if prefix, err := strconv.Atoi(part[:1]); err == nil && prefix > 0 {
codes = append(codes, prefix) // < 100 → wildcard
continue
}
} else if code, err := strconv.Atoi(part); err == nil && code >= 100 {
codes = append(codes, code)
continue
}
log.Warnf("failCodes: invalid token %q", part)
}
return codes
}
```
Wired inside the `if cfg.HTTP != nil` block in `ParseNode`:
```go
if v := strings.TrimSpace(cfg.HTTP.FailCodes); v != "" {
settings.FailCodes = parseFailCodes(v, nodeLogger)
}
```
### 4. `x/internal/util/forwarder/sniffer_http.go` — runtime
In `httpRoundTrip`, after `ro.HTTP.StatusCode = resp.StatusCode`:
```go
// failCodes: a matching upstream status marks the node failed so the
// selector's FailFilter skips it. The response still relays to the client;
// closing the connection makes the next client request re-select a node.
if hts := node.Options().HTTP; hts != nil && hts.FailCodes.Match(resp.StatusCode) {
log.Warnf("failCodes matched status %d for node %s", resp.StatusCode, node.Name)
if marker := node.Marker(); marker != nil {
marker.Mark()
}
shouldClose = true
}
```
No early return — the response writes through to the client normally.
`shouldClose = true` closes the upstream connection after this request so the
next client request triggers a fresh `hop.Select()`.
## YAML Example
```yaml
chains:
- name: llm-chain
hops:
- name: llm-hop
selector:
strategy: round
maxFails: 1
failTimeout: 15s
nodes:
- name: deepseek-proxy
addr: api.deepseek.com:443
connector: {type: http}
http:
failCodes: "429,502,503,504"
```
**Important**: `maxFails: 1` is recommended. The `dial()` function calls
`marker.Reset()` on successful TCP dials, which can wipe HTTP-based marks
before `maxFails` is reached. With `maxFails: 1`, one `Mark()` immediately
excludes the node.
## Scope
- **Implemented**: sniffer-based forwarder path (`forward/local`,
`forward/remote`).
- **Future work**: HTTP handler direct path (`x/handler/http/proxy.go`).
- **No retry**: retry without idempotency knowledge is unsafe — the response
is always relayed to the client for the client to decide.
## What's NOT changed
- `core/selector/` — Marker/FailFilter untouched
- `core/hop/` — Select() untouched
- `x/chain/router.go` — Router.Dial untouched
- `x/handler/http/proxy.go` — httpHandler direct path (future work)
## Verification
```bash
cd core && go build ./... && go vet ./...
cd x && go build ./... && go vet ./...
```
@@ -0,0 +1,492 @@
# Node-Level Liveness Probe
## Context
GOST 当前依赖**被动失败检测**:节点仅在真实连接失败时被标记(`FailFilter` 在 `failTimeout`
后让其回到候选池),没有主动健康检查、没有时延测量,因此无法做基于健康/时延的路由决策。
本计划实现 **node 级** 主动探测。选择 node 级而非 chain 级,原因:
- chain 级探测(`x/docs/plans/chain-liveness-probe.md`)用 `chain.Route(addr).Dial(addr)`,
而 `Route()` 会按每个 hop 的 selector(random/round-robin)**任选节点**,每次探测 tick 可能
走不同 node,结果(成功/时延)被盖到整条 chain 唯一的 `Marker` 上 → 假健康 + 假死亡,
**结果不准确**。
- node 级探测用**节点自身的 `Transport`** 钉死到该 node(`Dial(node.Addr)` → `Connect(probeAddr)`),
每个 node 独立记录健康与时延,`FailFilter` 在同 hop 内精准剔除死 node,
`LowestLatencyStrategy`/`LatencyFilter` 在同 hop 内选最优 node。
> 结论:`chain-liveness-probe.md` 应据此**改写**(probe 下沉到 node,chain 健康由 node 聚合
> 推导),不可照原样执行。本计划是 node 级版本的详细实现。
## Design Overview
```
YAML → ParseNode → xchain.StartNodeProbe → goroutine (per node)
|
ticker loop (interval)
|
tr.Dial(ctx, node.Addr) → Handshake → tr.Connect(ctx, conn, network, probeAddr)
|
success → node.Marker().Reset() + node.SetProbeResult(ok, latency)
failure → node.Marker().Mark() + node.SetProbeResult(fail)
|
FailFilter(hop) 自动跳过 marker 死亡的 node ← 零 selector 改动
lowestlatency / maxLatency 在同 hop 内按时延选 node
```
每个 `Node` 拥有自己的探测 goroutine,结果写入 `node.probeResult`(`atomic.Value`)与
`node.marker`(现有失败标记)。`Node` 实现 `chain.ProbeResultReader`,使泛型
`LowestLatencyStrategy[T]`/`LatencyFilter[T]` 对 `*chain.Node` 与 `chain.Chainer` 同时生效。
**准确性范围(Phase 1)**:探测用 node 自身 transport 直连 node,对**首跳 / 单跳节点**
(客户端可直接拨达的代理)准确有效。深跳节点(需经上游 hop 才能到达)的探测需 pin 住上游路径,
列为后续工作(见文末)。
## Config Schema
```yaml
chains:
- name: chain-1
hops:
- name: hop-1
selector:
strategy: lowestlatency # 同 hop 内选最低时延 node
maxLatency: 500ms # 超阈 node 被 LatencyFilter 过滤
nodes:
- name: proxy1
addr: 10.0.0.1:1080
connector: { type: socks5 }
dialer: { type: tcp }
probe: # node 级探测
type: tcp # "tcp" 或 "http"
addr: 8.8.8.8:53 # 探测目标(经此 node 穿透)
interval: 30s # 探测间隔(默认 30s)
timeout: 10s # 单次探测超时(默认 10s)
maxFails: 2 # 连续失败几次标记死亡(默认 1)
- name: proxy2
addr: 10.0.0.2:1080
probe:
type: http
addr: httpbin.org:80
httpPath: /get
expectedStatus: 200
interval: 60s
timeout: 15s
```
## Implementation Plan
### Step 1: `core/chain/probe.go`(NEW)
与 chain 计划同名的类型,供 node 与 chain 共用:
```go
package chain
import "time"
type ProbeType string
const (
ProbeTypeTCP ProbeType = "tcp"
ProbeTypeHTTP ProbeType = "http"
)
type ProbeConfig struct {
Type ProbeType
Addr string // 探测目标(经 node 穿透)
Interval time.Duration
Timeout time.Duration
MaxFails int
HTTPPath string
HTTPHost string
HTTPHeaders map[string]string
ExpectedStatus int
}
type ProbeResult struct {
Success bool
Latency time.Duration
Error string
Timestamp time.Time
}
// ProbeResultReader 由 Node 与 Chain 共同实现,供泛型时延策略/过滤器使用。
type ProbeResultReader interface {
ProbeResult() *ProbeResult
}
```
### Step 2: `core/chain/node.go`(MODIFY)
`Node` 增加探测状态字段与小访问器(与既有 `marker selector.Marker` 同风格;goroutine 逻辑放 `x/chain`,
core 只持有状态)。`ProbeResult()` 使 `Node` 实现 `ProbeResultReader`。
```go
import (
"context"
"sync/atomic"
)
type Node struct {
Name string
Addr string
marker selector.Marker
probeResult atomic.Value // NEW
probeCancel context.CancelFunc // NEW
options NodeOptions
}
// ProbeResult 实现 chain.ProbeResultReader。
func (node *Node) ProbeResult() *ProbeResult {
if v := node.probeResult.Load(); v != nil {
return v.(*ProbeResult)
}
return nil
}
func (node *Node) SetProbeResult(r *ProbeResult) { node.probeResult.Store(r) }
func (node *Node) SetProbeCancel(c context.CancelFunc) { node.probeCancel = c }
// Close 停止本 node 的探测 goroutine(被 hop/chain Close 级联调用)。
func (node *Node) Close() error {
if node.probeCancel != nil {
node.probeCancel()
}
return nil
}
```
### Step 3: `x/chain/node.go`(NEW)— node 探测 goroutine
实现逻辑放 `x`(core 不含实现)。`StartNodeProbe` 由 `ParseNode` 调用:
```go
package chain
import (
"context"
"time"
"github.com/go-gost/core/chain"
"github.com/go-gost/x/internal/probe"
"github.com/go-gost/core/logger"
)
func StartNodeProbe(node *chain.Node, cfg *chain.ProbeConfig, log logger.Logger) {
if cfg == nil || cfg.Addr == "" {
return
}
ctx, cancel := context.WithCancel(context.Background())
node.SetProbeCancel(cancel)
go runNodeProbe(ctx, node, cfg, log)
}
func runNodeProbe(ctx context.Context, node *chain.Node, cfg *chain.ProbeConfig, log logger.Logger) {
interval := cfg.Interval
if interval <= 0 {
interval = 30 * time.Second
}
ticker := time.NewTicker(interval)
defer ticker.Stop()
probeNode(node, cfg, log) // 首次立即探测
for {
select {
case <-ticker.C:
probeNode(node, cfg, log)
case <-ctx.Done():
return
}
}
}
func probeNode(node *chain.Node, cfg *chain.ProbeConfig, log logger.Logger) {
timeout := cfg.Timeout
if timeout <= 0 {
timeout = 10 * time.Second
}
ctx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
start := time.Now()
tr := node.Options().Transport
conn, err := tr.Dial(ctx, node.Addr) // 直连本 node
if err == nil {
if hc, err2 := tr.Handshake(ctx, conn); err2 == nil {
conn = hc
} else {
conn.Close()
err = err2
}
}
latency := time.Since(start)
result := &chain.ProbeResult{Timestamp: time.Now(), Latency: latency}
if err != nil {
result.Success = false
result.Error = err.Error()
node.Marker().Mark()
} else {
defer conn.Close()
if cfg.Type == chain.ProbeTypeHTTP {
if e := probe.NewHTTPProber(cfg).Probe(conn, cfg.Addr); e != nil {
result.Success = false
result.Error = e.Error()
node.Marker().Mark()
} else {
result.Success = true
node.Marker().Reset()
}
} else {
result.Success = true
node.Marker().Reset()
}
}
node.SetProbeResult(result)
log.Debugf("node probe %s: success=%v latency=%v", node.Name, result.Success, latency)
}
```
**要点**:探测走 `node.Options().Transport`(即 `ParseNode` 为该 node 构造的 transporter),
`Dial(node.Addr)` 接通本 node、`Connect` 穿透到 `probeAddr`,因此结果只反映这一个 node,
不会被 hop selector 漂走到其他 node。TCP 探测只验证隧道可达;HTTP 探测用 `HTTPProber` 校验状态码。
### Step 4: `x/internal/probe/probe.go`(NEW)— HTTPProber
复用 chain 计划的 HTTP 探测器(TCP 探测在 Step 3 内联处理,无需额外包):
```go
package probe
type HTTPProber struct {
Path string
Host string
Headers map[string]string
ExpectedStatus int
}
// Probe 在 conn 上发送 GET <path> 并校验状态码。
func (p *HTTPProber) Probe(conn net.Conn, addr string) error
```
实现:构造 `GET <path> HTTP/1.1\r\nHost: <host>\r\n...` 经 `conn` 发送,读取响应,校验
`status == ExpectedStatus`(默认 200),任何 I/O 错误或状态码不符返回 error。
### Step 5: `core/chain/probe.go` 的配置类型已在 Step 1 定义;`x/config/config.go`(MODIFY)
```go
// 新增共享类型
type ProbeConfig struct {
Type string `json:"type"`
Addr string `json:"addr,omitempty"`
Interval time.Duration `json:"interval"`
Timeout time.Duration `json:"timeout"`
MaxFails int `json:"maxFails"`
HTTPPath string `json:"httpPath,omitempty"`
HTTPHost string `json:"httpHost,omitempty"`
HTTPHeaders map[string]string `json:"httpHeaders,omitempty"`
ExpectedStatus int `json:"expectedStatus,omitempty"`
}
// NodeConfig 增加 probe
type NodeConfig struct {
// ... 现有字段不变 ...
Probe *ProbeConfig `yaml:",omitempty" json:"probe,omitempty"` // NEW
}
// SelectorConfig 增加 maxLatency
type SelectorConfig struct {
Strategy string `json:"strategy"`
MaxFails int `yaml:"maxFails" json:"maxFails"`
FailTimeout time.Duration `yaml:"failTimeout" json:"failTimeout"`
MaxLatency time.Duration `yaml:"maxLatency,omitempty" json:"maxLatency,omitempty"` // NEW
}
```
### Step 6: `x/config/parsing/node/parse.go`(MODIFY)
在 `chain.NewNode(...)` 返回后、函数返回前,若 `cfg.Probe != nil` 启动探测:
```go
node := chain.NewNode(cfg.Name, cfg.Addr, opts...)
if cfg.Probe != nil {
if pc := parseProbeConfig(cfg.Probe); pc != nil {
xchain.StartNodeProbe(node, pc, nodeLogger)
}
}
return node, nil
```
`parseProbeConfig` 同 chain 计划(addr 为空则返回 nil;interval/timeout 默认 30s/10s;
maxFails 默认 1;expectedStatus 默认 200;httpPath 默认 "/")。
> 因 `ParseNode` 同时被 `ParseHop`(初始内联节点)与 `hop.reload`(file/redis/http 热加载)
> 调用,上述挂接**同时覆盖初始与热加载**两处。
### Step 7: `x/selector/latency.go`(NEW)— 泛型时延策略/过滤器
与 chain 计划一致,泛型约束 `chain.ProbeResultReader`,对 `*chain.Node` 与 `chain.Chainer` 都生效:
```go
package selector
import (
"context"
"math"
"time"
"github.com/go-gost/core/chain"
)
// LatencyFilter 过滤掉 ProbeResult 失败或时延超过 maxLatency 的项;
// 无探测结果的项保守放行(不过滤未 instrumentation 的项)。
func LatencyFilter[T any](maxLatency time.Duration) selector.Filter[T] {
return selector.FilterFunc[T](func(ctx context.Context, vs ...T) []T {
var out []T
for _, v := range vs {
if reader, ok := any(v).(chain.ProbeResultReader); ok {
r := reader.ProbeResult()
if r != nil && (!r.Success || (maxLatency > 0 && r.Latency > maxLatency)) {
continue
}
}
out = append(out, v)
}
return out
})
}
// LowestLatencyStrategy 选 ProbeResult 时延最低的项;无结果者排最后。
func LowestLatencyStrategy[T any]() selector.Strategy[T] {
return selector.StrategyFunc[T](func(ctx context.Context, vs ...T) (v T) {
var best *chain.ProbeResult
for _, item := range vs {
reader, ok := any(item).(chain.ProbeResultReader)
if !ok {
if !isNil(v) { continue }
v = item
continue
}
r := reader.ProbeResult()
if r == nil || !r.Success {
continue
}
if best == nil || r.Latency < best.Latency {
best = r
v = item
}
}
return v
})
}
```
> 注:`selector.FilterFunc`/`selector.StrategyFunc` 若 core/selector 未提供,则在 `x/selector`
> 内用闭包实现 `selector.Filter[T]`/`selector.Strategy[T]` 接口即可(与现有
> `defaultSelector` 同构)。
### Step 8: `x/config/parsing/selector/parse.go`(MODIFY)
`ParseNodeSelector` 增加 `lowestlatency`/`lowest`/`ll` 策略,并在 `maxLatency > 0` 时追加
`LatencyFilter[*chain.Node]`(**node 级**):
```go
case "lowestlatency", "lowest", "ll":
strategy = xs.LowestLatencyStrategy[*chain.Node]()
filters := []selector.Filter[*chain.Node]{
xs.FailFilter[*chain.Node](cfg.MaxFails, cfg.FailTimeout),
xs.BackupFilter[*chain.Node](),
}
if cfg.MaxLatency > 0 {
filters = append(filters, xs.LatencyFilter[*chain.Node](cfg.MaxLatency))
}
return xs.NewSelector(strategy, filters...)
```
(`ParseChainSelector` 暂不动;chain 级 lowestlatency 留待 Phase 2 聚合推导。)
### Step 9: 生命周期级联(关键)
**`x/hop/hop.go`** — 扩展 `chainHop.Close()`,级联停止本 hop 所有 node 的探测:
```go
func (p *chainHop) Close() error {
p.cancelFunc()
for _, n := range p.Nodes() {
if n != nil {
n.Close() // 停止 node 探测 goroutine
}
}
// ... 现有 loader 关闭 ...
return nil
}
```
**`x/chain/chain.go`** — 新增 `Chain.Close()`(`io.Closer`),级联到各 hop:
```go
func (c *Chain) Close() error {
for _, h := range c.hops {
if closer, ok := h.(io.Closer); ok {
closer.Close()
}
}
return nil
}
```
registry `Unregister(chain)` 会对实现 `io.Closer` 的 chain 调 `Close()`(见
`x/registry/registry.go:87`),因此 **config reload 自动停止所有 node 探测 goroutine**,无泄漏。
## File Change Summary
| 文件 | 变更 |
|------|------|
| `core/chain/probe.go` | **NEW** — `ProbeType`/`ProbeConfig`/`ProbeResult`/`ProbeResultReader` |
| `core/chain/node.go` | 加 `probeResult`/`probeCancel` 字段 + `ProbeResult()`/`SetProbeResult()`/`SetProbeCancel()`/`Close()` |
| `x/chain/node.go` | **NEW** — `StartNodeProbe`/`runNodeProbe`/`probeNode` |
| `x/internal/probe/probe.go` | **NEW** — `HTTPProber` |
| `x/config/config.go` | 加 `ProbeConfig`;`NodeConfig` 加 `Probe`;`SelectorConfig` 加 `MaxLatency` |
| `x/config/parsing/node/parse.go` | 末尾 `node.SetProbe`→`StartNodeProbe`;加 `parseProbeConfig` |
| `x/selector/latency.go` | **NEW** — `LatencyFilter[T]`/`LowestLatencyStrategy[T]` |
| `x/config/parsing/selector/parse.go` | `ParseNodeSelector` 加 `lowestlatency` + `LatencyFilter` |
| `x/hop/hop.go` | `chainHop.Close()` 级联 `node.Close()` |
| `x/chain/chain.go` | 新增 `Chain.Close()` 级联 hop |
**总计:4 个新文件,6 个修改文件。不动 registry/loader 的清理机制。**
## 向后兼容
- 无 `probe` 字段 → node 不启探测 goroutine,`Close()` 对 nil `probeCancel` 为 no-op。
- `FailFilter` 已消费 `node.Marker()`,probe 仅更新该 marker → 现有 selector 行为不变。
- 现有 `round`/`random`/`fifo`/`hash` 策略不变;`lowestlatency` 为新增可选策略。
- 现有 YAML 全部有效。
## 已知限制 / 后续
- **深跳节点探测**:Phase 1 仅对首跳/单跳节点(客户端可直接拨达)准确。深跳节点需先穿透上游
hop 才能到达,其探测要 pin 住上游路径(构造仅含该路径的 route),列为 Phase 2。
- **热加载节点 churn**:`hop.reload` 追加新节点会启动新探测;被移除的旧节点探测在「整链 unregister」
时才经 `Chain.Close()` 级联停止。file/redis/http 增量热加载下的旧节点探测清理可后续细化
(reload 时对离场节点调 `Close()`)。
- **chain 级健康**:Phase 2 由 node `ProbeResult` 聚合推导(每 hop 至少一个健康 node 且取最优
路径时延),不在 Phase 1 跑独立 chain 级探测。
## Verification
1. **Build & vet**:`cd x && go build ./... && go vet ./...`(x/ 无单测,build+vet 为验证路径);
`cd core && go build ./...`。
2. **Unit — node TCP probe**:起本地 TCP listener;配 node TCP probe 指向它 →
验证 `node.ProbeResult().Success == true`;关 listener,下个 tick →
`Success == false` 且 `node.Marker().Count() > 0`;`FailFilter` 不再选该 node。
3. **Unit — node HTTP probe**:本地 HTTP server `/health` 返回 200/500 各验证一次。
4. **Unit — Close 停 goroutine**:`node.Close()` 后无更多 marker / probeResult 更新。
5. **Unit — LowestLatencyStrategy on nodes**:mock 不同 node 时延,验证选最快;
`LatencyFilter` 验证超阈被滤。
6. **Integration — hop failover**:hop 内一健康一死 node(`strategy: lowestlatency`),
验证死 node 被剔除、恢复后重新入选。
7. **手动 e2e**:`play/` 新增多 node hop 配置(`strategy: lowestlatency` + `maxLatency`),
`cd gost && go run ./cmd/gost/... -C play/xxx.yml` 观察节点分布与日志。
@@ -0,0 +1,124 @@
# Node matcher: HTTP request body matcher
## Context
GOST's node matcher ([x/routing/matcher.go](../../../x/routing/matcher.go)) routes requests to specific proxy nodes via a rule string (e.g. `Host("api.example.com") && Method("POST")`). It matches on `ClientIP`, `Host`, `Network`, `Proto`, `Method`, `Path`, `Query`, `Header` — but **not** the request body. A user wants to route on request body content (e.g. an OpenAI-style JSON body containing `"model":"gpt-4"`), which is impossible today.
The body must be available *before* node selection (the matcher runs inside `hop.Select`) and still forwarded afterward. HTTP-level select options (`Method`/`Path`/`Query`/`Header`) are populated in **one place only**: the forwarder/sniffer path at [sniffer_http.go:183-191](../../../x/internal/util/forwarder/sniffer_http.go#L183-L191) (transparent-proxy / TCP-forward-with-sniffing). The regular HTTP proxy handler doesn't populate them; the HTTP/2 sniffer doesn't either. This change is scoped to the HTTP/1.x sniffer path — consistent with current behavior.
Decisions confirmed with the user:
- Add a single **`BodyRegexp(regex)`** matcher keyword (one arg). Realistic use cases are regex over structured bodies (JSON). Substring/exact can be added later.
- Body reading is **opt-in per node** via a new `matcher.bodySize` field, sibling of `matcher.priority`. The forwarder reads up to the **max `bodySize` across all candidate nodes** in the hop before selection, restores the stream, and passes the prefix to the matcher. Nodes that don't set `bodySize` (the default) pay zero overhead.
## Approach
Seven small layers, each following an existing pattern in the touched file. The key invariant: the matcher tree knows nothing about body size — size is plain data on the node, aggregated by the hop, read by the forwarder.
### 1. Carry the body prefix — [core/routing/matcher.go](../../../core/routing/matcher.go)
Add `Body []byte` to `Request` (size-capped prefix read by the forwarder; nil/empty when no node opts in or the request is bodyless).
### 2. Plumb body through hop selection — [core/hop/hop.go](../../../core/hop/hop.go)
- Add `Body []byte` to `SelectOptions`.
- Add `BodySelectOption(body []byte) SelectOption` (mirrors `HeaderSelectOption`).
- Define a small optional interface for size aggregation:
```go
// BodySizer returns the max request body size (bytes) needed by node matchers
// in this hop, so callers holding the request body (e.g. the forwarder
// sniffer) can pre-read a sized prefix before selection. Returns 0 if no node
// needs body matching.
type BodySizer interface { BodySize() int }
```
### 3. Per-node size on node options — [core/chain/node.go](../../../core/chain/node.go)
- Add `MatcherBodySize int` to `NodeOptions` ([line 85](../../../core/chain/node.go#L85)).
- Add `MatcherBodySizeNodeOption(int) NodeOption`.
- Add a public cap constant `MaxMatcherBodySize = 1 << 20` (1MB), consistent with the recorder's body cap. The parser clamps to it.
### 4. Forward Body + expose hop size — [x/hop/hop.go](../../../x/hop/hop.go)
- At [line 177](../../../x/hop/hop.go#L177), add `Body: options.Body,` to the `routing.Request{}` literal in `chainHop.Select`.
- Implement `BodySize() int` on `*chainHop`: iterate `p.Nodes()` under RLock, return `max(node.Options().MatcherBodySize)`. This is the implementation of `hop.BodySizer` that the forwarder type-asserts for. (Plugin/wrapped hops don't implement it → no body matching through them; acceptable, documented.)
### 5. The matcher — [x/routing/matcher.go](../../../x/routing/matcher.go)
- Add `bodyRegexp(tree, args...)`: compile the regex once at parse time (modelled on `pathRegexp`); at request time return false if `req.Body` is empty, else `re.Match(req.Body)`.
- Register `"BodyRegexp": expectNParameters(bodyRegexp, 1)` in the `httpFuncs` map. Registering the keyword is the only config-side change needed to make it usable in rule strings (rules are free-form strings parsed via `routing.NewMatcher` at [node/parse.go:201](../../../x/config/parsing/node/parse.go#L201)).
- Add `TestMatcherBodyRegexp` in [x/routing/matcher_test.go](../../../x/routing/matcher_test.go): match, no-match, empty body → false, invalid regex fails at parse time.
### 6. Per-node config field — [x/config/config.go:425](../../../x/config/config.go#L425) + [x/config/parsing/node/parse.go:197](../../../x/config/parsing/node/parse.go#L197)
- Add `BodySize int` to `NodeMatcherConfig` (sibling of `Rule` and `Priority`):
```go
// BodySize is the max request body bytes made available to BodyRegexp
// matchers. 0 (default) disables body reading for this node. Capped at
// chain.MaxMatcherBodySize. Only takes effect under an HTTP sniffing handler.
BodySize int `yaml:",omitempty" json:"bodySize,omitempty"`
```
- In the existing `if cfg.Matcher != nil` block in `node/parse.go`, apply the parsed (and clamped) value: `chain.MatcherBodySizeNodeOption(clampMatcherBodySize(cfg.Matcher.BodySize))`, where `clampMatcherBodySize` is a tiny helper bounding to `[0, chain.MaxMatcherBodySize]`.
- Add a test in [x/config/parsing/node/parse_test.go](../../../x/config/parsing/node/parse_test.go) asserting the field is parsed and clamped.
### 7. Read + restore the body — [x/internal/util/forwarder/sniffer_http.go](../../../x/internal/util/forwarder/sniffer_http.go)
Inside `resolveHTTPNode` (the single function used by both the initial dial and the keep-alive re-dial path), before `ho.hop.Select(...)`:
```go
node = &chain.Node{}
if ho.hop != nil {
var bodyPrefix []byte
if bs, ok := ho.hop.(hop.BodySizer); ok {
if size := bs.BodySize(); size > 0 && req.Body != nil {
bodyPrefix, _ = io.ReadAll(io.LimitReader(req.Body, int64(size)))
req.Body = io.NopCloser(io.MultiReader(bytes.NewReader(bodyPrefix), req.Body))
}
}
node = ho.hop.Select(ctx,
hop.ClientIPSelectOption(clientIP),
hop.ProtocolSelectOption(sniffing.ProtoHTTP),
hop.HostSelectOption(host),
hop.MethodSelectOption(req.Method),
hop.PathSelectOption(req.URL.Path),
hop.QuerySelectOption(req.URL.Query()),
hop.HeaderSelectOption(req.Header),
hop.BodySelectOption(bodyPrefix),
)
}
```
`MultiReader` restores the stream so `httpRoundTrip`'s later `req.Write(cc)` still forwards the full body unchanged — Content-Length / chunked encoding stay valid because the bytes are reproduced in order. For bodyless requests (GET/HEAD), `req.Body` is `http.NoBody` → `ReadAll` returns empty immediately → negligible overhead.
No changes to: forwarder `sniffer.go` (no new `HandleOptions`), forward handler `metadata.go` / `sniffing.go` (no per-handler knob), HTTP/2 sniffer, or the regular HTTP proxy handler.
### Out of scope
- HTTP/2 sniffer path ([sniffer_h2.go](../../../x/internal/util/forwarder/sniffer_h2.go)) — doesn't populate `Method`/`Path`/etc. today.
- Regular HTTP proxy handler ([x/handler/http/](../../../x/handler/http/)) — doesn't populate HTTP-level select options today.
- Plugin/wrapped hops — don't implement `hop.BodySizer`, so they skip body reading (`BodyRegexp` returns false). Acceptable.
## Example config
```yaml
services:
- name: ...
handler:
type: tcp # forward with HTTP sniffing
metadata:
sniffing: true
listener: ...
forwarder:
nodes:
- name: gpt4-node
addr: ...
matcher:
rule: 'Method("POST") && Path("/v1/chat/completions") && BodyRegexp(`"model"\s*:\s*"gpt-4"`)'
bodySize: 65536 # opt-in; 0 (default) disables body reading
priority: 10
- name: default-node
addr: ...
```
## Verification
1. `cd x && go build ./... && go vet ./...` — compiles across the workspace.
2. `cd x && go test -race ./routing/... ./hop/... ./config/parsing/node/... ./internal/util/forwarder/...` — new tests pass; existing tests unaffected. Set `CGO_ENABLED=1` for `-race`.
3. Manual e2e: a `play/`-style forward+sniff config with two nodes whose `BodyRegexp` rules partition POST bodies by a JSON field, with `matcher.bodySize` set; send two `curl -d` POSTs with different bodies and confirm each routes to the correct node (verify via the debug log line `"node %s match request ..."` at [x/hop/hop.go:190](../../../x/hop/hop.go#L190), or per-node metrics counters). Confirm a large upload (>bodySize) still forwards correctly after the prefix read, and that a GET (no body) is unaffected.
@@ -0,0 +1,292 @@
# Plan: Add matcher + probe to chainGroup
## Problem
`chainGroup` currently only chains by name string with a selector — no matcher-based
filtering or health-check probing. `hopGroup` (already implemented in 4c2748b) has both.
This treats chains and hops asymmetrically: users who manage multiple upstream proxy
paths (chains) get no way to route by hostname or exclude dead chains.
## How it works (hopGroup pattern)
`x/hop/group.go`:
- `HopEntry` wraps a `hop.Hop` + optional `routing.Matcher` + optional `*chain.ProbeConfig`
+ independent `selector.Marker`
- `hopGroup.Select()` filters entries by matcher, runs the group selector over eligible
entries, delegates to the winning entry's underlying hop
- Entries with probes get a goroutine: selects a node from the hop, dials/handshakes/probes
`x/config/config.go` `ForwardHopGroupConfig`:
```go
type ForwardHopConfig struct {
Hop string `yaml:",omitempty" json:"hop,omitempty"`
Matcher *NodeMatcherConfig `yaml:",omitempty" json:"matcher,omitempty"`
Probe *ProbeConfig `yaml:",omitempty" json:"probe,omitempty"`
}
```
## Plan
### 1. Config types — `x/config/config.go`
Change `Chains` from `[]string` to `[]*ChainGroupEntry` — the new struct handles
both plain strings (old format) and structured entries (new format) via custom
`UnmarshalYAML` / `UnmarshalJSON`:
```go
type ChainGroupEntry struct {
Chain string `yaml:",omitempty" json:"chain,omitempty"`
Matcher *NodeMatcherConfig `yaml:",omitempty" json:"matcher,omitempty"`
Probe *ProbeConfig `yaml:",omitempty" json:"probe,omitempty"`
}
// UnmarshalYAML accepts either a plain string ("chain-a") or an object.
func (e *ChainGroupEntry) UnmarshalYAML(value *yaml.Node) error {
var s string
if value.Decode(&s) == nil {
e.Chain = s
return nil
}
type alias ChainGroupEntry // avoid recursion
return value.Decode((*alias)(e))
}
// UnmarshalJSON accepts either a plain string or an object.
func (e *ChainGroupEntry) UnmarshalJSON(data []byte) error {
var s string
if json.Unmarshal(data, &s) == nil {
e.Chain = s
return nil
}
type alias ChainGroupEntry
return json.Unmarshal(data, (*alias)(e))
}
type ChainGroupConfig struct {
Chains []*ChainGroupEntry `yaml:",omitempty" json:"chains,omitempty"`
Selector *SelectorConfig `yaml:",omitempty" json:"selector,omitempty"`
}
```
Backward compat — both of these work:
```yaml
# Old: plain strings → each decoded as ChainGroupEntry{Chain: "..."}
chainGroup:
chains:
- chain-a
- chain-b
selector:
strategy: round
```
```yaml
# New: structured entries
chainGroup:
chains:
- chain: chain-a
matcher:
rule: Host(`example.com`)
probe:
type: tcp
interval: 10s
- chain: chain-b
selector:
strategy: round
```
### 2. Chain entry type — `x/chain/group.go` (new file)
Mirrors `x/hop/group.go`:
```go
type ChainEntry struct {
chain chain.Chainer
matcher routing.Matcher
probe *chain.ProbeConfig
marker selector.Marker
}
func NewChainEntry(c chain.Chainer, m routing.Matcher, probe *chain.ProbeConfig) *ChainEntry
// Route delegates to the underlying chain — implements chain.Chainer so the
// entry itself can be passed to the selector (and selected via FailFilter).
func (e *ChainEntry) Route(ctx context.Context, network, address string, opts ...chain.RouteOption) chain.Route {
return e.chain.Route(ctx, network, address, opts...)
}
func (e *ChainEntry) Marker() selector.Marker // implements selector.Markable
```
### 3. Enhanced chainGroup — `x/chain/chain.go`
Current:
```go
type chainGroup struct {
chains []chain.Chainer
selector selector.Selector[chain.Chainer]
}
```
New:
```go
type chainGroup struct {
entries []*ChainEntry
selector selector.Selector[chain.Chainer]
logger logger.Logger
cancelFunc context.CancelFunc
}
```
`chains` is dropped — `NewChainGroup` wraps each bare chain as a `ChainEntry`
(no matcher, no probe). There is only one data path: `entries`.
New/updated methods:
**`NewChainGroup(chains ...chain.Chainer)`** — wraps each chain as
`&ChainEntry{chain: c, marker: selector.NewFailMarker()}`. Backward compat:
existing callers get a bare entry with no matcher/probe.
**`WithGroupEntries(entries ...*ChainEntry)`** — replaces `entries` with
fully populated entries (matchers + probes). Callers that need the old
behavior pass entries via the constructor; callers with matchers/probes use
this setter. The constructor still works standalone when only bare chains
are needed.
**`selectChain()` —** single path, iterates `entries`:
```go
func (p *chainGroup) selectChain(ctx context.Context, network, address string, opts ...chain.RouteOption) chain.Chainer {
var options chain.RouteOptions
for _, opt := range opts { opt(&options) }
req := routing.Request{
Network: network,
Host: options.Host,
}
// Stage 1: matcher filter — build eligible pool.
var eligible []chain.Chainer
for _, e := range p.entries {
if e.matcher != nil && !e.matcher.Match(&req) {
continue
}
eligible = append(eligible, e) // *ChainEntry as chain.Chainer
}
if len(eligible) == 0 {
return nil
}
if len(eligible) == 1 {
return eligible[0]
}
// Stage 2: selector (FailFilter + strategy).
return p.selector.Select(ctx, eligible...)
}
```
**`Route()` —** unchanged pattern, delegates to `selectChain` then
`chain.Route()`.
**Probe methods** — identical pattern to hopGroup:
- `runEntryProbe(ctx, e)` — ticker loop
- `probeEntry(e, cfg)` — cmd probe or TCP/HTTP through chain's Route
For chain probes, TCP/HTTP types use `e.chain.Route()` + `route.Dial()` to reach
the probe target (ProbeConfig.Addr). CMD probe runs a shell command.
**`Close()`** — stops probe goroutines, closes entries.
### 4. Config parsing — `x/config/parsing/service/parse.go`
`group.Chains` is now `[]*ChainGroupEntry` — no separate `Entries` field. The
parser always iterates `group.Chains`, and each entry may or may not have
matcher+probe (plain strings decode as entries with only `Chain` set).
Update `chainGroup()`:
```go
func chainGroup(name string, group *config.ChainGroupConfig, log logger.Logger) chain.Chainer {
// Collect chains: from 'name' param and from group.Chains (or group is nil).
var entries []*xchain.ChainEntry
if c := registry.ChainRegistry().Get(name); c != nil {
entries = append(entries, xchain.NewChainEntry(c, nil, nil))
}
if group != nil {
for _, ge := range group.Chains {
c := registry.ChainRegistry().Get(ge.Chain)
if c == nil {
log.Warnf("chain %q not found in chainGroup, skipping", ge.Chain)
continue
}
var m routing.Matcher
if ge.Matcher != nil && ge.Matcher.Rule != "" {
m, err = xrouting.NewMatcher(ge.Matcher.Rule)
if err != nil {
log.Warnf("chain %q: bad matcher rule: %v, skipping", ge.Chain, err)
continue
}
}
entries = append(entries, xchain.NewChainEntry(c, m, node_parser.ParseProbeConfig(ge.Probe)))
}
}
if len(entries) == 0 {
return nil
}
sel := selector_parser.ParseChainSelector(group.Selector)
if sel == nil {
sel = selector_parser.DefaultChainSelector()
}
return xchain.NewChainGroup().
WithGroupEntries(entries...).
WithSelector(sel).
WithGroupLogger(log.WithFields(...))
}
```
Update callers to pass logger (both use `log` from ParseService scope):
- line 207: `chainGroup(cfg.Listener.Chain, cfg.Listener.ChainGroup, log)`
- line 383: `chainGroup(cfg.Handler.Chain, cfg.Handler.ChainGroup, log)`
## Files changed
| File | Change |
|------|--------|
| `x/config/config.go` | Change `Chains` from `[]string` to `[]*ChainGroupEntry`; add `ChainGroupEntry` struct with custom `UnmarshalYAML`/`UnmarshalJSON` |
| `x/chain/group.go` | NEW — `ChainEntry` type, `NewChainEntry`, matcher+probe logic |
| `x/chain/chain.go` | Add entries/log/cancel to `chainGroup`, `WithGroupEntries`, `WithGroupLogger`, updated `Route`/`next`, `Close`, probe methods |
| `x/config/parsing/service/parse.go` | Update `chainGroup()` to parse `ChainGroupEntry` with matcher+probe, pass logger |
## What the matcher can use at chain level
`chain.RouteOptions` only carries `Host`. So chain-level matchers can match on:
- `Host()` — the destination hostname
- `ClientIP()` — available from context via ClientIP in ctx
If full HTTP-level matching (Method, Path, Header, Body) is needed at the chain
level in the future, `RouteOptions` in `core/chain/chain.go` can be extended.
For now the available fields handle the common "route host X through chain Y" case.
## Probe behavior for chain entries
- **TCP/HTTP probe**: uses `e.chain.Route(ctx, "tcp", cfg.Addr)` to get a route,
then `route.Dial()` to reach the probe target. Tests the chain end-to-end.
- **CMD probe**: runs a shell command (same as hopGroup).
- The probe marks the entry's independent `selector.Marker` on failure, so the
chain is excluded from selection by `FailFilter` until it recovers.
## Known gap: `Close()` caller
`chainGroup.Close()` cancels the probe context and stops goroutines, but there
is no obvious caller in the current codebase. `hopGroup.Close()` works because
it cascades through `Chain.Close() → hop.Close() → hopGroup.Close()` — chain
groups don't sit inside a `Chain`, so this path doesn't exist for them.
This is a pre-existing gap (hopGroup has the same problem when not inside a
`Chain`). Goroutines survive until the probe context is cancelled. Not blocking
this plan — worth a follow-up to have the router close its chainer on shutdown.
## Verification
- `cd x && go build ./... && go vet ./...`
- Spot-check: old config with `chains: ["a", "b"]` still parses (each string → `ChainGroupEntry{Chain: "a"}`)
- Spot-check: new config with `chains: [{chain: "a", matcher: {rule: Host(`foo`)}, probe: {type: tcp}}]` parses correctly
@@ -0,0 +1,503 @@
# Chain-Level Liveness Probe
## Context
GOST currently relies on **reactive failure detection**: nodes are marked failed only when real connection attempts fail. Recovery is passive — `FailFilter` lets nodes back in after `failTimeout` (default 10s). There is no proactive health checking and no latency tracking for informed routing decisions.
The user wants Kubernetes-style liveness probes, but **at the chain level only**. Node-level probes are not applicable because in a multi-hop chain (`client → hop1 → hop2 → target`), intermediate nodes are not directly reachable — they can only be accessed through previous hops. A chain-level probe tests end-to-end connectivity through the entire chain and correctly accounts for this topology.
## Design Overview
The probe is configured on a chain. It periodically dials a configurable TCP address or HTTP endpoint **through the full chain** (using `chain.Route().Dial()`), exercising the exact same code path as real traffic. Results update the chain's existing `selector.Marker` — making `FailFilter` automatically skip unhealthy chains with **zero selector changes**. Probe latency is also tracked for optional latency-aware chain selection strategies.
```
YAML config → ParseChain() → Chain.SetProbe() → background goroutine
|
ticker loop
|
chain.Route(probeAddr).Dial()
|
success → Marker.Reset()
failure → Marker.Mark()
```
## Config Schema
```yaml
chains:
- name: chain-1
hops:
- name: hop-1
nodes:
- name: proxy1
addr: 10.0.0.1:1080
connector:
type: socks5
dialer:
type: tcp
probe:
type: tcp # "tcp" or "http"
addr: 8.8.8.8:53 # probe target (required for chain-level)
interval: 30s # probe interval (default 30s)
timeout: 10s # probe timeout (default 10s)
maxFails: 2 # consecutive failures before marking dead (default 1)
- name: chain-2
hops:
- name: hop-2
...
probe:
type: http
addr: httpbin.org:80
httpPath: /get
httpHost: httpbin.org
expectedStatus: 200
interval: 60s
timeout: 15s
```
For chains in a `chainGroup`, the selector can also use probe latency:
```yaml
services:
- name: service-0
addr: ":8080"
handler:
type: http
chain: chain-1
chainGroup:
chains:
- chain-2
- chain-3
selector:
strategy: lowestlatency # prefers lowest probe latency
maxLatency: 500ms # filters chains exceeding this threshold
```
## Implementation Plan
### Step 1: Core types — `core/chain/probe.go` (NEW FILE)
```go
// ProbeType enumerates supported probe protocols.
type ProbeType string
const (
ProbeTypeTCP ProbeType = "tcp"
ProbeTypeHTTP ProbeType = "http"
)
// ProbeConfig holds configuration for a chain's liveness probe.
type ProbeConfig struct {
Type ProbeType
Addr string // probe target address (required)
Interval time.Duration // probe interval
Timeout time.Duration // per-probe timeout
MaxFails int // consecutive failures before marking dead
HTTPPath string // HTTP request path (default "/")
HTTPHost string // HTTP Host header
HTTPHeaders map[string]string
ExpectedStatus int // expected HTTP status (default 200)
}
// ProbeResult holds the outcome of the most recent probe check.
type ProbeResult struct {
Success bool
Latency time.Duration
Error string
Timestamp time.Time
}
// ProbeResultReader is implemented by types that expose probe results.
// Chain implements this for selector integration.
type ProbeResultReader interface {
ProbeResult() *ProbeResult
}
```
### Step 2: Extend `x/chain/chain.go`
**Add to `Chain` struct:**
```go
type Chain struct {
name string
hops []hop.Hop
marker selector.Marker
metadata metadata.Metadata
logger logger.Logger
// NEW: probe support
probeCfg *chain.ProbeConfig
probeResult atomic.Value // stores *chain.ProbeResult
probeCancel context.CancelFunc
}
```
**Add `ProbeResult()` method** (implements `chain.ProbeResultReader`):
```go
func (c *Chain) ProbeResult() *chain.ProbeResult {
if v := c.probeResult.Load(); v != nil {
return v.(*chain.ProbeResult)
}
return nil
}
```
**Add `SetProbe(cfg)` method** — starts the background probe goroutine. Called by `ParseChain()` after all hops are added:
```go
func (c *Chain) SetProbe(cfg *chain.ProbeConfig) error {
c.probeCfg = cfg
ctx, cancel := context.WithCancel(context.Background())
c.probeCancel = cancel
go c.runProbe(ctx)
return nil
}
```
**Add `Close() error`** — stops the probe goroutine. Makes `Chain` implement `io.Closer` so the registry's `Unregister()` automatically stops probes during config reload:
```go
func (c *Chain) Close() error {
if c.probeCancel != nil {
c.probeCancel()
}
return nil
}
```
**Add `runProbe()`** — the ticker loop. Follows the existing `periodReload` pattern from `x/bypass/bypass.go`:
```go
func (c *Chain) runProbe(ctx context.Context) {
interval := c.probeCfg.Interval
ticker := time.NewTicker(interval)
defer ticker.Stop()
// Run first probe immediately
c.probe(ctx)
for {
select {
case <-ticker.C:
c.probe(ctx)
case <-ctx.Done():
return
}
}
}
func (c *Chain) probe(ctx context.Context) {
cfg := c.probeCfg
timeout := cfg.Timeout
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
start := time.Now()
route := c.Route(ctx, "tcp", cfg.Addr)
conn, err := route.Dial(ctx, "tcp", cfg.Addr)
latency := time.Since(start)
result := &chain.ProbeResult{Timestamp: time.Now(), Latency: latency}
if err != nil {
result.Success = false
result.Error = err.Error()
c.marker.Mark()
c.logger.Warnf("probe failed: %v (latency %v)", err, latency)
} else {
conn.Close()
result.Success = true
c.marker.Reset()
c.logger.Debugf("probe success: latency %v", latency)
}
c.probeResult.Store(result)
}
```
**Key insight**: The probe uses `c.Route()` + `route.Dial()` — the same path as real traffic through the chain. It updates `c.marker` which is the same marker used by `FailFilter` on chainGroup selectors. Zero selector changes needed for basic liveness.
### Step 3: Probe implementation — `x/internal/probe/` (NEW PACKAGE)
This package provides the HTTP prober variant. The TCP probe is handled inline in the chain since it's trivial (just `route.Dial()` + close). The HTTP prober needs more logic.
**`x/internal/probe/probe.go`**:
```go
// HTTPProber performs an HTTP GET request over an established chain connection.
type HTTPProber struct {
Path string
Host string
Headers map[string]string
ExpectedStatus int
}
// Probe sends an HTTP GET over conn and validates the response.
func (p *HTTPProber) Probe(conn net.Conn, addr string) error
```
The HTTP prober:
1. Builds URL from `addr` + `Path`
2. Sends `GET <path> HTTP/1.1\r\nHost: <host>\r\n...` over `conn`
3. Reads response, checks status code matches expected
4. Returns error if status mismatch or any I/O error
### Step 4: Config struct — `x/config/config.go`
**Add `ProbeConfig`:**
```go
type ProbeConfig struct {
Type string `json:"type"` // "tcp" or "http"
Addr string `json:"addr,omitempty"` // probe target address
Interval time.Duration `json:"interval"`
Timeout time.Duration `json:"timeout"`
MaxFails int `json:"maxFails"` // defaults to 1 (via FailFilter)
HTTPPath string `json:"httpPath,omitempty"`
HTTPHost string `json:"httpHost,omitempty"`
HTTPHeaders map[string]string `json:"httpHeaders,omitempty"`
ExpectedStatus int `json:"expectedStatus,omitempty"`
}
```
**Add to `ChainConfig`:**
```go
type ChainConfig struct {
Name string `json:"name"`
Hops []*HopConfig `json:"hops"`
Probe *ProbeConfig `json:"probe,omitempty"` // NEW
Metadata map[string]any `json:"metadata,omitempty"`
}
```
### Step 5: Config parsing — `x/config/parsing/chain/parse.go`
In `ParseChain()`, after creating the chain and adding all hops, attach the probe:
```go
func ParseChain(cfg *config.ChainConfig, log logger.Logger) (chain.Chainer, error) {
// ... existing code: create chain, add hops ...
if cfg.Probe != nil {
probeCfg := parseProbeConfig(cfg.Probe, c.Name)
if probeCfg != nil {
c.SetProbe(probeCfg)
}
}
return c, nil
}
func parseProbeConfig(cfg *config.ProbeConfig, chainName string) *chain.ProbeConfig {
if cfg.Addr == "" {
return nil // address is required for chain-level probe
}
interval := cfg.Interval
if interval <= 0 {
interval = 30 * time.Second
}
timeout := cfg.Timeout
if timeout <= 0 {
timeout = 10 * time.Second
}
maxFails := cfg.MaxFails
if maxFails <= 0 {
maxFails = 1
}
expectedStatus := cfg.ExpectedStatus
if expectedStatus <= 0 {
expectedStatus = 200
}
path := cfg.HTTPPath
if path == "" {
path = "/"
}
return &chain.ProbeConfig{
Type: chain.ProbeType(cfg.Type),
Addr: cfg.Addr,
Interval: interval,
Timeout: timeout,
MaxFails: maxFails,
HTTPPath: path,
HTTPHost: cfg.HTTPHost,
HTTPHeaders: cfg.HTTPHeaders,
ExpectedStatus: expectedStatus,
}
}
```
### Step 6: HTTP probe support in `Chain.probe()`
Extend the `Chain.probe()` method to handle HTTP type:
```go
func (c *Chain) probe(ctx context.Context) {
cfg := c.probeCfg
timeout := cfg.Timeout
ctx, cancel := context.WithTimeout(ctx, timeout)
defer cancel()
start := time.Now()
route := c.Route(ctx, "tcp", cfg.Addr)
conn, err := route.Dial(ctx, "tcp", cfg.Addr)
latency := time.Since(start)
result := &chain.ProbeResult{Timestamp: time.Now(), Latency: latency}
if err != nil {
result.Success = false
result.Error = err.Error()
c.marker.Mark()
} else {
defer conn.Close()
if cfg.Type == chain.ProbeTypeHTTP {
err = probe.NewHTTPProber(cfg).Probe(conn, cfg.Addr)
if err != nil {
result.Success = false
result.Error = err.Error()
c.marker.Mark()
} else {
result.Success = true
c.marker.Reset()
}
} else {
result.Success = true
c.marker.Reset()
}
}
c.probeResult.Store(result)
}
```
### Step 7: New selector strategy — `x/selector/latency.go` (NEW FILE)
Generic latency-aware selection — works with any `T` implementing `ProbeResultReader`.
**`LatencyFilter`:**
```go
func LatencyFilter[T any](maxLatency time.Duration) selector.Filter[T]
```
Filters out items whose `ProbeResult().Latency > maxLatency` or `ProbeResult().Success == false`. Items without probe results pass through (conservative: don't filter uninstrumented items).
**`LowestLatencyStrategy`:**
```go
func LowestLatencyStrategy[T any]() selector.Strategy[T]
```
Selects the item with the lowest `ProbeResult().Latency`. Items without probe results are sorted last. Items with failed probes are filtered out. Uses `math.MinInt64` for the initial comparison.
To access probe results, both filter and strategy use:
```go
if reader, ok := any(v).(chain.ProbeResultReader); ok {
result := reader.ProbeResult()
...
}
```
`Chain` already implements `ProbeResultReader` via `ProbeResult()` added in Step 2.
### Step 8: Selector config — `x/config/parsing/selector/parse.go`
**Add to `SelectorConfig`:**
```go
type SelectorConfig struct {
Strategy string `json:"strategy"`
MaxFails int `json:"maxFails"`
FailTimeout time.Duration `json:"failTimeout"`
MaxLatency time.Duration `json:"maxLatency,omitempty"` // NEW
}
```
**Extend `ParseChainSelector()`:**
```go
func ParseChainSelector(cfg *config.SelectorConfig) selector.Selector[chain.Chainer] {
if cfg == nil {
return nil
}
var strategy selector.Strategy[chain.Chainer]
switch cfg.Strategy {
case "round", "rr":
strategy = xs.RoundRobinStrategy[chain.Chainer]()
case "random", "rand":
strategy = xs.RandomStrategy[chain.Chainer]()
case "fifo", "ha":
strategy = xs.FIFOStrategy[chain.Chainer]()
case "hash":
strategy = xs.HashStrategy[chain.Chainer]()
case "lowestlatency", "lowest", "ll": // NEW
strategy = xs.LowestLatencyStrategy[chain.Chainer]() // NEW
default:
strategy = xs.RoundRobinStrategy[chain.Chainer]()
}
filters := []selector.Filter[chain.Chainer]{
xs.FailFilter[chain.Chainer](cfg.MaxFails, cfg.FailTimeout),
xs.BackupFilter[chain.Chainer](),
}
if cfg.MaxLatency > 0 { // NEW
filters = append(filters, xs.LatencyFilter[chain.Chainer](cfg.MaxLatency)) // NEW
}
return xs.NewSelector(strategy, filters...)
}
```
### Step 9: Registry integration (automatic cleanup)
The chain registry (`x/registry/registry.go`) already calls `Close()` on any registered value that implements `io.Closer` when `Unregister()` is called. During config reload, `unregisterAll(ChainRegistry)` is called, which calls `Unregister()` on each chain — automatically stopping all probe goroutines.
No changes needed to `x/registry/` or `x/config/loader/`.
## File Change Summary
| File | Change |
|------|--------|
| `core/chain/probe.go` | **NEW** — `ProbeType`, `ProbeConfig`, `ProbeResult`, `ProbeResultReader` |
| `x/chain/chain.go` | Add probe fields + `SetProbe()` + `ProbeResult()` + `Close()` + `runProbe()` + `probe()` methods |
| `x/internal/probe/probe.go` | **NEW** — `HTTPProber` struct with `Probe()` method |
| `x/config/config.go` | Add `ProbeConfig` struct; add `Probe` field to `ChainConfig`; add `MaxLatency` to `SelectorConfig` |
| `x/config/parsing/chain/parse.go` | Parse probe config + call `chain.SetProbe()` |
| `x/selector/latency.go` | **NEW** — `LatencyFilter[T]`, `LowestLatencyStrategy[T]` |
| `x/config/parsing/selector/parse.go` | Add `"lowestlatency"` strategy; add `LatencyFilter` when `MaxLatency > 0` |
**Total: 3 new files, 5 modified files. No changes to registry, loader, hop, node, or service parsing.**
## What stays the same (backward compatibility)
- **Zero probe config changes**: No `probe` field → chain has no probe goroutine, no `Close()` behavior change (existing `Close()` is a no-op since `probeCancel` is nil, which is the same as before).
- **Existing `FailFilter`**: Already works with chain's `Marker`. Probes update the same marker — no change needed.
- **Existing selectors**: RoundRobin, Random, FIFO, Hash all work unchanged.
- **All existing YAML configs**: Remain valid.
- **Node/hop parse**: No changes.
## Verification
1. **Build & vet**: `cd x && go build ./... && go vet ./...`
2. **Unit test — TCP probe on chain**:
- Create a simple chain (direct dial, no proxy hops)
- Start a local TCP listener on a random port
- Create a chain with TCP probe pointing at the listener
- Verify `ProbeResult().Success == true` after first tick
- Close listener, wait for next tick
- Verify `ProbeResult().Success == false` and `Marker().Count() > 0`
3. **Unit test — HTTP probe on chain**:
- Start a local HTTP server with `/health` returning 200
- Configure chain with HTTP probe
- Verify success
- Change server to return 500, verify failure
4. **Unit test — Chain Close stops probe**:
- Create chain with probe
- Call `Close()`
- Verify probe goroutine exits (no more marker updates after close)
5. **Unit test — LowestLatencyStrategy**:
- Create mock chains with different probe latencies
- Verify `LowestLatencyStrategy` selects the fastest
6. **Unit test — LatencyFilter**:
- Create mock chains with various latencies
- Verify chains exceeding `maxLatency` are filtered out
7. **Integration — chainGroup failover with probe**:
- Create two chains in a chainGroup, one with a valid probe target and one with an unreachable target
- Verify the dead chain gets marked and `FailFilter` stops selecting it
- Verify recovery when the target becomes reachable again
+231
View File
@@ -0,0 +1,231 @@
# hopGroup: Conditional Hop Group Selection for Forwarder
## Problem
### Conflict: probe `Reset()` vs `failCodes` `Mark()`
`chain.Node` has a single `selector.Marker` ([core/chain/node.go](core/chain/node.go#L218)). Two mechanisms share it:
| Mechanism | Calls | Lives in |
|-----------|-------|----------|
| `failCodes` | `node.Marker().Mark()` on matching HTTP status | HTTP response path (`sniffer_http.go:577`) |
| Probe | `node.Marker().Reset()` on successful health check | Per-node probe goroutine |
The conflict:
```
1. HTTP response 429 → failCodes → node.Marker().Mark() // count=1, node excluded
2. Probe goroutine → TCP health check succeeds → node.Marker().Reset() // count=0, node back!
3. Next request hits same node → 429 again
```
`Reset()` has no way to know whether the mark came from failCodes or a genuine network outage. It clears both. The probe restores a node that failCodes intentionally excluded — rendering failCodes ineffective.
### Current workaround: probe-based pseudo-grouping
Users work around this by putting nodes with different probe configs into one hop, using probe to create implicit groups. This makes the hop configuration bloated and the grouping intent invisible.
### Chain level has `chainGroup`, hop level has nothing
`x/chain/chain.go` already has `chainGroup` — groups multiple `chain.Chainer` with a selector, used at the router level. The forwarder (`handler.Forwarder`) can only accept a single `hop.Hop`. No equivalent grouping exists at the hop level.
## Solution
**`hopGroup` separates the two markers into different levels:**
| Level | Marker owner | Controlled by | Semantics |
|-------|-------------|---------------|-----------|
| **hopGroup entry** | `hopEntry.marker` | Probe (network health) | Is the hop reachable at all? |
| **Node (within hop)** | `node.Marker()` | failCodes (HTTP response status) | Should this node get this type of traffic? |
Probe runs at the hop entry level and marks the entry — not individual nodes. A marked hop entry is excluded from the group selector's pipeline (FailFilter). failCodes continues to work at the node level within each hop — no Reset() interference.
**Secondary benefit:** explicit grouping replaces the implicit probe-based pseudo-grouping. Each named hop in the group has its own selector, nodes, and reload source — independently manageable via API.
## Design
**`hopGroup` implements `hop.Hop`** — zero handler changes, drop-in compatible with the existing `Forwarder.Forward(hop.Hop)`.
### Per-hop config
| Field | Purpose |
|-------|---------|
| `hop` | Named hop reference from registry |
| `matcher` | `routing.Matcher` DSL — conditional routing. `Bypass(...)`, `Host(...)`, `PathPrefix(...)`, etc. Nil = unconditional (catch-all) |
| `probe` | `ProbeConfig` — hop-level health check for dynamic failover |
### hopEntry struct: hop + markable wrapper
```go
type hopEntry struct {
hop hop.Hop
matcher routing.Matcher // nil = unconditional
probe *probeConfig // internal parsed probe config
marker selector.Marker // for selector.FailFilter exclusion (hop-level, NOT node-level)
}
```
`hopEntry` implements **both** `hop.Hop` (via `Select()` delegation) and `selector.Markable` (via `Marker()`) so it can be passed to `selector.Selector[hop.Hop]` — the selector's `FailFilter` checks `selector.Markable` dynamically. Same mechanism as node-level fail marking in `chainHop`, but at a different level.
### Select() logic
```
1. Matcher filter:
for each entry in group:
if entry.matcher == nil → eligible
else → build routing.Request from SelectOptions, entry.matcher.Match(request)
2. Selector:
cast eligible entries to []hop.Hop (type-erased slice, one allocation per Select)
apply FailFilter (exclude marked entries via selector.Markable) then strategy
→ pick one hopEntry (type-assert back from hop.Hop)
3. Delegate:
return entry.hop.Select(ctx, opts...)
```
The `[]*hopEntry` → `[]hop.Hop` conversion allocates a new slice per call. Acceptable — hop selection is per-connection, not per-packet. An alternative (store entries as `[]hop.Hop`, type-assert in the probe goroutine) would push the allocation to the non-hot-path, but makes matcher access uglier. Either way works; pick the cleaner one.
Nil matcher + nil probe = always eligible (catch-all). Hops with matchers that don't match simply never get selected (correct — traffic ruled out by condition).
### Probe: hop-level health check
Runs at the **hop group entry** level, not at the node level. Periodically samples a node from the hop and marks the **hop entry** (not the node) on failure.
- Probe goroutine runs on `hopGroup` lifetime (started on creation, stopped on Close)
- Each probe tick: `hop.Select()` → get a node → TCP/HTTP/ICMP probe to node.Addr
- Default probe interval: 30s (same as node-level default from `parseProbeConfig`)
- On failure: **`entry.marker.Mark()`** (marks the hop entry, not the node)
- On success: **`entry.marker.Reset()`** (clears the hop entry mark, not the node)
- If a hop has no probe config: entry is never marked (always eligible)
**Key design property:** probe writes to `entry.marker`, failCodes writes to `node.Marker()`. They never touch the same marker. Probe Reset() only clears the hop entry mark — it cannot undo a failCodes mark on a node.
**Sampling caveat:** `hop.Select()` picks one node per tick (round-robin/random). A single bad node in an otherwise-healthy hop could mark the entry if `maxFails=1` and the probe hits that node repeatedly. Mitigation: set `maxFails > 1` on the hop entry metadata, or ensure hop-level probes use `maxFails=N` where N ≥ the number of nodes in the hop.
This integrates with the selector's FailFilter — a marked hop entry is excluded from selection until a probe succeeds.
### Selector
Reuses `selector.Selector[hop.Hop]` with full pipeline: FailFilter → BackupFilter → strategy. Since `hopEntry` implements `Markable`, the selector can exclude unhealthy hop entries.
## Files
### NEW: `x/hop/group.go`
- `hopEntry` struct — implements `hop.Hop` (via `Select()` delegation) and `selector.Markable` (via `Marker()`)
- `hopGroup` struct: `{entries []*hopEntry, selector selector.Selector[hop.Hop], logger logger.Logger, cancelFunc context.CancelFunc}`
- `HopGroupOption` functional options type — mirrors the existing options pattern in `x/hop/hop.go`:
```go
type HopGroupOption func(*hopGroupOptions)
type hopGroupOptions struct {
entries []*hopEntry
selector selector.Selector[hop.Hop]
logger logger.Logger
}
```
- `NewHopGroup(opts ...HopGroupOption) *hopGroup`
- `Select(ctx, opts...) *chain.Node` — matcher filter → selector pipeline → delegate
- `Nodes() []*chain.Node` — aggregate all nodes across all entries for introspection/API. Implements `hop.NodeList`. Returns a snapshot — hot-reload entries may change the underlying set between calls (consistent with `chainHop.Nodes()`).
- `Close() error` — stop probe goroutines
- Probe goroutine: reuses `x/hop/hop.go` probe infrastructure (same as node probes)
### MODIFY: `x/config/config.go`
```go
type ForwarderConfig struct {
// ... existing fields unchanged ...
HopGroup *ForwardHopGroupConfig `yaml:"hopGroup,omitempty" json:"hopGroup,omitempty"`
}
type ForwardHopGroupConfig struct {
Hops []*ForwardHopConfig `yaml:",omitempty" json:"hops,omitempty"`
Selector *SelectorConfig `yaml:",omitempty" json:"selector,omitempty"`
}
type ForwardHopConfig struct {
Hop string `yaml:",omitempty" json:"hop,omitempty"`
Matcher *NodeMatcherConfig `yaml:",omitempty" json:"matcher,omitempty"`
Probe *ProbeConfig `yaml:",omitempty" json:"probe,omitempty"`
}
```
### MODIFY: `x/config/parsing/service/parse.go`
Extend `parseForwarder()` with precedence:
```
HopGroup > Hop > Name > Nodes
```
- If `cfg.HopGroup != nil`: build `hopGroup` from listed hops. `cfg.Hop`, `cfg.Name`, and `cfg.Nodes` are ignored.
- Else if `cfg.Hop != ""` or `cfg.Name != ""`: look up named hop from `registry.HopRegistry()` (existing behavior).
- Else fall back to inline `cfg.Nodes` (existing behavior).
When building from `HopGroup`:
- Resolve `ForwardHopConfig.Hop` from `registry.HopRegistry()`
- Parse `Matcher` via existing matcher parsing (same as node parsing)
- Parse `Probe` via existing probe parsing (same as node probe parsing). Default probe interval: 30s (matches node-level default from `parseProbeConfig`).
- Reuse existing selector parsing for group selector
- `Name` set on `ForwarderConfig` is ignored when `HopGroup` wins (no shadow hop ref)
### NO CHANGES
- `core/hop/hop.go` — hopGroup implements existing `Hop` interface
- `core/handler/handler.go` — `Forwarder.Forward(hop.Hop)` unchanged
- `x/handler/forward/local/` and `remote/` — call `h.getHop().Select()`, no knowledge of group
- `x/registry/` — hopGroup is constructed at config parse time
- Probe infrastructure — reuse existing `x/hop/hop.go` probe patterns (TCP/HTTP/ICMP dial checks)
## Config Example
```yaml
hops:
- name: h-web
nodes:
- addr: 10.0.1.1:8080
- addr: 10.0.1.2:8080
- name: h-api
nodes:
- addr: 10.0.2.1:8080
failCodes: [429, 502, 503] # node-level: HTTP status failover
- addr: 10.0.2.2:8080
failCodes: [429, 502, 503]
- name: h-internal
nodes:
- addr: 192.168.1.1:3128
services:
- name: multi-hop
addr: :8000
handler:
type: tcp
forwarder:
hopGroup:
hops:
- hop: h-api
matcher: PathPrefix("/api/") && Host("*.example.com")
probe:
type: http # hop-level: network health (won't touch node markers)
- hop: h-internal
matcher: Bypass("10.0.0.0/8", "*.internal")
- hop: h-web # no matcher, no probe = catch-all
selector: rr
```
| Traffic | Match | Result |
|---------|-------|--------|
| `GET /api/users` on `app.example.com` | matcher hit | `h-api` (if probe healthy) |
| `GET /` on `www.example.com` | no matcher hit → catch-all | `h-web` |
| Raw TCP to `10.0.5.3:5432` | Bypass match | `h-internal` |
| All `h-api` nodes return 429 → failCodes marks nodes → excluded by FailFilter | → fallback to catch-all | `h-web` |
| `h-api` server network-down → probe fails → hop entry marked | → FailFilter excludes | fallback to catch-all |
## Verification
```bash
cd x && go build ./... && go vet ./...
```
Config validation: create a test config YAML in `play/` with hopGroup + probe and verify parses without error.
@@ -0,0 +1,167 @@
# Migrate DNS cache to the generic `cache.Cache` module
## Context
GOST has **two** unrelated cache implementations for `[]byte`-style data:
- A **bespoke DNS cache** at `x/internal/util/resolver/cache.go` — a `*resolver.Cache`
type that stores `*dns.Msg` values, decrements answer TTLs on read, and supports
serve-stale. It is used only by `x/resolver/resolver.go` (`localResolver.cache`)
and `x/handler/dns/handler.go` (`dnsHandler.cache`).
- A **generic cache module** (`core/cache.Cache` interface: `Get/Set/Delete` over
`[]byte`/`*Entry`) with `x/cache` in-memory (`NewMemoryCache`) and Redis
(`NewRedisCache`) backends, wired through `registry.CacheRegistry()` via
`x/config/parsing/cache.ParseCache`. It already supports TTL, LRU eviction,
background cleanup, and named-config backends.
The task: delete the bespoke DNS cache and back the DNS resolver with the
shared generic cache. This removes a duplicate, less-capable implementation and
lets DNS caching inherit the generic module's eviction/cleanup/Redis backends.
The migration is **behavior-preserving**: DNS caching stays in-process and
serve-stale still works, because the generic `Entry.Expiration` model maps cleanly
onto the resolver's `Load() → (msg, remainingTTL)` contract (an expired-but-present
entry is returned with `Expired()==true`, exactly the serve-stale trigger the
resolver/handler already key off).
## Key insight — how to preserve DNS TTL semantics on `[]byte`
`dns.Msg` serializes cleanly via `miekg/dns` (`Pack() ([]byte,error)`,
`(*Msg).Unpack([]byte) error`, present in v1.1.61). The original code, on
**Store**, rewrote every answer record's `Header().Ttl` to the cache TTL, and on
**Load** decremented each by elapsed time — so after Load every answer TTL equals
*remaining* cache life. We reproduce that on read by simply **setting each answer
TTL to `max(1, int(remaining.Seconds()))`** where `remaining = entry.TTL()`. This
matches the old behavior exactly and avoids needing storedAt. (For an expired
entry, `remaining <= 0` → answer TTL clamped to 1, same as before.)
## Plan
### 1. Replace bespoke cache with adapter helpers (delete the `Cache` type)
In `x/internal/util/resolver/` replace `cache.go` with `key.go` keeping
`NewCacheKey`/`CacheKey`, and add the three adapter helpers (in a new small file
`cache_adapter.go`, or append to `key.go`). The helpers take the generic
`cache.Cache` from `github.com/go-gost/core/cache`.
```go
// CacheKey is a DNS-specific cache key string.
type CacheKey = string
func NewCacheKey(q *dns.Question) CacheKey { /* unchanged */ }
const defaultTTL = 60 * time.Second
// LoadMsg returns the cached DNS message and its remaining TTL (<=0 => stale/expired).
// Miss (ErrNotFound) returns (nil, 0).
func LoadMsg(ctx context.Context, c cache.Cache, key CacheKey) (*dns.Msg, time.Duration) {
ent, err := c.Get(ctx, string(key))
if err != nil { // includes ErrNotFound
return nil, 0
}
mr := new(dns.Msg)
if err := mr.Unpack(ent.Data); err != nil {
return nil, 0
}
remaining := ent.TTL()
for i := range mr.Answer {
if d := uint32(remaining.Seconds()); d > 0 {
mr.Answer[i].Header().Ttl = d
} else {
mr.Answer[i].Header().Ttl = 1
}
}
return mr, remaining
}
// StoreMsg serializes mr into the cache. ttl<0 => skip (do not cache);
// ttl==0 => use min answer TTL (or defaultTTL); ttl>0 overrides all answer TTLs.
func StoreMsg(ctx context.Context, c cache.Cache, key CacheKey, mr *dns.Msg, ttl time.Duration) {
if key == "" || mr == nil || ttl < 0 {
return
}
if ttl == 0 {
ttl = defaultTTL
for _, a := range mr.Answer {
if v := time.Duration(a.Header().Ttl) * time.Second; v < ttl {
ttl = v
}
}
}
b, err := mr.Pack()
if err != nil {
return
}
c.Set(ctx, string(key), b, cache.WithTTL(ttl))
}
// RefreshMsgTTL re-stores mr under key, extending its life by ttl (replaces old RefreshTTL).
func RefreshMsgTTL(ctx context.Context, c cache.Cache, key CacheKey, mr *dns.Msg, ttl time.Duration) {
StoreMsg(ctx, c, key, mr, ttl)
}
```
Delete the old `Cache` struct and `NewCache/WithLogger/WithMaxSize/Load/Store/
RefreshTTL/cleanupLocked` methods from this package.
### 2. Update `x/resolver/resolver.go`
- Field: `cache *resolver_util.Cache` → `cache cache.Cache` (import
`github.com/go-gost/core/cache` and `xcache "github.com/go-gost/x/cache"`).
- `NewResolver`: replace `resolver_util.NewCache().WithLogger(options.logger)`
with `xcache.NewMemoryCache(xcache.Options{DefaultTTL: defaultTTL,
CleanupInterval: time.Minute, Logger: options.logger})`.
- `lookupCache` (resolver.go:246) and `resolveIPs` (resolver.go:294,306):
`r.cache.Load(...)` → `resolver_util.LoadMsg(r.cache, ...)`;
`r.cache.Store(...)` → `resolver_util.StoreMsg(r.cache, ...)`. The returned
`(mr, ttl)` semantics are unchanged (ttl = remaining; `ttl <= 0` is the
serve-stale/refresh trigger).
### 3. Update `x/handler/dns/handler.go`
- Field (handler.go:45): `cache *resolver_util.Cache` → `cache cache.Cache`.
- Construction (handler.go:74): `resolver_util.NewCache().WithLogger(log)` →
`xcache.NewMemoryCache(xcache.Options{DefaultTTL: h.md.ttl, CleanupInterval:
time.Minute, Logger: log})` (store path passes an explicit TTL, so DefaultTTL is
only a fallback).
- handler.go:320 `h.cache.Load(...)` → `resolver_util.LoadMsg(...)`.
- handler.go:343 `h.cache.RefreshTTL(...)` → `resolver_util.RefreshMsgTTL(ctx,
h.cache, resolver_util.NewCacheKey(&mq.Question[0]), mr, h.md.ttl)`.
- handler.go:386 `h.cache.Store(...)` → `resolver_util.StoreMsg(...)`.
### 4. Update tests
- `x/resolver/resolver_test.go:81` `resolver_util.NewCache().WithLogger(...)` →
`xcache.NewMemoryCache(xcache.Options{Logger: xlogger.Nop()})` (import
`xcache`, drop `resolver_util.NewCache`). Test `TestResolve_AsyncCacheHitWithRefresh`
at :308 uses `r.cache.Store(ctx, key, mr, -1*time.Second)` → `resolver_util.StoreMsg(r.cache, key, mr, -1*time.Second)` (negative TTL skips, same as before, so the entry won't be cached and async refresh fires — assertion unchanged).
- `x/handler/dns/handler_test.go`: `h.cache = resolver_util.NewCache().WithLogger(...)`
(lines 905, 1095) → `h.cache = xcache.NewMemoryCache(xcache.Options{Logger: nopLog()})`;
`h.cache.Store(ctx, key, respMsg, <ttl>)` (lines 786, 934, 1035) →
`resolver_util.StoreMsg(h.cache, key, respMsg, <ttl>)`. The `TestRequest_CacheHit`
assertions on `ttl.Seconds() > 0` and stale behavior are preserved by the
`LoadMsg` normalization.
### 5. Verify
```bash
cd x && go build ./... && go vet ./...
# Tests live in the gost module's e2e; resolver/handler unit tests:
cd x && go test ./resolver/... ./handler/dns/... -race
```
(Per `x/CLAUDE.md`, unit tests are the exception; `go build ./... && go vet ./...`
is the primary gate for this module. Run the resolver/dns package tests only if
they exist locally — they do.)
## Trade-offs / skipped
- **No `CacheRegistry` wiring for DNS** (resolver does not yet pull a named cache
by name from config). The migration swaps only the backing store; making DNS use
a user-configured named Redis cache is a separate enhancement. Add when a config
knob is wanted.
- **No Redis-by-default for DNS** — keeps current in-memory behavior; just now
backed by the shared, eviction-capable `x/cache` memory backend.
- Helper functions instead of a wrapper type: the generic `cache.Cache` interface
can't carry DNS-typed methods, so three small package-level adapters keep call
sites nearly identical and avoid inventing a new type with one impl.
@@ -0,0 +1,361 @@
# TCP 嗅探新增 Redis 与 PostgreSQL 协议支持(两阶段)
## Context
GOST 当前的 TCP 嗅探框架(`x/internal/util/sniffing/` 与 `x/internal/util/forwarder/`)已支持 HTTP/1.x、HTTP/2、TLS(SNI 提取)、WebSocket 以及 SSH(仅检测,未处理)共 5 类协议。用户希望扩展应用层明文协议嗅探,使转发 handler 能够在"端口复用"场景下对**明文直连**的数据库流量做协议识别 + 首包元数据提取,进而用于路由决策和流量录制。
经调研行业做法(nginx `ssl_preread`/njs、HAProxy `payload` 字节匹配、Zeek/Suricata DPD、nDPI)与具体协议特征字节,确认 **Redis** 与 **PostgreSQL** 是最值得优先支持的两个协议:
- 二者均为**客户端先发包**(符合当前嗅探架构的 preread 模型),而 MySQL/SMTP 是服务端先发包,当前架构无法检测。
- 特征字节极稳定,不与现有 HTTP(`0x47/0x50`)、TLS(`0x16`)、SSH(`SSH-2`) 冲突。
- Zeek/nDPI/Suricata 均将二者列为核心检测协议,属于标准实践。
- 元数据对路由决策有实际价值:Redis 可按命令名路由、PostgreSQL 可按 `database` 路由。
**分两阶段交付**:第一阶段先完成 Redis(风险低、价值已论证、仅改 Peek(1));第二阶段再做 PostgreSQL(需要 Peek(8),阻塞风险更高,独立验证)。两阶段独立 build + 集成测试,互不影响。
> 注意:`core/` 是框架基础接口层,按 CLAUDE.md 规定**不修改 core/**。所有改动在 `x/` 模块内部,复用现有 `sniffing`/`forwarder` 包。
---
## 目标行为
### 第一阶段(Redis)
1. 在嗅探路径中新增 `ProtoRedis` 常量,识别 RESP 首字节。
2. 解析 RESP 首命令,提取 `command`(如 `PING`/`AUTH`/`SET`/`GET`/`HELLO`)、`key`(首参数后的第一个 bulk 字符串)。
3. 将协议名写入 `ro.Proto`,元数据写入新增的 `ro.Redis` 子记录(仿 `ro.TLS` / `ro.DNS`)。
4. 通过 `hop.Select(ctx, hop.ProtocolSelectOption(ProtoRedis), ...)` 参与路由,可用 `protocol: redis` 把 Redis 流量导到专用 hop。
5. 透传:把已 read 的首包字节 replay 给上游,然后 `xnet.Pipe` 双向转发。
### 第二阶段(PostgreSQL)
1. 在嗅探路径中新增 `ProtoPostgres` 常量,识别 StartupMessage。
2. 解析 StartupMessage,提取 `database`、`user`、`application_name`。
3. 写入 `ro.Postgres` 子记录。
4. 通过 `protocol: postgres` 参与路由。
5. 透传同 Redis。
---
## 架构与改动点
### 0. 分层 Peek 重构(`x/internal/util/sniffing/sniff.go`)—— 两阶段共享
当前 `Sniff()` 单次 `Peek(dissector.RecordHeaderLen)`(5 字节)。盲目改成 `Peek(8)` 有**阻塞死锁风险**:客户端只发 <8 字节时(`bufio.Reader.Peek` 会等满 8 字节或 `sniffingTimeout`),与 [auto handler 的 SOCKS 死锁先例](x/handler/auto/handler.go#L102-L107) 同类。
**采用分层 Peek**:先 Peek(1) 检测可仅凭首字节判定的协议,再按需 Peek 更多。
```go
func Sniff(ctx context.Context, r *bufio.Reader) (proto string, err error) {
// Tier 1: Peek 1 byte — 所有可仅凭首字节判定的协议在此判定。
// Peek(1) 不会阻塞(除非连接真的一个字节都无)。
b, err := r.Peek(1)
if err != nil {
return
}
switch b[0] {
case '*', '+', '-', ':', '$':
// Redis RESP type byte(第一阶段启用)
return ProtoRedis, nil
case 0x00:
// PostgreSQL StartupMessage 首字节(第二阶段启用)——需 Peek(8) 校验 version
hdr, err := r.Peek(8)
if err != nil {
return
}
if pgVersion := binary.BigEndian.Uint32(hdr[4:8]); pgVersion == 196608 || pgVersion == 196610 {
return ProtoPostgres, nil
}
return
}
// Tier 2: Peek 5 bytes for TLS/HTTP/SSH(保持现有行为)
// 第一阶段:case 0x00 在此进入 Tier 2(非 PG 的 0x00 session 罕见,与现行为无异)
hdr, err := r.Peek(dissector.RecordHeaderLen)
if err != nil {
return
}
// TLS check → HTTP check (isHTTP) → SSH check(不变)
return
}
```
**阶段划分**:
- **第一阶段**:Tier-1 仅启用 `case '*','+','-',':','$': return ProtoRedis`(Peek(1) 即返回,无 Peek(8),零阻塞风险)。Tier-2 不变。
- **第二阶段**:在 Tier-1 增加 `case 0x00:` PG 分支(仅当 `byte0==0x00` 才 Peek(8),PG StartupMessage 始终 ≥8 字节,不会阻塞)。Tier-2 不变。
**为何不在第一阶段预留 PG 分支**:保持第一阶段最小改动面。PG 分支是单个 `case`,第二阶段加一行即可,不破坏已稳定的 Tier-1 结构。
---
### 第一阶段:Redis
#### 1. 协议常量与检测(`sniff.go` 中 Tier-1 Redis 分支)
```go
const (
ProtoHTTP = "http"
ProtoTLS = "tls"
ProtoSSH = "ssh"
ProtoRedis = "redis" // 第一阶段新增
)
```
Tier-1 `switch b[0]` 增加 RESP 标记分支(见第 0 节)。Redis 仅需首字节,`Peek(1)` 即返回。
#### 2. Redis 解析 + Sniffer 方法(新建 `x/internal/util/sniffing/sniffer_redis.go`)
**ParseRedisMetadata 签名**:`ParseRedisMetadata(r io.Reader, ro *xrecorder.HandlerRecorderObject) error`(导出,供 forwarder 包复用)。
调用方式必须精确镜像 `HandleTLS`:
```go
buf := new(bytes.Buffer)
if err := ParseRedisMetadata(io.TeeReader(conn, buf), ro); err != nil {
return err
}
```
`io.TeeReader(conn, buf)` 消费恰好一个 RESP 消息并同时写入 `buf`,之后 `buf.WriteTo(cc)` 将首消息 replay 给上游。**这是 replay 正确性的唯一事实来源**:如果改用 `*bufio.Reader` 传给 ParseRedisMetadata,`buf` 不会被填充,replay 的数据为空/截断,上游服务端读到残缺流量。
解析行为:
- 读首个 RESP 消息(以 `*` 开头的行为 RESP 数组)。
- 格式:`*<n>\r\n` → 读 `n` 个 bulk string `$<len>\r\n<bytes>\r\n`。
- 第 1 个 bulk string = command(`strings.ToUpper` 存入 `ro.Redis.Command`,RESP 命令大小写不敏感);第 2 个(若存在)= key(**保持原大小写**存入 `ro.Redis.Key`,Redis key 大小写敏感),后续参数丢弃(仅路由/录制不需要)。
- **关键约束**:解析器必须消费恰好一个完整的 RESP 消息,不得多读一个字节。RESP 数组的 `*<n>\r\n` 前缀提供了精确的消息边界(`n` 个 bulk string),确定性可控。
- 解析失败(非 RESP、截断)时返回 error,上层回退 raw forwarding(不丢连接)。
**Pipelined/多命令流量**:后续命令(消息 2..N)仍留在 `conn`(`br` 缓冲区 → 底层 `net.Conn`),`xnet.Pipe` 会将其完整转发给上游。因此 `AUTH + SELECT + GET` 三连发只解析 AUTH 用于路由,其余透明透传。**不需要**消费多消息或特殊处理 pipeline。
`(h *Sniffer) HandleRedis(ctx, network, conn, opts...)` 方法,与现有 `HandleTLS(ctx, network, conn, opts...)` 同形(`network` 参数沿用 sniffing 包约定)。实现:
1. `io.TeeReader(conn, buf)` 读取首包,`ParseRedisMetadata` 写入 `ro.Redis`。
2. 通过 `ho.dial`(handler 注入,同 HTTP/TLS)建立上游 `cc`。
3. `buf.WriteTo(cc)` 将缓存首包 replay 给 `cc`(仿 `HandleTLS` line 87-89)。
4. `xnet.Pipe(ctx, conn, cc)` 双向透传(含消息 2..N)。
5. 解析失败返回 error,调用方回退。
#### 3. forwarder 变体(新建 `x/internal/util/forwarder/sniffer_redis.go`)
与 `forwarder/sniffer_tls.go` 同构:实现 `HandleRedis`,内部复用 `sniffing.ParseRedisMetadata`(单一事实来源,forwarder 包只做拨号+路由包装)。
**路由集成**:forwarder 路径必须通过 `hop.Select` 传递 protocol 维度才能让 `protocol: redis` 选择器生效。参照 `resolveTLSNode`([sniffer_tls.go:129](x/internal/util/forwarder/sniffer_tls.go#L129)),新建 `resolveRedisNode` 辅助函数:
- `resolveRedisNode(ctx, ro, ho)` → `ho.hop.Select(ctx, hop.ProtocolSelectOption(sniffing.ProtoRedis), ...)`
> sniffing 包路径(relay、http、sni handler)不需要 `resolveRedisNode`——它们的 dial 函数由 handler 注入,不做 hop 选择。
#### 4. recorder 子结构(`x/recorder/recorder.go`)—— 第一阶段仅 Redis
```go
type RedisRecorderObject struct {
Command string `json:"command"`
Key string `json:"key"`
}
```
`HandlerRecorderObject` 新增字段(第一阶段仅 Redis;Postgres 字段第二阶段加):
```go
Redis *RedisRecorderObject `json:"redis,omitempty"`
```
#### 5. handler 分派接入(5 处 dispatch,仅 Redis case)
5 处分派点分两类——**注入式 dial**(3 处,`sniffing.Sniffer`)和 **hop 选择式 dial**(2 处,`forwarder.Sniffer`)——各需不同的 HandleRedis。
**(A)注入式 dial**(3 处,`sniffing.Sniffer.HandleRedis`,签名含 `network`,无 hop):
在 `switch proto` 中新增 `case sniffing.ProtoRedis`,调用 `sniffer.HandleRedis(ctx, "tcp", conn, sniffing.WithService(...), sniffing.WithDial(...), sniffing.WithBypass(...), sniffing.WithRecorderObject(ro), sniffing.WithLog(log))`。
- `x/handler/sni/sniffing.go:99` — `handleSniffedProtocol` 辅助方法(已有 HTTP/TLS case)
- `x/handler/http/connect.go:131` — HTTP CONNECT 隧道内 `sniffAndHandle()`,内联 switch
- `x/handler/relay/connect.go:234` — relay 隧道内 `handleConnect()`,内联 switch(最易遗漏)
> 这三处的 HandleRedis 不需要 `WithHop`——dial 闭包由 handler 注入(sni 通过 Router.Dial,http/relay 直接复用已建立的 cc)。
**(B)hop 选择式 dial**(2 处,`forwarder.Sniffer.HandleRedis`,签名无 `network`,有 `resolveRedisNode`):
在 `handleSniffedProtocol` 的 `switch proto` 中新增 `case sniffing.ProtoRedis`,调用 `sniffer.HandleRedis(ctx, conn, forwarder.WithService(...), forwarder.WithHop(h.options.Hop), forwarder.WithBypass(...), forwarder.WithRecorderObject(ro), forwarder.WithLog(log))`。
- `x/handler/forward/local/sniffing.go:77` — `handleSniffedProtocol`(已有 HTTP/TLS case)
- `x/handler/forward/remote/sniffing.go:77` — `handleSniffedProtocol`(已有 HTTP/TLS case)
> 这两处的 HandleRedis 不接收 `network` 参数——现有 forwarder.HandleTLS 也无 `network`,保持一致。`resolveRedisNode` 内部使用 `"tcp"` 和 `sniffing.ProtoRedis`。
#### 6. 第一阶段验证
- `Sniff()` 对构造的 Redis(`*1\r\n$4\r\nPING\r\n`)首包返回 `ProtoRedis`;对 TLS/HTTP/SSH/非法首包返回正确结果(Redis 分支不影响现有协议)。
- `ParseRedisMetadata` 对真实首包提取正确 command/key,且底层 `bufio.Reader` 首包可完整 replay(下游能读到完整握手)。
- **Pipelined traffic**:`AUTH\r\nSELECT 0\r\nSET key val\r\n` ——仅首命令 `AUTH` 被解析用于路由/录制,后续 SELECT + SET 经 `xnet.Pipe` 透传到上游,`redis-cli` 端验证三条命令全部执行成功。
- `ro.Proto` 在所有 5 个 dispatch site 正确设置为 `"redis"`(验证方式:recorder 输出或 debug log 中各 site 的 `ro.Proto` 字段)。
- 对随机/非法首包返回 error,不 panic。
- 集成测试(`x/tests/e2e`,仿 `sniffing_test.go`):起 `forward`/`tcp` 服务开 `?sniffing=true`,后端跑真实 Redis(Docker),用 `redis-cli` 连接断言 SET/GET 通、`ro.Proto=="redis"`、含 command 元数据、配置 `protocol: redis` 选择器能导到指定后端。
- 构建:`cd x && go build ./... && go vet ./...`
---
### 第二阶段:PostgreSQL
> 前置条件:第一阶段已合入且验证通过。
#### 7. 协议常量与检测(`sniff.go` 中 Tier-1 PG 分支)
```go
const (
ProtoPostgres = "postgres" // 第二阶段新增
)
```
Tier-1 `switch b[0]` 增加 `case 0x00:` 分支(见第 0 节):仅当 `byte0==0x00` 才 `r.Peek(8)` 校验 version(`196608`/`196610`),避免 Peek(8) 阻塞。
#### 8. PG 解析 + Sniffer 方法(新建 `x/internal/util/sniffing/sniffer_postgres.go`)
`ParsePostgresMetadata(r io.Reader, ro *xrecorder.HandlerRecorderObject) error`:
- 与 Redis 阶段相同 pattern:`HandlePostgres` 内 `buf := new(bytes.Buffer)` + `ParsePostgresMetadata(io.TeeReader(conn, buf), ro)` → `buf.WriteTo(cc)` replay。
- 读 4 字节 length + 4 字节 version(确认 version = 196608 或 196610)。
- 读后续 KV 对(null-terminated 字符串),遇到 `user\0`、`database\0`、`application_name\0` 取其后 value 存入 `ro.Postgres.User` / `ro.Postgres.Database` / `ro.Postgres.ApplicationName`。
- 以末尾单个 `\0` 结束。
- 用 `io.TeeReader` + `buf.WriteTo(cc)` 缓存首消息并 replay。
- 解析失败返回 error 回退。
`(h *Sniffer) HandlePostgres(ctx, network, conn, opts...)` 方法,与 `HandleRedis` 同形。
#### 9. forwarder 变体(新建 `x/internal/util/forwarder/sniffer_postgres.go`)
实现 `HandlePostgres`,复用 `sniffing.ParsePostgresMetadata`。新建 `resolvePostgresNode`:
- `resolvePostgresNode(ctx, ro, ho)` → `ho.hop.Select(ctx, hop.ProtocolSelectOption(sniffing.ProtoPostgres), ...)`
#### 10. recorder 子结构(`x/recorder/recorder.go`)—— 第二阶段加 Postgres
```go
type PostgresRecorderObject struct {
User string `json:"user"`
Database string `json:"database"`
ApplicationName string `json:"applicationName,omitempty"`
}
```
`HandlerRecorderObject` 新增字段:`Postgres *PostgresRecorderObject \`json:"postgres,omitempty"\``
#### 11. handler 分派接入(5 处 dispatch,仅 Postgres case)
同第一阶段第 5 节 5 个文件,新增 `case sniffing.ProtoPostgres` 调用 `sniffer.HandlePostgres`。
#### 12. 第二阶段验证
- `Sniff()` 对构造的 PG(`00 00 00 24 00 03 00 02 ...`)首包返回 `ProtoPostgres`;PG 分支不影响 Redis/TLS/HTTP/SSH 检测。
- `ParsePostgresMetadata` 提取正确 database/user/application_name,首包可完整 replay。
- 集成测试:后端跑真实 PostgreSQL(Docker),用 `psql` 连接断言 SELECT 通、`ro.Proto=="postgres"` 含 database 元数据、`protocol: postgres` 选择器导到指定后端。
- 构建 + vet。
---
## 跨 repo 联动(仅标注,不在本计划执行范围内)
以下两个独立 repo 的改动**不在本计划各阶段的执行范围内**,仅作为联动项标注,由后续单独处理:
- `gost-plugins/recorder/recorder.go:125-134` 的 type 推断 `switch` 需加对应 `o.Redis != nil` / `o.Postgres != nil` case(否则 recorder 链路 type 字段缺失)。
- `inspector/pkg/model/record.go:81-107` 的 `HandlerRecorderObject` 需加 `Redis` / `Postgres` 字段(否则 inspector 展示缺失)。
> 注意:`x/` 模块内 `HandlerRecorderObject` 加了 Redis/Postgres 字段后,跨 repo 的 recorder/inspector 若未同步,会导致 JSON 解析/type 推断忽略这些字段。联动改动虽不阻塞 `x/` 构建,但应在对应阶段发布前补齐。
---
## 冲突分析(为什么 Tier 1 首字节检测不会误判)
| 协议 | 首字节 | 与 Redis/PG 检测的关系 |
|------|--------|------------------------|
| **Redis** (RESP) | `*`(0x2A), `+`(0x2B), `-`(0x2D), `:`(0x3A), `$`(0x24) | Tier 1 命中即返回 |
| **PG** (StartupMessage) | `0x00` | Tier 1 进入 Peek(8) 二次校验(仅第二阶段) |
| TLS | `0x16` | 不冲突:`0x16` ≠ RESP byte,≠ `0x00` |
| HTTP/1.x | `G/P/U/D/O/H/C/T` | 不冲突:均非 RESP 标记或 `0x00` |
| HTTP/2 | `PRI *` | byte0=`P`(0x50),不冲突 |
| SSH | `SSH-2` | byte0=`S`(0x53),不冲突 |
**关键安全论证**(第一阶段 Redis):
- RESP 的 5 种类型字节 (`*`/`+`/`-`/`:`/`$`) 均不与 TLS(`0x16`)、PG(`0x00`)、SSH(`0x53`) 重叠。
- HTTP 方法首字母在 `0x41-0x54` 范围内,与 RESP 的 `0x24-0x3A` 不重叠。
- 唯一歧义点:`P`(0x50) 同时是 HTTP `POST`/`PATCH`/`PRI*` 和 Redis inline 命令 `PING` 首字节。但 `P` 不在 RESP type-byte 集合,走 Tier-2 后 `isHTTP()` 不匹配 `PING`,最终 `("", nil)` 走 raw forwarding。**正确行为。**
> 误判回退:非协议流量首字节恰好是 RESP marker → `Sniff()` 返回 `ProtoRedis` → `HandleRedis` 解析失败 → error → 调用方回退 raw forwarding。不丢连接(与 TLS 解析失败一致)。
---
## 已验证设计约束
本节记录代码审查中确认的关键事实,避免实施时重复验证。
### ProtocolSelectOption 无需修改 core/
`core/hop/hop.go:68-72` 已定义 `ProtocolSelectOption(protocol string) SelectOption`,设置 `SelectOptions.Protocol`(普通 string 字段,无枚举限制)。`resolveRedisNode` / `resolvePostgresNode` 直接传入 `sniffing.ProtoRedis` / `sniffing.ProtoPostgres` 即可,无需改动 `core/`。
### ro.Proto 赋值位置
所有 5 处 dispatch site 在调用 `Sniff()` 后**立即设置** `ro.Proto = proto`(已存在),HandleRedis / HandlePostgres 自身不设置 `ro.Proto`(与 HandleTLS / HandleHTTP 一致——TLS/HTTP sniffer 也不设置 `ro.Proto`)。见:
- `relay/connect.go:208`: `ro.Proto = proto`
- `http/connect.go:114`: `ro.Proto = proto`
- Forward local/remote + SNI handler: 在 `Sniff()` 后同样位置赋值
### 两个 Sniffer 类型的差异
| 特性 | `sniffing.Sniffer` | `forwarder.Sniffer` |
|------|-------------------|---------------------|
| 使用的 dispatch site | sni, http/connect.go, relay/connect.go | forward/local, forward/remote |
| HandleRedis 签名 | `HandleRedis(ctx, network, conn, opts...)` | `HandleRedis(ctx, conn, opts...)`(无 `network`) |
| 协议路由 | 不参与(`ho.dial` 由 handler 注入) | 通过 `resolveRedisNode` → `ho.hop.Select(...ProtocolSelectOption...)` |
**无需抽象统一**:现有 TLS/HTTP 已维持这个两包分裂模式(`sniffing.HandleTLS` vs `forwarder.HandleTLS`),Redis 按同样模式新增两个方法即可。
### relay/connect.go Sniff() 到达性
`relay/connect.go:207` 在写入 relay 响应 header(`resp.WriteTo(conn)`)后对原始 client conn 调用 `sniffing.Sniff()`,preread 字节通过 `br := bufio.NewReader(conn)` 缓存,再经 `xnet.NewReadWriteConn(br, conn, conn)` 传给 sniffer。`HandleRedis` 从该 conn 读取时,`br` 已含 preread 的 RESP 前缀字节。无需额外处理。
### PG Peek(8) 的安全性
`byte0==0x00` 才进入 Peek(8)——PG StartupMessage 始终 ≥8 字节(4 字节 length + 4 字节 version + 至少一个 `\0`),不会阻塞真实 PG 客户端。所有 dispatch site 在 Peek 前设置 `sniffingTimeout` 作为兜底保护。非 PG 客户端首个字节为 `0x00` 的概率在现实流量中可忽略。
### RESP command vs key 大小写
Redis 命令大小写不敏感 → `strings.ToUpper` 存入 `ro.Redis.Command`。Redis key 大小写敏感 → 保持原大小写存入 `ro.Redis.Key`。路由匹配时 `protocol: redis` 用于 hop 选择器,`command` 和 `key` 用于 recorder/日志,不做 key 的大小写归一化。
---
## 改动文件清单(精简)
### 第一阶段(Redis)
**sniffing 包(单事实来源):**
- `x/internal/util/sniffing/sniff.go` — Tier-1 Redis 分支(Peek(1))
- `x/internal/util/sniffing/sniffer_redis.go` — 新增(`HandleRedis` + `ParseRedisMetadata`)
**forwarder 变体:**
- `x/internal/util/forwarder/sniffer_redis.go` — 新增(`HandleRedis` + `resolveRedisNode`)
**recorder:**
- `x/recorder/recorder.go` — `RedisRecorderObject` + `HandlerRecorderObject.Redis` 字段
**handler 分派(5 处,仅 Redis case):**
- `x/handler/forward/local/sniffing.go`
- `x/handler/forward/remote/sniffing.go`
- `x/handler/sni/sniffing.go`
- `x/handler/http/connect.go`
- `x/handler/relay/connect.go`
**文档:** `play/` 或 `docs/` 增加 Redis sniffing 示例(可选)
> 跨 repo 联动(gost-plugins/recorder、inspector)仅标注,不在本计划执行范围,见「跨 repo 联动」小节。
### 第二阶段(PostgreSQL)
- `x/internal/util/sniffing/sniff.go` — Tier-1 PG 分支(byte0=0x00 → Peek(8))
- `x/internal/util/sniffing/sniffer_postgres.go` — 新增(`HandlePostgres` + `ParsePostgresMetadata`)
- `x/internal/util/forwarder/sniffer_postgres.go` — 新增(`HandlePostgres` + `resolvePostgresNode`)
- `x/recorder/recorder.go` — `PostgresRecorderObject` + `HandlerRecorderObject.Postgres` 字段
- 5 处 handler 分派,加 Postgres case
> 跨 repo 联动(gost-plugins/recorder、inspector)仅标注,不在本计划执行范围,见「跨 repo 联动」小节。
---
## 范围之外(明确不做)
- MySQL / SMTP:服务端先发包,当前 preread 架构无法检测。
- Mongo/Memcached/OpenVPN/RTMP:可作为后续独立 enh,本期不做。
- TLS 内层解密后的 Redis/PG 明文嗅探(MITM 仅 HTTP):不做。
- SSH 嗅探 handler:已讨论,价值低(无路由元数据),本期不接(Sniff() 已有 ProtoSSH 检测)。
- auto handler 的 Redis/PG 接入:不做(auto 语义是 SOCKS/TLS/HTTP 三元分发,且 auto 路径缺乏 `HandleRedis`/`HandlePostgres` 需要的 sniffing Sniffer)。
@@ -0,0 +1,324 @@
# GitHub 完整镜像:GOST 头重写 + URL 编码回程路由
## Context
前一轮已验证:GOST 内置 **body 重写**(`http.rewriteResponseBody`)对真实 github.com 完全生效。要做成**完整 GitHub 镜像站**,剩余缺口:
1. **响应头重写**:`Location`(302 跳回原域)、`Set-Cookie`(Domain 绑定原域 → 登录挂)、`Content-Security-Policy`(阻断镜像资源)。目前只有静态 `responseHeader` 注入(`.Set(k,v)`),无对头**值**的正则重写。全模块 grep 确认这三个头目前完全透传。
2. **请求头重写**(用户要求对称补齐):如 `Referer`/`Origin` 指回原域、`Cookie` 域相关值、`X-Forwarded-Host` 等。
3. **URL 编码 + 回程路由**(用户提出的关键闭环问题):body/header 里的 URL 改成镜像地址后,用户**再次请求**时如何还原"原始上游域名"来路由回正确 origin。
4. **多域路由**:GitHub 资源分散在 `*.githubusercontent.com`、`gist`、`codeload` 等。
5. **HTTPS 内可见性**:标准反向代理即可——客户端对 GOST 走 HTTP(`127.0.0.1:8080`),GOST 对上游走 HTTPS(`tls.secure: true` 校验 GitHub 证书)。**无需 MITM / 客户端信任证书**。
用户目标:完整 GitHub 镜像(JS 运行时注入的 URL 明确排除)。镜像服务可能直接用 **IP**(`127.0.0.1:8080`)而非域名,故不采用子域编码。
## 核心设计 A:URL 编码 + 回程路由(纯既有机制,零新参数)
镜像 URL = `MIRROR_HOST/<编码后的host>/<原始path>`。闭环各环节全部用既有机制实现,**不新增任何回程解码参数**:
| 环节 | 机制 | 状态 |
|------|------|------|
| 出程:origin URL → 镜像 URL(body/Location/Referer 等) | body/header 正则重写 | ✅ 已有 body 重写,本次补 header 重写 |
| 回程:按编码前缀路由到 origin node | `matcher.rule` 用**原始 path**([sniffer_http.go:279](x/internal/util/forwarder/sniffer_http.go#L279) `PathSelectOption(req.URL.Path)`) | ✅ 纯配置 `PathPrefix`/`PathRegexp` |
| 回程:剥首段 | `http.rewriteURL`([sniffer_http.go:476-492](x/internal/util/forwarder/sniffer_http.go#L476-L492)) | ✅ 已有,`match: '^/github\.com/'` → `replacement: '/'` |
| 回程:还原上游 Host | `http.host` **静态值**(:464-466) | ✅ 已有,`host: github.com` |
**关键洞察(用户提出,取代原 `HostFromPathPrefix` 方案)**:SNI/拨号目标钉死在 `node.Addr`([sniffer_http.go:345](x/internal/util/forwarder/sniffer_http.go#L345)、`:360` `tlsWrapConn`),故一个 node 只能正确服务一个 origin。既然每 node 的 Host 在配置时**静态已知**,就不需要"运行时从 path 动态提取"——用 `http.host` 写死 + `rewriteURL` 剥前缀即可。原 `HostFromPathPrefix` 动态提取是死灵活性(它唯一能买的"单 node 多 host"场景在单 origin 约束下本就不该做,且 SNI 静态 + Host 动态拼不出可用 CDN 回源),**已删除**。
**编码分叉(关键约束:正则做不了 base64)**:`regexp.ReplaceAll` 只能拼固定文本,算不出 base64。所以:
- **明文 host 前缀**(选定):`https://github.com/x` → `https://MIRROR_HOST/github.com/x`。body/header 改写全纯正则,**零插件**。github 的 origin host 只有 `[a-z0-9-]+\.(github|githubusercontent|githubassets)\.com`,path 首段绝对安全,调试直观(curl 可读)。
- **base64url 前缀**(不做):编码需走插件(正则算不出 base64),收益仅"支持含 `:` `/` 特殊字符的任意 host",github 用不上。见"不做"。
**头重写的传递方向(与 body URL 非对称,三类)**:
| 方向 | 作用对象 | 例子 | 传递机制 |
|------|---------|------|---------|
| **正向编码** origin→mirror | 响应 body + 响应 `Location` | body 链接、302 | 与 body 同构,回程 `http.host` + `rewriteURL` 解码 |
| **逆向解码** mirror→origin(或删) | 请求 `Referer`/`Origin`/`X-Forwarded-Host` | 客户端回传已带 mirror URL | 需解码或删除,**不可正向编码** |
| **纯删除**(浏览器态) | 响应 `Set-Cookie` Domain、CSP、HSTS | 无回程 | `match: '.*'` → `replacement: ''` |
关键:请求头是 client→上游,浏览器访问镜像时 `Referer`/`Origin` 已是 `https://127.0.0.1:8080/...`(编码态)。请求头重写只能**解码回 origin 或删除**,绝不能正向编码。GitHub 浏览场景 CSRF 靠 token 而非 Referer,故**删除最简且正确**——同时避免把 mirror host 泄漏给上游。
## 核心设计 B:头重写支持插件
`core/rewriter.Rewriter` 接口是字节式(`Rewrite(ctx, b []byte, opts ...RewriteOption) ([]byte, error)`),gRPC/HTTP 插件后端已按此实现(`x/rewriter/plugin/{grpc,http}.go`)。为头复用该接口,采用**头块序列化**:
- **序列化**:`http.Header.Write(buf)` → MIME 字节(`Key: value\r\n`,多值 `Set-Cookie` 一值一行)
- **插件调用**:`rw.Rewriter.Rewrite(ctx, buf.Bytes(), MetadataRewriteOption(rewriteMeta(...)))`
- **解析回**:`textproto.ReadMIMEHeader` → `http.Header`
插件可增删改任意头;`Name` 正则作为"是否调用插件"的门控(与 body 的 `shouldApply`/`Pattern` 门控同构,`sniffer_sse.go:491-493`)。零改动 `core/`、`plugin/`、`gost-plugins/`。
⚠️ **序列化/解析回的两个保真度细节(落地时已踩坑并修复)**:
- `http.Header.Write` 只输出 `Key: value\r\n` 行、**不带结尾空行**,而 `textproto.ReadMIMEHeader` 需要空行才能判定结束 → 序列化后必须补 `\r\n`;插件输出缺结尾空行时解析前补 `\r\n`。
- `http.Header.Write` 会**排序** key、`ReadMIMEHeader` 会**规范化**名字——插件拿到的头块不是线序/原始字节。正则场景无影响,但需在 `HTTPHeaderRewriteSettings.Rewriter` 注释里点明该保真度上限。
## 核心设计 C:元数据标准化(`kind` + 收敛公共字段)
现状:插件元数据是**四处手搓 map**,公共字段 `sid/direction/uri` 反复复制,且全部缺 `kind`:
- `baseMetadata()`(`sniffer_sse.go:380-396`)——非 SSE body 路径
- SSE 三个内联 map(`sniffer_sse.go:299`、`:323`、`:353`)——各自再拼 `sse_phase`/`event_index`/`stream_error`
本次加头重写是**第六个使用点**。收敛到一个共享 helper,公共字段统一为 `sid / direction / uri / kind`:
新增 `x/internal/util/forwarder/rewrite_metadata.go`:
```go
// Rewriter-plugin metadata common fields (JSON convention, opaque to
// core/plugin — carried in RewriteRequest.Metadata).
const (
MetaKeySid = "sid"
MetaKeyDirection = "direction"
MetaKeyURI = "uri"
MetaKeyKind = "kind"
)
// kind values: what the plugin is receiving (byte payload vs header block).
const (
KindBody = "body"
KindHeader = "header"
)
// rewriteMeta builds the standard plugin metadata: sid/direction/uri/kind
// plus dimension-specific extras. Empty common fields are omitted so plugins
// can rely on presence rather than empty strings.
func rewriteMeta(sid, direction, uri, kind string, extras map[string]any) map[string]any {
md := map[string]any{}
if sid != "" { md[MetaKeySid] = sid }
if direction != "" { md[MetaKeyDirection] = direction }
if uri != "" { md[MetaKeyURI] = uri }
if kind != "" { md[MetaKeyKind] = kind }
maps.Copy(md, extras)
return md
}
```
改造:
- `baseMetadata()` → `return rewriteMeta(b.sid, b.direction, b.uri, KindBody, nil)`
- 三个 SSE 内联 map → `rewriteMeta(..., KindBody, map[string]any{"sse_phase": ..., "event_index": ...})`(`stream_error` 同理)
- 头重写 → `rewriteMeta(sid, direction, uri, KindHeader, nil)`
`kind` 为**新增字段**,对现有插件(llm-api-converter 等)是加法式向后兼容——未知字段忽略。`direction` 沿用既有取值 `"request"/"response"`。
## 代码改动(5 处,全部镜像现有 body-rewrite 模式)
### 1. `core/chain/node.go` — 纯数据结构体
> ✅ **已获批**:用户确认 core/ 可修改(2026-08-28)。本改动是既有 body-rewrite 模式的延续(`HTTPBodyRewriteSettings` :55、`RewriteRequestBody`/`RewriteResponseBody` :84-86 已同构引入),非新创接口。
`HTTPNodeSettings`(:71-89)加两个字段:
```go
// RewriteRequestHeader holds the request-header rewrite rules.
RewriteRequestHeader []HTTPHeaderRewriteSettings
// RewriteResponseHeader holds the response-header rewrite rules.
RewriteResponseHeader []HTTPHeaderRewriteSettings
```
新增结构体(对齐 `HTTPBodyRewriteSettings` :54-69,MIME Type→Name、去 MaxChunkSize):
```go
// HTTPHeaderRewriteSettings defines an HTTP header rewrite rule.
type HTTPHeaderRewriteSettings struct {
// Name is the regex matched against header names (case-insensitive).
// In plugin mode it gates whether the plugin is invoked.
// Note: *regexp.Regexp (not string like HTTPBodyRewriteSettings.Type)
// is intentional — header *names* need regex matching, while body rules
// match a MIME prefix. Do not "align" this back to string.
Name *regexp.Regexp
// Pattern is the regex matched against each header value (regex mode);
// in plugin mode it is unused.
Pattern *regexp.Regexp
// Replacement is the replacement bytes (regex mode).
Replacement []byte
// Rewriter is an optional plugin-based rewriter. When set, Rewrite
// delegates to the plugin over the serialized header block. The plugin
// receives a sorted, canonicalized header block (http.Header.Write
// normalizes names and sorts keys), not the original wire bytes.
Rewriter rewriter.Rewriter
}
```
### 2. `x/config/config.go` — 配置结构
`HTTPNodeConfig`(:505-530)加两个字段(与 `RewriteRequestBody`/`RewriteResponseBody` 对称):
```go
// rewrite request header values
RewriteRequestHeader []HTTPHeaderRewriteConfig `yaml:"rewriteRequestHeader,omitempty" json:"rewriteRequestHeader,omitempty"`
// rewrite response header values
RewriteResponseHeader []HTTPHeaderRewriteConfig `yaml:"rewriteResponseHeader,omitempty" json:"rewriteResponseHeader,omitempty"`
```
新增(对齐 `HTTPBodyRewriteConfig` :457-465):
```go
type HTTPHeaderRewriteConfig struct {
// Name is the header name regex, e.g. "(?i)^(location|set-cookie)$".
Name string
// Match is the header value regex.
Match string
// Replacement is the replacement string.
Replacement string
// Rewriter is the name of the rewriter plugin (via registry).
Rewriter string `yaml:",omitempty" json:"rewriter,omitempty"`
}
```
### 3. `x/config/parsing/node/parse.go` — 解析
新增 `parseHeaderRewrites(vs []config.HTTPHeaderRewriteConfig, log logger.Logger) []chain.HTTPHeaderRewriteSettings`,镜像 `parseBodyRewrites`(:89-144):
- `Name` 与 `Match` 各自 `regexp.Compile`(`Match` 为空 → Pattern nil;`Name` 为空但 `Rewriter` 非空 → 插件门控"始终调用")
- `v.Rewriter != ""` → `registry.RewriterRegistry().Get()`(未注册 `log.Warnf`,同 :133-138)
- ⚠️ **空 Name 门控必须用配置字符串判空**:`regexp.Compile("")` 返回**非 nil** 的 match-all 正则。若拿编译结果 `name != nil` 判空,`name: ""` + 无 rewriter 的规则会通过门控、对每个头名匹配但 `Pattern==nil` 跳过值 → **静默 no-op**。正确做法:append 门控判 `v.Name != "" || rw.Rewriter != nil`(编译前,用 `v.Name` 字符串);插件"始终调用"门控在 `v.Name == ""` 且 `v.Rewriter != ""` 时把 `Name` 字段留 nil。
组装块(:315-357)追加:
```go
settings.RewriteRequestHeader = parseHeaderRewrites(cfg.HTTP.RewriteRequestHeader, log)
settings.RewriteResponseHeader = parseHeaderRewrites(cfg.HTTP.RewriteResponseHeader, log)
```
### 4. `x/internal/util/forwarder/rewrite_metadata.go` — 新增
`rewriteMeta()` helper + `MetaKey*`/`Kind*` 常量(见核心设计 C)。
### 5. `x/internal/util/forwarder/sniffer_rewrite.go` + `sniffer_http.go` — 头重写实现 + 接线
`rewriteRespBody`(:44)的兄弟;新增一个共享实现 + 两个薄封装:
```go
// rewriteHeaderBlock applies the rewrite chain to h in place.
func rewriteHeaderBlock(ctx context.Context, h http.Header, rewrites []chain.HTTPHeaderRewriteSettings, direction, uri string) error
```
语义(逐规则):
- **插件模式**(`rw.Rewriter != nil`):
- 门控:`rw.Name == nil` 或存在任一头名匹配 `rw.Name`(大小写不敏感)→ 否则跳过
- 序列化整个 `h`(补结尾空行)→ 插件(元数据 `rewriteMeta(sid, direction, uri, KindHeader, nil)`)→ `textproto.ReadMIMEHeader` 解析回 → 替换 `h`
- ⚠️ **`h` 是 map,不可靠函数内重赋值**:`h = newHeader` 只改形参引用、调用方无感。必须 `clear(h)` 后从解析结果逐 key 拷贝回(或 `maps.Copy`)。
- **正则模式**(`rw.Rewriter == nil && rw.Name != nil`):
- 遍历头名匹配 `rw.Name` 的头(`strings.ToLower` 双方归一)
- 每个值独立 `rw.Pattern.ReplaceAll`(`Pattern == nil` 跳过该值)
- 替换结果为空 → 丢弃该值;该头值全空 → `h.Del(name)`(**删除语义**,沿用 `RequestHeader` `v=="" → Del` 的既有约定,`sniffer_http.go:467-474`)
薄封装:
```go
func rewriteReqHeader(ctx context.Context, req *http.Request, rewrites ...) error
func rewriteRespHeader(ctx context.Context, resp *http.Response, rewrites ...) error
```
(`req.Header`/`resp.Header` nil 守卫;`direction`="request"/"response",`uri`=`req.RequestURI`。)
**`httpRoundTrip`** 接线(行号已核对):
- 请求侧:`hasReqRewrite`(:413-415)追加 `|| len(httpSettings.RewriteRequestHeader) > 0`;捕获 `reqHeaderRewrites`;在 RequestHeader 注入(:467-474)与 RewriteURL(:476-492)**之后**、body 快照(:502)之前调用 `rewriteReqHeader`(先于 :518/522 的 `req.Write(cc)` 上游写出)
- ⚠️ **recorder 请求侧保真度**:`ro.HTTP.Request.Header` 最后一次赋值在 `:473`(静态注入循环内),`rewriteReqHeader` 插在它之后 → recorder 仍显示删除前的 `Referer`/`Origin`/`Cookie`。需在 `rewriteReqHeader` 后补 `ro.HTTP.Request.Header = req.Header.Clone()`。
- 响应侧:`hasRespRewrite`(:416-417)追加 `|| len(httpSettings.RewriteResponseHeader) > 0`;捕获 `respHeaderRewrites`;在静态 `responseHeader` 覆盖块(:622-629)**之后**、:630 `ro.HTTP.Response.Header.Clone()` 之前调用 `rewriteRespHeader`(recorder 记录改写后结果;满足 :620-621 的 Content-Type 顺序约束)
## 配置:完整 GitHub 镜像参考(`play/github-mirror.yaml`)
```yaml
services:
- name: github-mirror
addr: :8080
handler:
type: tcp
metadata:
sniffing: true
listener:
type: tcp
forwarder:
nodes:
- name: github
matcher:
rule: PathPrefix(`/github.com/`) || PathPrefix(`/www.github.com/`)
addr: github.com:443
tls: { secure: true }
http:
host: github.com # 静态 Host(回程还原,替代 hostFromPathPrefix)
rewriteURL:
- match: '^/github\.com/'
replacement: '/' # 剥 path 首段
rewriteResponseBody:
- type: text/html
match: 'https://(github\.com|www\.github\.com|gist\.github\.com)'
replacement: 'https://127.0.0.1:8080/$1'
maxChunkSize: 8388608
rewriteRequestHeader:
- name: '(?i)^(referer|origin|x-forwarded-host)$'
match: '.*'
replacement: '' # 删除:客户端 Referer 已是 mirror URL,回传上游会泄漏 mirror host
rewriteResponseHeader:
- name: '(?i)^location$'
match: 'https://(github\.com|www\.github\.com|gist\.github\.com)'
replacement: 'https://127.0.0.1:8080/$1'
- name: '(?i)^set-cookie$'
match: 'Domain=\.?github\.com;?\s*'
replacement: ''
- name: '(?i)^content-security-policy(-report-only)?$'
match: '.*'
replacement: '' # 值空 → 删头
- name: '(?i)^strict-transport-security$'
match: '.*'
replacement: ''
- name: ghusercontent
matcher:
rule: PathPrefix(`/raw.githubusercontent.com/`) || PathPrefix(`/avatars.githubusercontent.com/`) || PathPrefix(`/objects.githubusercontent.com/`)
addr: raw.githubusercontent.com:443
tls: { secure: true }
http:
host: raw.githubusercontent.com
rewriteURL:
- match: '^/raw\.githubusercontent\.com/'
replacement: '/'
# 匿名场景:剥 Domain 后 session cookie 会随 /raw.../ 路径跨 origin 泄漏,必须删
rewriteRequestHeader:
- name: '(?i)^cookie$'
match: '.*'
replacement: ''
# body/header 同 github 节点,match 换成 githubusercontent 域名
```
> ⚠️ **单 origin 约束**:SNI/拨号目标钉死在 `node.Addr`(`github.com:443`),故一个 node 只能正确服务一个 origin。`www.github.com` 复用 github node 可行(github 证书含 `*.github.com`);`gist.github.com` 须**单独 node**(`addr: gist.github.com:443` + 自己的 `host`/`rewriteURL`),不可塞进 github node 的 matcher。e2e 须单独验证 gist 回程。
说明:
- **出程编码**:body/header 正则把 `https://<origin-host>/...` 换成 `https://MIRROR_HOST/<origin-host>/...`,`$1` 捕获 host 整段,明文前缀直接可读。
- **回程解码**:客户端请求 `GET /github.com/ginuerzh/gost`(Host=`MIRROR_HOST`),node 按 `PathPrefix('/github.com/')` 选中,`rewriteURL` 剥前缀 → `/ginuerzh/gost`,`http.host` 还原 `req.Host=github.com`。
- **正则捕获组**:Go RE2 不支持 `(?:)` 非捕获组,`(a|b)` 交替按需编号 `$1`。`githubusercontent` 子域清单(raw/avatars/objects/camo/media/cloud/codeload)实现时按真实资源需求枚举 node + PathPrefix,可合并到一个 node 用 `PathRegexp`。
## 验证
1. `cd x && go build ./... && go vet ./...`;`cd ../core && go build ./... && go vet ./...`
2. 单测(`x/internal/util/forwarder/`,参考现有 rewrite 测试风格):
- `rewriteHeaderBlock` 正则模式:Location 改写、多值 Set-Cookie 逐值改写、`.*`→`` 删头、头名大小写混排、`Name` 不匹配时不动
- 插件模式:fake `Rewriter`(`type fakeRewriter func(...)`)验证门控 + 序列化/解析回 round-trip + 元数据含 `kind:"header"`
- 插件模式 `clear`+拷贝语义:fake `Rewriter` 返回增/删/改头块后,验证调用方 `http.Header` 被原地更新(`h = newHeader` 重赋值不生效的坑)
- 插件 round-trip 补结尾空行:验证 `http.Header.Write` 无空行、插件输出无空行两种情况下解析都不报 EOF
- `rewriteMeta`:非空字段保留、空字段省略;`KindBody`/`KindHeader` 正确落位;SSE 三处 map 改造后 `sse_phase`/`event_index` extra 与公共字段并存
- 空 `Name` 门控:`name: ""` + 无 rewriter → **不 append**(防静默 no-op,见 parse 警告)
3. e2e(真实 github.com + curl,沿用上轮已跑通的镜像 config):
- `curl -si http://127.0.0.1:8080/github.com/...` 验证:302 `Location` 指向 `127.0.0.1:8080/<host>/...`、`Set-Cookie` 无 `Domain=github.com`、`content-security-policy` 头消失
- 验证请求 `Referer`/`Origin`/`X-Forwarded-Host` 已删除(上游收不到 mirror host);body 里 `https://github.com/...` 全换成 `https://127.0.0.1:8080/github.com/...`(复跑 `/explore` 回归,替换不残留)
- **闭环回归**:`curl` 跟着 body 里改写后的 URL 二次请求,确认路由回正确 origin 且内容可加载
- **剥前缀边界**:`rewriteURL` 的 `match` 作用于 `req.URL.Path`,`PathPrefix('/github.com/')` 已含尾斜杠,剥完是 `/...`,行为正确——单独验证一次
4. `CGO_ENABLED=1 go test -race ./internal/util/forwarder/...`
## 不做(明确留白)
- JS 运行时注入的 URL(用户明确排除)
- **登录态**:v1 边界 = 匿名只读浏览 + 公开仓库/raw/avatars 资源浏览下载。登录链路有硬断点(CAPTCHA 第三方 iframe 未镜像、WebAuthn `rpId` 绑 github.com、SAML/SSO 回调 RelayState 绑 origin、Secure cookie 要求 HTTPS),不保证、不投精力。v1 匿名场景下仍需处理**跨 origin Cookie 泄漏**:ghusercontent 节点加 `rewriteRequestHeader` 删除 `Cookie`,防止剥 Domain 后 session cookie 随 `/raw.../` 路径泄漏到 githubusercontent 上游
- base64url host 前缀——明文前缀 github 场景够用;需要支持任意 host(含 `:` `/` 特殊字符)时再加:编码走插件
- 头重写的 `delete bool` 显式字段——用 `match: '.*'` + `replacement: ''` 表达删除
- 扩展 `core/rewriter.Rewriter` 接口为结构化头接口——字节式序列化已复用现有插件链路,零改动
- githubusercontent 各子域 SNI 动态化(node addr 固定、tls.serverName 静态)——多子域真实 SNI 需按回程 host 联动 `tlsWrapConn`,属部署细节,e2e 遇到再补
- gist 单独 node(`addr: gist.github.com:443`)的 SNI/证书验证——单 origin 约束见 config 内警告,e2e 单独跑 gist 回程
- 修正 `x/CLAUDE.md` 过时描述("There are no tests in this module")——`x/` 下现有 182 个 `_test.go`,本次在 `x/internal/util/forwarder/` 加单测符合现状,无需为加测改文档
+383
View File
@@ -0,0 +1,383 @@
# Plan: Move the tun-to-tun link out of p2p into GOST, over a p2p datagram tunnel
**Date:** 2026-09-11
**Status:** Implemented (see Implementation notes at the end)
**Scope:** `plugin/p2p` (protocol), `x/` (p2p wrapper + tun handler), `p2p/` (tunnel host)
## Context
The tun→tun link currently lives entirely inside the `p2p` host: `--link "p2p0=<key>"`
opens a device that must already exist (`p2p/dev.go`, `p2p/link.go`) and bridges raw
chunks over one persistent smux stream. p2p therefore owns device attach, framing and
the link role, while device creation, addressing, MTU, routes and DNS are the caller's
problem — and a tun is exclusive-open, so it can never be shared with GOST's own tun
listener.
GOST already has a complete, documented tun stack: the `tun` listener creates *and*
configures the device (`net`, `mtu`, `name`, `dns`, `route`, `routes`, `gw`), and the
`tun` handler in client mode bridges that device to a datagram connection obtained from
the service's chain. The goal is therefore: **GOST owns the device, p2p is only the
pipe**, reproducing the existing `--link` capability by configuration instead of by p2p
code.
Two questions were raised:
**1. How does p2p replace the `router` handler?** `x/handler/router/` does two jobs:
(a) it is the *transport hub* — clients dial it with the relay associate protocol, it
frames IP packets (`packetConn`, 2-byte length prefix) and reaches other nodes over its
UDP `entrypoint` mesh; (b) it holds the *routing table* (dst IP → gateway →
connector, fed by `routers:`/ingress) and dispatches to the peer owning the destination.
With p2p, (a) disappears: every peer pair gets its own direct datagram channel, with
relay fallback and NAT traversal handled by p2p — no hub node, no associate framing, no
`entrypoint` UDP mesh. (b) moves to each node: the device's own addressing plus the tun
listener's `route`/`routes`/`gw` metadata (a single link is point-to-point, so there is
nothing to route), and for a hub the OS routing table across per-peer devices. What is
given up: the router's single-node fan-out and its `ingress`/`sd` integration.
**2. Does `OpenTunnel` need a `network`?** Yes. A tun handler writes one IP packet per
`Write` and expects one packet per `Read`; a TCP endpoint coalesces and splits, which
corrupts packet boundaries (the same reason `packetConn` and p2p's own `writeFrame`
exist). `OpenTunnelRequest` gains `network`; the host binds a TCP or a UDP socket
accordingly, and for UDP each datagram travels as one length-prefixed frame over the
smux stream. The reply's `endpoint` stays scheme-less — the caller knows which network
it asked for. TCP is the default, so existing deployments are untouched.
## Topology
Chosen: **symmetric — both ends are tun clients** (no hub, no tun server).
```
host A host B
gost tun listener (10.10.0.1/30) gost tun listener (10.10.0.2/30)
tun handler (client, chain → node addr = B key) tun handler (client, chain → A key)
│ dialer: udp, connector: forward │ (mirror)
▼ ▼
p2p udp endpoint E_A ◄─ channel(A→B) ─► E_B p2p (same, mirrored)
+ smux stream: smaller key opens, larger accepts
```
Both sides run the *same* gost config (only the peer key and device address differ).
The p2p stream is a virtual cable between two tun clients, and each side's local UDP
datagram endpoint is what the tun handler dials.
**Star (hub + N spokes)**: the hub runs N tun services, one device per peer (one /30
each), and the kernel routes between them (`net.ipv4.ip_forward=1`, the documented
multi-interface VPN setup). p2p provides one channel per peer. Full mesh on a *single*
device (N peers per tun handler) is out of scope — that would be a tun-handler redesign,
not a p2p one.
## Final config shape (side A; side B mirrored)
```yaml
p2ps:
- name: p2p-1
plugin:
type: grpc
addr: 127.0.0.1:8003
# token: gost # set when the p2p host runs with --token
services:
- name: tun-0
addr: :0 # never bound in client mode, but must not be empty
handler:
type: tun
chain: chain-0 # chain without forwarder → client mode (new rule)
listener:
type: tun
metadata:
net: 10.10.0.1/30
mtu: 1420
# route: 192.168.123.0/24 # to reach networks behind the peer (hub/star)
chains:
- name: chain-0
hops:
- name: hop-0
nodes:
- name: node-0
addr: <B public key> # in p2p mode addr IS the peer key (base64 rawurl)
dialer:
type: udp # datagram semantics: chooses the UDP tunnel endpoint
connector:
type: forward # transparent; must be explicit, the default is http
metadata:
p2p: p2p-1
log:
level: debug
```
```bash
# side A p2p host (no --target, no --link)
p2p --derp wss://derp.example.com/derp --key a.key \
--addr 127.0.0.1:8003 --stun derp.example.com:3478
```
The CLI form now reads equally well:
```bash
gost -C p2ps.yaml \
-L "tun://:0?net=10.10.0.1/30&mtu=1420" \
-F "forward+udp://<B key>?node.p2p=p2p-1"
```
`-F` attaches the chain to the service (which triggers client mode); unprefixed query
parameters land in handler, listener and service metadata alike
(`x/config/cmd/cmd.go:592`), and `node.`-prefixed ones in node metadata
(`x/config/cmd/cmd.go:642`). `p2ps:` only exists in the config file, so `-C` is still
required.
### Why not `direct`/`virtual` nodes
`dialer: virtual`/`direct` (`x/dialer/direct/dialer.go:37`) returns `&conn{}` without
ever touching `options.Dialer` — and the p2p wrapper takes over dialing precisely by
replacing `options.Dialer`, so the tunnel would never be opened. `connector:
virtual`/`direct` (`x/connector/direct/connector.go:51`) is the same type and *ignores
the dialed conn*, dialing the destination itself with the internal net dialer — it would
bypass the tunnel and try to dial the public key as an address. The documented
transparent connector is `forward` ("does no logic of its own, used with a data
forwarding channel", `docs/en/docs/reference/connectors/forward.md`, whose example —
`dialer: tls` + `connector: forward` — has exactly this shape); `tcp` is code-equivalent
but reads like a protocol.
### Client/server decision (the handler change)
Today `h.hop` is set only by the service-level `forwarder:`
(`x/config/parsing/service/parse.go:430` is the only caller of `Forward()` in the tree),
and `x/handler/tun/handler.go:92` sends `h.hop == nil` straight to `handleServer` — a
server that binds a UDP socket and waits for peer heartbeats. The handler cannot reach
the chain's nodes either (`Chainer` only has `Route()`, nothing to enumerate nodes), so
without a forwarder there is no way into client mode.
The change (the same idiom `serial`/`unix` handlers already use — see
`x/handler/serial/handler.go:229` and `x/handler/unix/handler.go:217`):
- `forwarder` present → the hop node it selects (unchanged);
- no `forwarder` but `h.options.Router.Options().Chain != nil` → **client mode**, with an
empty dial address;
- neither → server mode (unchanged). `h.options.Router` and `Options()` both need nil
guards.
An empty dial address is safe: the link is transparent to tun, the `forward` connector
never reads the address, and `Chain.Route` / `Resolve` / `Route.Dial` all tolerate an
empty string (`x/chain/chain.go:132` does not parse the address). The p2p **peer comes
from the chain node's addr** (`Transport.Dial(ctx, node.Addr)`), not from this address.
The one consequence: a non-transparent connector (http/socks5/ss/…) has no destination,
so those still need an explicit `forwarder`.
**Compatibility note (goes into the release notes and the tun docs):** `tun server +
chain` used to be dead config — the chain was built and never read (`h.options.Router` is
read only at `x/handler/tun/client.go:41`; the server branch and the tun listener never
touch the chain). It is now interpreted as client mode. The shape does occur in practice
because `-F` attaches the chain to every `-L` service in CLI mode
(`x/config/cmd/cmd.go:210`). Treating that combination as invalid was a deliberate call;
the docs must say so.
**The forwarder still earns its place** for direct (no-chain) dialing — the documented
client example `tun://:0/SERVER_IP:8421`, where it is the real peer address — and for
non-transparent connectors. It also decides whether the handler's `network` is `udp`
(address has a port → heartbeats) or `ip` (no port → none).
## Changes
### plugin/ — protocol
- `plugin/p2p/proto/p2p.proto`: add `string network = 2;` to `OpenTunnelRequest` (values
`tcp`/`udp`, empty = `tcp`). Regenerate from the module root as `plugin/CLAUDE.md`
requires (`protoc --proto_path=. --go_out=. --go_opt=paths=source_relative
--go-grpc_out=... p2p/proto/p2p.proto`) so the registered file name stays
`p2p/proto/p2p.proto`.
### x/ — GOST side
- `x/handler/tun/handler.go`: the client/server decision above, extracted into a small
`selectTarget(ctx)`; one branch left in `Handle`. Nil guards on `h.options.Router` and
`Options()`. The client-mode `dst` log field degrades to empty without a forwarder —
keep the existing format.
- `x/p2p/tunnel_dialer.go`:
- `TunnelProvider.OpenTunnel(ctx, network, peer)` — thread the network the inner dialer
asked for.
- `tunnelBaseDialer.Dial` currently hardcodes `"tcp"`: normalize the incoming network
(`udp`/`udp4`/`udp6` → `udp`, everything else → `tcp`), pass it to `OpenTunnel` and
use it for the endpoint dial.
- `SupportedDialer`: allow `"udp"` (its comment says the wrapper's contract is "inner
dials one plain TCP stream" — it must become "a stream, or a datagram conn for udp").
- `x/p2p/plugin/grpc.go`: set `Network` on the request (`grpc_test.go` too).
- `x/dialer/udp/dialer.go`: `c.(*net.UDPConn)` is a hard assertion, so any wrapped base
dialer panics — and the p2p wrapper returns a `tunnelConn`. Keep the existing branch
when the assertion holds (downstream code uses `*net.UDPConn` to mean "direct"), add a
`net.Conn`-based fallback with the same `ReadFrom`/`WriteTo` semantics otherwise.
- `play/p2p-tun.yaml`: the new example (both sides + the p2p host commands), i.e. the
config above. Note that **`connector: {type: forward}` must be explicit** — the node
parse default is `http`, which would speak CONNECT over the link; and that
`direct`/`virtual` mean "no hop, dial the destination directly", the opposite of what is
wanted here.
- Optional, **not required by this design**: making the tun handler start keepalives when
`keepAlive`/`ttl` metadata is set instead of only when the forwarder address carries a
port (`x/handler/tun/client.go:51`). With two tun clients there is no server learning
addresses, so heartbeats only add noise plus a 30 s read deadline that forces pointless
re-dials; the example therefore omits `keepAlive`. Can be added on request.
- Docs (separate repo, follow-up): `docs/*/tutorials/tuntap.md` and the p2p blog post,
both gaining the new "chain means client" rule and the compatibility note.
### p2p/ — the datagram tunnel replaces the device link
New `p2p/udp.go`:
- **Datagram endpoint** — `net.ListenPacket("udp", bind:0)`, tracking the last-seen client
address (gost dials a connected socket, so the source is stable; a re-dial changes it
and last-writer-wins). Datagram in → `writeFrame(current stream)`, dropped when the
stream is down (IP tolerates loss). One reader goroutine per stream: `readFrame` →
`WriteTo(client)`.
- **Per-peer channel** — `{endpoint, current stream, opener, refs}`. Role by public-key
ordering, the same rule as the smux session and the old `link.go`: the smaller key
opens. The opener runs `channelLoop` (the shape of the old `linkLoop`: `OpenStream` →
serve → backoff retry, `<-e.stop` to exit); the responder opens nothing and waits.
- **Stream tagging** — a stream carrying a datagram channel is prefixed with a 4-byte
magic (`P2PU`) before the first frame, so the responder can tell it apart from an
ordinary tunnel stream and still bridge untagged ones to `--target` as today (peek 4
bytes, re-prepend on mismatch). Length alone is not enough: a normal tunnel stream's
payload is arbitrary bytes.
- **Reuse** `writeFrame`/`readFrame`/`writeAll`/`maxFrame`, moved out of `dev.go` into
`p2p/frame.go` (the same 2-byte BE prefix format as `router`'s `packetConn`).
`p2p/server.go`:
- `OpenTunnel`: validate `network` (`tcp`/`udp`, else `InvalidArgument`); for `udp`
require engine mode, get or create the peer channel and reply with its endpoint.
`CloseTunnel` decrements the channel's refcount and tears it down at zero (closes the
endpoint socket and the stream; the next `OpenTunnel` rebuilds it).
- Stub mode (no `--derp`): `udp` tunnels pump datagrams straight to the peer `host:port`
— a dev aid mirroring the existing stub TCP bridge, and the hook for a cheap e2e test.
`p2p/engine.go` + `p2p/direct.go`: add the `channels` map and accessor. In `acceptLoop`
(`direct.go:639`, which already branches on `e.dev` for device mode), peek the 4-byte tag
on every inbound stream: tagged → serve on the peer's channel, and if the channel does
not exist yet (the peer's gost has not dialled) close it at debug level and let the
opener's backoff retry — a normal startup race, not an error. Untagged → re-prepend and
bridge to `--target` exactly as today.
Delete `dev.go`, `dev_linux.go`, `dev_other.go`, `dev_test.go`, `link.go`, and the
`--link` flag, the `links:` config and its validation in `p2p/main.go` / `p2p/config.go`.
p2p stops opening devices entirely. (tap users keep GOST's `tap` handler.)
Docs: `p2p/CLAUDE.md` (flag table, positioning, trust boundary — the endpoint is
loopback-only like the gRPC control plane, the data plane is still unencrypted),
`p2p/README.md` and `README.zh-CN.md`.
## Verification
Builds and static checks:
```bash
cd plugin && go build ./... && go vet ./...
cd x && go build ./... && go vet ./...
cd p2p && go build ./... && go vet ./... && GOWORK=off go build ./...
cd p2p && CGO_ENABLED=1 go test -race ./... # -race needs CGO
```
New/replacement tests (p2p):
- `udp_test.go` replacing `dev_test.go`: framing through the channel with irregular reads
and partial writes (the cases `dev_test.go` covered), the endpoint's last-seen-client
update after a re-dial, and magic-tag sniffing (tagged → channel, untagged →
`--target` bridge).
- One happy path through two in-process engines over the existing test relay
(`engine_test.go` harness): datagrams written on one endpoint come out the other with
bytes and boundaries intact.
- x side: a network pass-through assertion in `x/p2p/plugin/grpc_test.go`.
- x side, tun mode decision (extend the existing `x/handler/tun/` tests): with a fake
`chain.Router` whose `Options().Chain` is set or nil, assert the three combinations land
in client or server mode — forwarder present (client), no forwarder with a chain
(client), neither (server) — and that a nil `Router` does not panic.
End to end (Linux, `NET_ADMIN`; the old behaviour is the baseline):
1. Baseline: **before the change** (or with a released p2p binary), two hosts linked with
`--link` (`10.10.0.1/30` ↔ `10.10.0.2/30`), `ping` plus one `iperf3` run.
2. New path: side B `p2p --derp wss://<derper>/derp --key B.key --stun <derper>:3478`
(no `--target`, no `--link`) plus gost with the tun client config from
`play/p2p-tun.yaml`; side A mirrored with B's key and `--addr 127.0.0.1:8003`.
`ping 10.10.0.2` from A and `ping 10.10.0.1` from B, then `iperf3` both ways. Logs must
show the p2p host opening exactly one channel stream (the smaller key) and the peer
accepting it.
3. Kill the derper mid-transfer → traffic continues over the direct path (the same
expectation as the M2 milestone); a relay-only run (no `--stun`) must also work.
4. Regression: a plain TCP tunnel through the same p2p host (`play/p2p-derp.yaml` +
`curl`) still works — proving the `network` default is `tcp` and the untagged-stream
path is unchanged.
5. Idle check: leave the link idle for a few minutes; no tunnel churn (no repeated
Open/CloseTunnel in the gost log).
6. Mode-decision regression: `-L "tun://:8421?net=..."` (no chain, no forwarder) is still
a server — `ss -ulnp` shows it bound to 8421 and accepting peer heartbeats; the
documented client example (forwarder, no chain) still dials the forwarder address
directly.
## Steps
1. plugin: proto + regenerate.
2. x: tun handler mode decision first (independently verifiable); then the provider
signature, gRPC client, network threading, `SupportedDialer`, udp dialer fallback;
build + vet; run `go test ./p2p/... ./handler/tun/...`.
3. p2p: extract `frame.go`, write `udp.go` (endpoint + channel + tagging), `server.go`
dispatch and refcounting, the accept branch in `engine.go`/`direct.go`; delete dev/link;
manual loopback test.
4. p2p: tests, docs, README.
5. gost: `play/p2p-tun.yaml`, then the two-host e2e above.
## Implementation notes (2026-09-11)
The design review flagged seven unpinned details before coding; each is resolved as follows.
1. **Tag peek blocked the accept loop.** Classification moved out of `acceptLoop` into a
per-stream goroutine (`serveInbound`), and the peek is bounded (`channelTagTimeout`, a
var so tests can shorten it) with a `prefixConn` replaying whatever a partial read
consumed. Bounding is not optional: an *untagged* stream may legitimately be silent (a
bridged protocol whose target speaks first) and would otherwise deadlock that bridge.
Covered by `TestPeekTag` and `TestEngineIdleStreamBridged` (greeter-first target).
2. **`channelLoop` outlived its channel.** The loop now selects on `ch.stop` (channel-scoped,
closed by teardown) instead of the engine's, and re-checks `ch.stopped()` after
`OpenStream` returns so a stream opened during teardown is closed rather than published
to a dead channel. `TestChannelRefcount` asserts a rebuilt channel gets a fresh stop.
3. **Tag vs. frame ordering.** The opener writes the tag *before* the stream is published
(`serveStream`/`setStream`), so the endpoint reader cannot frame ahead of it.
4. **Frames before the local gost dialled.** Dropped instead of written to an unknown
address (`TestChannelDropsWithoutClient`).
5. **Stub-mode udp.** Removed rather than built: `udp` requires engine mode
(`codes.InvalidArgument` otherwise), and the in-process relay harness covers the test
need, so the stub pump would have been dead spec.
6. **`addr: :0` comment.** Corrected — `net.ResolveUDPAddr("udp", "")` yields `:0` without
error, the listener does no further validation.
7. **Chain-means-client switch.** Now logged (`tun: chain without forwarder, running in
client mode`) instead of being silent.
One gap the design missed and the tests caught: the endpoint learned its client address only
from a datagram it received, so a link whose peer speaks first would never come up (both
sides waiting). GOST now announces itself with an empty datagram right after dialling a udp
endpoint (`tunnelBaseDialer.Dial`), and `readLocal` records the address for zero-length
datagrams too.
Verification as run:
- `plugin`, `x`, `gost`, `p2p`: build + vet clean; `p2p` and the touched `x` packages pass
`CGO_ENABLED=1 go test -race`.
- `p2p`, new tests: framing, channel endpoint/boundaries/last-client/refcount, tag
classification, engine round trip over the test relay, the idle-stream bridge.
- `x`, new tests: the three mode combinations plus a nil router (`selectTarget`), and the
udp network pass-through over the real gRPC seam.
- End to end (two network namespaces, a real `derper`, real tun devices, `NET_ADMIN`):
`ping` and a 20x1200B burst both ways at 0% loss, the channel on the **direct**
hole-punched path; killing the derper mid-run leaves the link up; restarting one gost
brings it back; a plain TCP tunnel through the same host (`--target`, curl) still returns
200, proving the untagged path and the `tcp` default are unchanged; `-L tun://:8421?...`
without a chain is still a server.
- `plugin` released as **v0.7.0** (the new field), and both `p2p` and `x` bumped to it, so
the standalone `GOWORK=off go build ./...` checks pass in each module. Before the tag,
`p2p`'s standalone build failed against v0.6.1 — the field did not exist there.
- Module states: `p2p` ships this as `p2p: replace the device link with datagram channels`
(plus the image going lean and the stale compose file going away); `x` ships it as
`feat(p2p): datagram tunnel endpoints for tun-to-tun links`.
- Deployment (`p2p/docker-compose.tun.yaml`) was dropped rather than rewritten: the old file
deployed the link with this host owning the device, which is exactly what this design
removes. `play/p2p-tun.yaml` above is the config side.
@@ -0,0 +1,38 @@
# x/p2p 接口改名:TunnelProvider → Tunnel,OpenTunnelStream → Dial
> 2026-09-22。与 p2p 的 `Tunnel` 接口对齐:`Tunnel` 是唯一对外的传输接口
> (`Dial` / `Listen` / `Close`,与 `net` 包一致);`Provider` 名字含义不清,一并改名。p2p 与 x 之间靠**结构化匹配**耦合
> (`registry.P2PRegistry().Register(name, host.Provider())`),所以接口方法名必须两仓同步;
> **x 先改并发版,p2p 与 wisper 跟随**(见 `p2p/docs/2026-09-22-p2p-host-listen.md`)。
## 改动(3 个文件)
1. `x/p2p/tunnel_dialer.go`
- 接口名 `TunnelProvider` → **`Tunnel`**;
- 方法 `OpenTunnelStream(ctx, network, peer string) (net.Conn, error)` → **`Dial(...)`**,接口注释同步;
- `NewTunnelDialer(inner dialer.Dialer, pr TunnelProvider)` → `(inner dialer.Dialer, t Tunnel)`;结构体字段 `provider TunnelProvider` → `tunnel Tunnel`(按 Go 习惯避免与类型名重复,实际命名以实现为准);
- 调用点(约 `:143`)`d.pr.OpenTunnelStream(...)` → `d.pr.Dial(...)`;
- 注释中提及方法名的两处(`:85`、`:124`、`:137`)同步。
2. `x/p2p/plugin/grpc.go`
- `func (p *grpcPlugin) OpenTunnelStream(...)` → `Dial(...)`;`:34`、`:61` 的注释同步。
3. `x/p2p/plugin/grpc_test.go`
- `countingProvider.OpenTunnelStream` → `Dial`;三处断言文案同步。
同时改:`x/registry/p2p.go`(`reg.Registry[xp2p.TunnelProvider]` → `Registry[xp2p.Tunnel]`,注释同步);
`x/config/parsing/p2p/parse.go` 与 `x/config/loader/loader.go` 中出现的 `xp2p.TunnelProvider` 类型引用同步改名。
不改:gost 主仓(无引用,已 grep 确认)。
## 验证
```bash
cd x && go build ./... && go vet ./...
grep -rn "OpenTunnelStream" x/ || echo "no residue"
```
## 发布(需用户确认)
```bash
git push && git tag v0.18.0 && git push origin v0.18.0
```
(接口改名是破坏性变更 → 按 0.x 语义升 minor;若倾向 patch 则 v0.17.3。)
File diff suppressed because it is too large Load Diff
@@ -0,0 +1,251 @@
# Tun device 写侧探针(x 侧)Implementation Plan
> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking.
**Goal:** x tun handler 在 client 模式下每 30s 往 device 写一个自回环 UDP 探针,
把发送/确认计数交给调用方(wisper 经 metadata 回调收数),读环路零改动。
**Architecture:** 新文件 `probe.go`(组包纯函数 + 探针循环,与 `keepalive` 同款
per-dial-iteration 生命周期);`x/observer/stats` 加两个计数器(加法字段,
core 不动);metadata 加 `probe` 开关(缺省关,gost 零行为变化)。
**Tech Stack:** Go, `golang.org/x/net/ipv4`(已在用)组包,UDP checksum 置零。
**Spec:** `wisper/docs/superpowers/specs/2026-10-07-wisper-tun-probe-design.md`
(§1/§2 是本 plan 的依据;本 plan 只覆盖 x 侧,wisper 侧见 Plan B
`wisper/docs/superpowers/plans/2026-10-07-tun-probe-watchdog.md`)
## Global Constraints
- `core/` 一行不碰(Kind 常量在 x 内定义,见 Task 1 精确值)。
- 缺省关闭:metadata 无 `probe` 键时不建 socket、不起 goroutine、不写计数
(gost 路径零变化)。
- 本 plan 的每一步都不 `commit`/不 `push`(仓库约定 no-auto-commit);
每个 Task 结束时工作区只留该 Task 的文件待审。
- 发版(Task 4 的 tag+push)由 maintainer 手动做,执行人不碰。
## Review Focus
- 探针包绝不能离开本机:dst 必须是自绑 socket 的地址——Task 2 的测试 pin 死。
- IPv6-only 设备:探针禁用记 warn,不 crash——Task 2 的测试 pin 死。
- IP 头 checksum 必须正确(内核静默丢坏包)——Task 2 的测试用解析器验。
- `probeReport` 为 nil(gost/旧调用方):调用前判空——Task 3 的测试 pin 死。
- 重拨时旧探针必须退出(per-iteration ctx),且双 writer 并发写 tun fd 是安全的
(datagram 原子,不加锁)——Task 3 接线时保持 `keepalive` 同款模式。
---
### Task 1: metadata 开关 + 探针计数器
**Files:**
- Modify: `x/handler/tun/metadata.go`
- Modify: `x/observer/stats/stats.go`
- Test: `x/handler/tun/metadata_test.go`(若无则新建;先 `ls` 确认)、
`x/observer/stats/stats_test.go`(追加)
**Interfaces:**
- Consumes: `mdutil.GetBool` 现有模式(`metadata.go:23,31`)。
- Produces: `metadata.probe bool`、`metadata.probeReport func(sentDelta, ackedDelta uint64)`;
`xstats.KindProbeSent stats.Kind = 101`、`xstats.KindProbeAcked stats.Kind = 102`
(101/102:core 用 1–5,x 内无其他自定义 Kind,测试用 999 避开);
`Stats.probeSent/probeAcked atomic.Uint64` + `Add`/`Get` 的 case
(Task 2/3 用 `h.options` 之外的 `stats.Stats` 句柄传参,签名见各 Task)。
- [ ] **Step 1: 写 failing test(metadata)**
```go
func TestParseMetadataProbeDefaultOff(t *testing.T) {
h := &tunHandler{}
if err := h.parseMetadata(mdx.NewMetadata(map[string]any{})); err != nil {
t.Fatal(err)
}
if h.md.probe {
t.Fatal("probe defaults off")
}
}
```
(`mdx` 是 `x/metadata` 下 metadata 构造的包名,先 grep 确认,`tun` 包内已有用法。)
- [ ] **Step 2: 运行,确认 FAIL(`h.md.probe` 未定义)**
Run: `go test ./handler/tun/ -run TestParseMetadataProbe -v`
Expected: 编译失败。
- [ ] **Step 3: 实现 `metadata` 加两个字段 + `parseMetadata` 解析**
`probe = mdutil.GetBool(md, "tun.probe", "probe")`;
`probeReport` 从 `md.Get("tun.probeReport")` 取不到则试 `md.Get("probeReport")`,
类型断言 `func(uint64, uint64)`,失败留 nil(不断言 panic)。
- [ ] **Step 4: 写 failing test(Stats 计数器)**
```go
func TestStatsProbeKinds(t *testing.T) {
s := NewStats(false)
s.Add(KindProbeSent, 1)
s.Add(KindProbeAcked, 1)
if s.Get(KindProbeSent) != 1 || s.Get(KindProbeAcked) != 1 {
t.Fatal("probe counters not tracked")
}
}
```
- [ ] **Step 5: 运行,确认 FAIL**
Run: `go test ./observer/stats/ -run TestStatsProbeKinds -v`
Expected: FAIL(未知 Kind 被 switch 吞掉,读回 0)。
- [ ] **Step 6: 实现 Kind 常量 + Stats 字段 + Add/Get case**
- [ ] **Step 7: 补 metadata 的 `probe:true` 解析测试 + report-func 透传测试**,
跑全包
Run: `go test -count=1 ./handler/tun/ ./observer/stats/`
Expected: PASS。
---
### Task 2: 探针组包 + 循环
**Files:**
- Create: `x/handler/tun/probe.go`
- Test: `x/handler/tun/probe_test.go`
**Interfaces:**
- Consumes: Task 1 的 `metadata.probe`(调用方判断,本文件不管开关)。
- Produces: `probeInterval = 30 * time.Second`、`probeRecvTimeout = 5 * time.Second`、
`probeMagic = "WSPROBE!"`(8 字节);
`func buildProbePacket(src net.IP, dstPort uint16, seq uint64) []byte`;
`func runDeviceProbe(ctx context.Context, dev io.Writer, ip net.IP, report func(sentDelta, ackedDelta uint64), log logger.Logger)`。
包格式(IPv4 头 20B + UDP 头 8B + magic 8B + seq 8B = 44B):
`src == dst == ip`(第一个 v4 spoke 地址),UDP checksum 置零(IPv4 下 0 = 无校验,
内核接受),IP 头 checksum 必须正确计算(标准 internet checksum,
全零和取反——这是唯一需要手写的算法,其他照抄 `ipv4.Header.Marshal` 或手排字节均可,
测试会验)。
循环语义:每 tick 发一次(`report(1,0)` 先记 sent,write 报错也记——acked 停滞
本身就是信号);`SetReadDeadline(now+5s)` 等 socket 回包,magic+seq 命中记
`report(0,1)`;超时/错包不记;`ctx.Done()` 退出。socket 为
`net.ListenUDP("udp4", &net.UDPAddr{IP: ip, Port: 0})`,`ip.To4()==nil` 时直接
返回(IPv6-only 禁用,调用方记 warn——调用方是 Task 3,见下)。
- [ ] **Step 1: 写 failing test(组包纯函数)**
```go
func TestBuildProbePacket(t *testing.T) {
src := net.ParseIP("10.10.100.250")
pkt := buildProbePacket(src, 54321, 7)
if len(pkt) != 44 { t.Fatalf("len=%d", len(pkt)) }
if pkt[0] != 0x45 || pkt[9] != 17 { t.Fatal("not v4/UDP") }
if !net.IP(pkt[12:16]).Equal(src) || !net.IP(pkt[16:20]).Equal(src) {
t.Fatal("src/dst must both be the spoke address")
}
// IP 头 checksum 自验;UDP checksum 必须为零;magic+seq 位置值正确。
}
```
- [ ] **Step 2: 运行,确认 FAIL(函数未定义)**
Run: `go test ./handler/tun/ -run TestBuildProbePacket -v`
Expected: 编译失败。
- [ ] **Step 3: 实现 `buildProbePacket`**
- [ ] **Step 4: 运行,确认 PASS**
- [ ] **Step 5: 写 failing test(循环:活设备推进 sent,acked 停滞可观测)**
```go
func TestRunDeviceProbeReportsSent(t *testing.T) {
device, kernelSide := net.Pipe() // 同 client_test.go 模式
defer device.Close()
defer kernelSide.Close()
var sent, acked uint64
ctx, cancel := context.WithCancel(context.Background())
defer cancel()
go runDeviceProbe(ctx, device, net.ParseIP("10.10.100.250"),
func(sd, ad uint64) { sent += sd; acked += ad }, xlogger.Nop())
// kernelSide 读到第一个包:断言 44B 且 magic 命中;此时 sent>=1 且 acked==0。
}
```
(`xlogger.Nop()` 用法同 `client_test.go:67`;`runDeviceProbe` 内 socket 绑
`10.10.100.250:0` 在容器里合法——无该地址也 bind 得上?`ListenUDP` 绑非本地
IP 会报错!测试改用 `127.0.0.1` 作 spoke IP,语义等价。)
- [ ] **Step 6: 运行,确认 FAIL**
- [ ] **Step 7: 实现 `runDeviceProbe`**
- [ ] **Step 8: 补测试——写坏的设备不 panic 且 acked 永远为 0**
```go
func TestRunDeviceProbeDeadDevice(t *testing.T) {
device, kernelSide := net.Pipe()
kernelSide.Close() // 写端坏掉:write 报错(备用:device.Close() 后写)
// 跑两个周期:report 的 sent 在涨,acked==0,全程无 panic。
}
```
- [ ] **Step 9: 跑全包**
Run: `go test -count=1 ./handler/tun/`
Expected: PASS(约 15s,内有既有 14s 的用例)。
---
### Task 3: 接入 handleClient
**Files:**
- Modify: `x/handler/tun/client.go`(`handleClient` 内 `go h.keepalive(...)` 旁,
约 62 行)
- Test: 无新增集成测试(`handleClient` 需要 `Router.Dial`,既有测试只到
`transportClient` 层,keepalive 同款无覆盖;覆盖由 Task 1 的解析测试 +
Task 2 的循环测试承担,此处只做接线)
**Interfaces:**
- Consumes: Task 1 的 `h.md.probe`/`h.md.probeReport`,Task 2 的 `runDeviceProbe`,
`handleClient` 已有的 `iterCtx`、`conn`(device)、`ips`。
- Produces: 无新导出(行为:`probe` 开时每轮 dial 起一个探针,`iterCtx` 结束即退)。
- [ ] **Step 1: 接线(幂等小改)**
```go
if h.md.probe {
go h.probeDevice(iterCtx, conn, ips)
}
```
其中 `probeDevice` 为 `tunHandler` 的小方法:取 `ips` 第一个 `To4()` 非空者,
nil report 判空(gost 路径),无 v4 时 `log.Warn` 后返回;其余直调
`runDeviceProbe`。`report` 透传 `h.md.probeReport`(nil-safe:调用前判空,
判空逻辑放在 `probeDevice` 一处,`runDeviceProbe` 假定非空——测试只测
`runDeviceProbe` 非空路径 + `probeDevice` 的空 report 不起循环)。
- [ ] **Step 2: 补 `probeDevice` 的两个小测试**(nil report 不启动;
全 v6 ips 直接返回),跑全包
Run: `go test -count=1 ./handler/tun/ ./observer/stats/`
Expected: PASS。
- [ ] **Step 3: `go vet` + `gofmt -l` 扫一遍改动文件**
---
### Task 4: 发版 x v0.23.0(maintainer 手动)
- [ ] **Step 1: 确认工作区只有本 plan 的文件**(`git status --short`)
- [ ] **Step 2: 全量构建 + 相关测试**
Run: `go build ./... && go test -count=1 ./handler/tun/ ./observer/stats/`
Expected: PASS。
- [ ] **Step 3: 打 tag 并推**(minor:新功能;patch 不够格)
```bash
git tag -a v0.23.0 -m "v0.23.0: tun device write-side probe (metadata probe, probe counters)"
git push origin v0.23.0
```
- [ ] **Step 4: 通知 Plan B 可开始**(bump 到 v0.23.0)。
@@ -0,0 +1,217 @@
# Router Handler 重构设计
## 概述
对 `x/handler/router/` 包进行重构和全面单元测试。该包实现了一个 UDP/TUN 路由处理器,通过 Relay 协议的 `CmdAssociate` 接收客户端连接,将虚拟 IP 数据包路由到远程连接器。
**原则**:仅重构现有代码结构 + 补全测试,不新增功能,不改变外部行为。
## 当前状态
- 5 个源文件,1056 行代码,零测试
- 核心数据结构:`Router`、`Connector`、`ConnectorPool`
- 核心处理逻辑:`handleAssociate`(UDP 关联)、`handlePacket`(IP 包路由)、`handleEntrypoint`(UDP 入口)
- 生命周期:`Init` → `Handle` → `Close`
## 重构后文件布局
```
handler/router/
│
├── router.go # Router / Connector / ConnectorPool — 纯数据结构
├── handler.go # routerHandler 结构体, NewHandler, Init, Handle, Close
├── associate.go # handleAssociate 主流程 + handlePacket + getRoute + getAddrforRoute + sdRenew
├── entrypoint.go # handleEntrypoint UDP 入口
├── metadata.go # metadata struct, parseMetadata, 常量
├── conn.go # packetConn(带长度前缀的 Read/Write)+ lockWriter
├── observe.go # observeStats + checkRateLimit
│
├── helpers_test.go # 所有测试辅助类型
├── router_test.go # Router / Connector / ConnectorPool 测试
├── handler_test.go # NewHandler, Init, Handle, Close 测试
├── associate_test.go # handleAssociate + handlePacket + getRoute + getAddrforRoute + sdRenew 测试
├── entrypoint_test.go # handleEntrypoint 测试
├── metadata_test.go # parseMetadata 测试
├── conn_test.go # packetConn + lockWriter 测试
└── observe_test.go # observeStats + checkRateLimit 测试
```
### 提取操作(纯移动,不修改逻辑)
| 操作 | 源文件 | 目标文件 | 内容 |
|------|--------|----------|------|
| 提取 | `associate.go` | `conn.go` | `packetConn`、`lockWriter`、`LockWriter` |
| 提取 | `handler.go` | `observe.go` | `observeStats`、`checkRateLimit` |
### 需修复的 Bug
1. **`router.go:DelConnector` — 匹配后缺少 `break`**:找到匹配的 connector 后未跳出循环,可能误删后续元素
## 测试策略
### Mock 辅助类型(`helpers_test.go`)
| 类型 | 实现 | 用途 |
|------|------|------|
| `testLogger` | 所有方法空实现 | 测试日志 |
| `testMD` | 包装 map → Metadata | 构建 metadata |
| `fakeConn` | bytes.Buffer 后端的 net.Conn | Handler 输入 |
| `pipeConn` | io.Pipe 后端的双向 net.Conn | 集成测试双向通信 |
| `fakePacketConn` | channel 后端的 net.PacketConn | Entrypoint 测试 |
| `fakeObserver` | channel 后端的 observer.Observer | observeStats 测试 |
| `mockAuther` | 函数注入的 auth.Authenticator | 认证测试 |
| `mockRateLimiterContainer` | 函数注入的 rate.RateLimiter | checkRateLimit 测试 |
| `mockTrafficLimiter` | 空实现的 traffic.TrafficLimiter | Init 测试 |
| `mockRouter` | 函数注入的 router.Router | getRoute 测试 |
| `mockSD` | 函数注入的 sd.SD | SD 注册/查找测试 |
| `mockIngress` | 函数注入的 ingress.Ingress | handleAssociate 路由验证 |
| `buildRelayAssociateRequest` | 序列化 relay.Request | 构造测试输入 |
| `readRelayResponse` | 反序列化 relay.Response | 验证输出 |
### 测试文件
#### `metadata_test.go` — parseMetadata 全面测试
- 空 metadata
- 仅 readTimeout
- bufferSize 默认值 / 自定义值
- entryPoint 设置
- ingress 注册查找
- sd / sd.cache.expiration / sd.renewInterval 默认值
- router / router.cache / router.cache.expiration 默认值
- observePeriod 默认值 / 最小值限制
- observer.resetTraffic
- limiter.refreshInterval / limiter.cleanupInterval
#### `router_test.go` — Router / Connector / ConnectorPool
**Connector:**
- NewConnector(nil opts、完整 opts)
- ID / Writer / Close(nil-safe)
- Close 重复调用
**Router:**
- NewRouter
- AddConnector(nil 跳过、多元素添加)
- GetConnector(单元素、多元素、权重选择、MaxWeight 优先级、空 host)
- DelConnector(精确删除、不存在 host、不存在 cid、匹配后 break 验证)
- Close(重复调用安全、connector 清理)
**ConnectorPool:**
- NewConnectorPool
- Add(新 Router 创建、已有 Router 追加)
- Get(nil-safe、存在/不存在 rid、存在/不存在 host)
- Del(nil-safe、精确删除)
- Close(nil-safe、重复调用)
**parseRouterID:**
- 空字符串
- 有效 UUID
- 无效 UUID
#### `conn_test.go` — packetConn + lockWriter
**packetConn:**
- Read 正常(2 字节长度前缀 + 数据)
- Read 空数据(长度 0)
- Read 缓冲区小于数据(截断)
- Write 正常
- Write 超过 MaxUint16(错误)
- Write 后 Read 往返
**lockWriter:**
- Write 正常
- 并发写入安全性
- Close(io.Closer 包装/非 Closer 包装)
- Close 重复调用
#### `observe_test.go` — observeStats + checkRateLimit
**checkRateLimit:**
- RateLimiter 为 nil(返回 true)
- Limiter Allow 返回 true
- Limiter Allow 返回 false
**observeStats:**
- Observer 为 nil(立即返回)
- 正常周期事件发送
- Observer.Observe 返回 error(重试机制)
- 重试成功后清空 events
- context cancel 退出
- observerResetTraffic 设置
#### `handler_test.go` — NewHandler, Init, Handle, Close
**NewHandler:**
- 最小构造
- 带 Options 构造
**Init:**
- 最小 Init(验证 id、pool、cancel 初始化)
- 带 Observer(验证 stats 初始化、goroutine 启动)
- 带 Limiter
- entrypoint Init 错误
- parseMetadata 错误
**Handle:**
- 请求读取错误
- Bad Version
- Unknown Command
- Rate Limit 拒绝
- 认证成功/失败
- 正常 Associate 流程(发送 relay request → 读取 response → 发送 packet)
- 多 Feature 请求(auth + addr + tunnel + network)
**Close:**
- 正常关闭
- 重复关闭
- entrypoint 未初始化
#### `associate_test.go` — handleAssociate + handlePacket + getRoute + getAddrforRoute + sdRenew
**handleAssociate:**
- 正常关联流程(ingress 验证通过 → connector 注册 → packet 循环)
- ingress 验证失败(host 不匹配 → StatusHostUnreachable)
- ingress 为 nil(跳过验证)
- connectorID 生成
- SD 注册/注销
- connector 在 defer 中清理
**handlePacket:**
- IPv4 包解析 + 路由查找 + connector 转发
- IPv6 包解析 + 路由查找 + connector 转发
- 未知包类型(错误返回)
- 无路由(no route to host 错误)
- connector 不存在时通过 entrypoint 转发
- SD 查找 + entrypoint 写入
**getRoute:**
- 缓存命中(未过期)
- 缓存未命中 → registry 查找
- registry 未找到 → fallback router
- 缓存未启用
**getAddrforRoute:**
- SD 缓存命中
- SD 缓存未命中 → 查找
- 自身节点跳过
- SD 为 nil
**sdRenew:**
- 正常 tick 续期
- context cancel 退出
#### `entrypoint_test.go` — handleEntrypoint
- 正常 relay request 转发到 connector
- 非 Associate cmd 跳过
- ReadFrom error 返回
- 空 connector(无匹配 connector 时忽略)
- connector Writer 写入错误(仅日志记录)
## 测试基础设施
- 使用 `testing/quick` 或手动构造 relay 协议字节流
- 使用 `io.Pipe` 模拟双向网络连接
- 使用 channel 同步异步 goroutine(observeStats、sdRenew)
- 所有测试运行 `go test -race` 验证无竞态
@@ -0,0 +1,281 @@
# Tunnel Handler Multiplexing Protocol Research
**Date:** 2026-06-08
**Scope:** Investigate alternatives to SMUX for the tunnel handler's stream multiplexing, and assess feasibility of making the mux layer pluggable.
## Current State
- **Package:** `internal/util/mux/mux.go`
- **Library:** `github.com/xtaci/smux` v1.5.31
- **Default protocol version:** v1 (no per-stream flow control)
- **Consumers:** tunnel handler (bind/connect), relay connector, SOCKS5 connector, SSH dialer
- **Session surface:** `GetConn()`, `Accept()`, `Close()`, `IsClosed()`, `NumStreams()`
---
## 1. Protocol Comparison
### 1.1 SMUX v1 (current default)
- **Author:** xtaci (same as kcp-go)
- **Wire header:** 8 bytes (VER + CMD + LEN + SID)
- **Flow control:** None — sender pushes data without receiver feedback
- **Memory model:** Session-wide shared receive buffer; bounded overall memory
- **Keepalive:** NOP frames, configurable interval; known stability issues ([Issue #67](https://github.com/xtaci/smux/issues/67))
- **Throughput:** ~2247 MB/s single-stream (benchmark below)
- **Overhead vs raw TCP:** ~2.3x (5124 MB/s raw → 2247 MB/s muxed)
- **Allocations:** 1208 B/op, 19 allocs/op per stream operation
**Verdict:** Minimal overhead, but v1 lacks flow control — a single fast sender can starve other streams.
### 1.2 SMUX v2
- **Wire header:** Same 8 bytes, adds `cmdUPD` (window update)
- **Flow control:** Per-stream sliding window via `cmdUPD` frames — receiver sends consumed bytes + available window
- **Memory model:** Same session-wide buffer as v1, plus per-stream window tracking
- **Backpressure:** Stream-level — sender stalls when window is exhausted
- **Upgrade path:** Zero code change — just set `Version: 2` in config
**Verdict:** Strictly better than v1 for multi-stream workloads. Fair bandwidth distribution without sacrificing the session-wide memory model. **Recommended as immediate upgrade.**
### 1.3 Yamux (hashicorp/yamux)
- **Author:** HashiCorp; also forked by libp2p as go-yamux/v2
- **Wire header:** 12 bytes (Version + Type + Flags + StreamID + Length)
- **Flow control:** Window-based per-stream; default 256 KB window
- **Memory model:** Per-stream buffer (not shared)
- **Throughput:** ~24-28 Mbps per stream ([Issue #24](https://github.com/hashicorp/yamux/issues/24)) — limited by conservative window updates falling back to "safe speed"
- **Stream limit:** Hardcoded upper bound ([Issue #7](https://github.com/hashicorp/yamux/issues/7))
- **Backpressure:** On data streams only, NOT on stream open operations
- **API:** Implements `net.Conn` / `net.Listener` natively
- **Maintenance:** Low activity / stable; updated to Go 1.23
- **Ecosystem:** Used in HashiCorp products (Consul, Nomad), libp2p; sing-box defaults to yamux
**Verdict:** Wider ecosystem adoption than SMUX, but throughput-limited by window flow control. SMUX v2 has comparable flow control with better performance for proxy/tunnel workloads. Not a compelling upgrade over SMUX v2, but worth offering as a pluggable alternative.
### 1.4 QUIC Streams (quic-go)
- **Package:** `github.com/quic-go/quic-go` v0.59.1 (already a GOST dependency)
- **Multiplexing:** Native — streams are a first-class protocol primitive
- **Wire overhead:** QUIC frames (variable, typically 2-8 bytes per frame header)
- **Flow control:** Per-stream AND per-connection, protocol-level enforcement
- **Head-of-line blocking:** Eliminated — each stream is independent
- **Encryption:** Mandatory TLS 1.3 (built-in)
- **Connection setup:** 0-RTT possible; eliminates TCP+TLS handshake latency
- **Transport:** UDP (userspace)
- **CPU overhead:** 2x-4x higher per byte vs raw TCP ([Fastly benchmark](https://www.fastly.com/blog/measuring-quic-vs-tcp-computational-efficiency), [HN discussion](https://news.ycombinator.com/item?id=19476439))
- **Loss performance:** Significantly better than TCP at >5% packet loss
- **Equivalent to:** `kcp-go + smux` per [Turbo Tunnel evaluation](https://github.com/net4people/bbs/issues/14)
**Verdict:** Strongest technical alternative for new deployments, but NOT a mux-layer concern. QUIC multiplexes at the transport layer and requires UDP. GOST already handles QUIC at the listener/dialer level (`listener/http3/`, `dialer/http3/`). Not a candidate for the `internal/util/mux/` plugin interface.
### 1.5 HTTP/2 Streams (golang.org/x/net/http2)
- **Multiplexing:** Full HTTP/2 stream multiplexing with HPACK header compression
- **Wire overhead:** 9-byte frame header + HPACK-encoded headers per stream
- **Flow control:** Per-stream and per-connection
- **Overhead:** HTTP/2 framing + headers add significant overhead vs raw mux protocols
- **Use case:** sing-box uses this for `h2mux` mode
**Verdict:** Overkill for raw TCP proxy/tunneling. The HTTP/2 framing and header layers add unnecessary overhead. Better suited for HTTP-based proxying where header semantics are needed.
### 1.6 Mux.Cool (Xray/V2Ray)
- **Purpose:** Reduce TCP handshake latency for short-lived proxy connections
- **Throughput:** Officially documented as NOT designed for high throughput
- **Integration:** Tightly coupled to Xray/V2Ray proxy ecosystem
**Verdict:** Not a standalone mux library. Purpose-built for Xray ecosystem. Not applicable.
### 1.7 alecthomas/multiplex
- **Interface:** Multiplexes over any `io.ReadWriteCloser`
- **Approach:** Fragmentation-based fairness across channels
- **Maintenance:** Minimal activity
**Verdict:** General-purpose but not optimized for proxy/tunnel workloads. Insufficient battle-testing in high-throughput scenarios.
---
## 2. Benchmarks
### SMUX (from upstream README, macOS/amd64)
```
BenchmarkMSB-4 30000000 51.8 ns/op
BenchmarkAcceptClose-4 50000 36783 ns/op
BenchmarkConnSmux-4 30000 58335 ns/op 2246.88 MB/s 1208 B/op 19 allocs/op
BenchmarkConnTCP-4 50000 25579 ns/op 5124.04 MB/s 0 B/op 0 allocs/op
```
**Key finding:** SMUX achieves ~44% of raw TCP throughput (2247 vs 5124 MB/s). The overhead comes from frame allocation (~1208 B/op, 19 allocs/op) and the session-level coordination.
### Yamux (from [Issue #24](https://github.com/hashicorp/yamux/issues/24))
- ~24-28 Mbps per stream with 2 concurrent streams
- Limited by window flow control falling back to "safe speed"
- Per-stream window: 256 KB (configurable via `MaxStreamWindowSize`)
**Comparison:** SMUX achieves ~18 Gbps (2247 MB/s) single-stream on the same benchmark hardware. Yamux's ~24-28 Mbps is orders of magnitude lower. The difference is primarily due to Yamux's conservative window-based flow control vs SMUX's token-bucket approach. However, the Yamux benchmark is from a real-world multi-stream scenario, while SMUX's is single-stream — not directly comparable.
### QUIC (from [QUIC tunneling blog](https://mranv.pages.dev/posts/quic-tunneling-go-implementation/), [arXiv paper](https://arxiv.org/html/2504.10054v2))
- ~66% lower latency vs TCP-based tunneling
- ~70% higher throughput vs TCP in lossy/unstable networks
- CPU overhead: 2x-4x higher vs raw TCP (userspace UDP processing)
- At <1% packet loss: performance roughly equivalent to TCP
- At >5% packet loss: QUIC significantly outperforms TCP
### Summary Table
| Protocol | Single-stream throughput | Multi-stream fairness | CPU overhead | HOL blocking | Encryption | Transport |
|----------|------------------------|----------------------|-------------|-------------|------------|-----------|
| SMUX v1 | ~2247 MB/s (44% TCP) | Poor (no flow control) | ~2.3x TCP | Present | None | TCP |
| SMUX v2 | ~2247 MB/s (44% TCP) | Good (per-stream window) | ~2.3x TCP | Present | None | TCP |
| Yamux | ~24-28 Mbps/stream | Fair (window-based) | ~2-3x TCP | Present | None | TCP |
| QUIC | Better at >5% loss | Excellent (native) | 2x-4x TCP | Eliminated | TLS 1.3 | UDP |
| HTTP/2 | Good | Good (HTTP/2 flow ctrl) | Moderate | Present | TLS | TCP |
---
## 3. Config Integration: Making Mux Pluggable
### Current Architecture
The mux layer is a thin wrapper in `internal/util/mux/mux.go`:
- `Config` struct — SMUX-specific parameters
- `ClientSession(conn, cfg)` / `ServerSession(conn, cfg)` — factory functions
- `Session` struct — concrete type wrapping `*smux.Session`
- `streamConn` — wraps `*smux.Stream` as `net.Conn`
All consumers use `*mux.Session` (pointer to concrete struct):
- `handler/tunnel/bind.go`, `handler/tunnel/connect.go`
- `handler/relay/entrypoint.go`
- `connector/socks/v5/bind.go`, `connector/relay/bind.go`
- `dialer/ssh/dialer.go`, `dialer/sshd/dialer.go`
### Recommended Approach: Interface Extraction
Convert `mux.Session` from a concrete struct to an interface, keeping everything in `internal/util/mux/`:
```go
// Session is the interface for a multiplexed connection session.
type Session interface {
GetConn() (net.Conn, error) // Open stream (client)
Accept() (net.Conn, error) // Accept stream (server)
Close() error
IsClosed() bool
NumStreams() int
}
```
Add `MuxType` to `Config`:
```go
type Config struct {
MuxType string // "smux" (default), "yamux"
// SMUX fields (existing)
Version int
// ...
// Yamux fields (new)
MaxStreamWindowSize uint32
}
```
Factory functions dispatch on `MuxType`:
```go
func ClientSession(conn net.Conn, cfg *Config) (Session, error) {
switch muxType(cfg) {
case "yamux":
return newYamuxClientSession(conn, cfg)
default:
return newSMUXClientSession(conn, cfg)
}
}
```
Consumer changes: replace `*mux.Session` → `mux.Session` (interface value). Minimal mechanical change.
### YAML Config Surface
The mux config flows through metadata. Today it's parsed in `handler/tunnel/metadata.go`:
```go
// Current: tunnel handler metadata
type metadata struct {
muxCfg *mux.Config
// ...
}
```
With the pluggable design, users configure it via the service metadata:
```yaml
services:
- name: tunnel-server
handler:
type: tunnel
metadata:
mux.type: yamux # "smux" | "yamux" (default: "smux")
mux.version: 2 # SMUX protocol version (only for smux)
mux.maxFrameSize: 32768
mux.maxStreamWindow: 524288 # Yamux window size (only for yamux)
```
The existing `mdutil.GetString(md, "mux.type", "muxType")` pattern works naturally.
### What About QUIC?
QUIC is fundamentally different — it multiplexes at the transport layer, not over an existing TCP connection. It cannot be a drop-in alternative for the `internal/util/mux/` interface because:
1. QUIC requires UDP transport, not TCP
2. QUIC connections are created differently (not by upgrading an existing `net.Conn`)
3. QUIC streams already exist as first-class objects
QUIC-based multiplexing is better addressed as a separate listener/dialer pair (which GOST already has: `listener/http3/`, `dialer/http3/`, etc.), not as a mux plugin.
### Backward Compatibility
- Default `MuxType: ""` → `"smux"` — existing configs unchanged
- Default SMUX version stays at v1 (no behavior change for existing deployments)
- Users opt into v2 or yamux explicitly via metadata
### Candidates Worth Implementing
| Priority | Protocol | Rationale |
|----------|----------|-----------|
| 1 (immediate) | SMUX v2 | Zero-dependency upgrade, per-stream flow control, just change default version |
| 2 (high value) | Yamux | Broad ecosystem adoption (sing-box defaults to it), well-tested, different trade-off profile |
| 3 (future) | None of the others | QUIC is already handled at the listener/dialer level; HTTP/2 and Mux.Cool add unnecessary complexity |
---
## 4. Conclusions
1. **SMUX v2 should be the default.** Per-stream sliding window flow control is strictly better than v1 for multi-stream tunnel workloads. Zero code risk — same library, just `Version: 2`.
2. **Yamux is the most viable alternative** for users who want different flow control characteristics (window-based vs token-bucket). It's well-tested in production at HashiCorp and libp2p. The throughput concerns (24-28 Mbps/stream) may not apply to GOST's typical workload where streams are I/O-bound rather than CPU-bound.
3. **Making mux pluggable is straightforward.** The `Session` interface is only 5 methods. All 8 consumers just change `*mux.Session` → `mux.Session`. Config flows through existing metadata parsing. No new `core/` package or registry needed.
4. **QUIC is not a mux-layer concern.** It's already handled at the listener/dialer level in GOST. QUIC's native stream multiplexing is a transport-level feature, not something to bolt onto TCP connections.
5. **SMUX's keepalive issue ([#67](https://github.com/xtaci/smux/issues/67)) should be tracked.** If it causes production problems, Yamux becomes a more attractive fallback.
---
## Sources
- [xtaci/smux GitHub](https://github.com/xtaci/smux)
- [hashicorp/yamux GitHub](https://github.com/hashicorp/yamux)
- [Yamux Performance Issue #24](https://github.com/hashicorp/yamux/issues/24)
- [SMUX Keepalive Issue #67](https://github.com/xtaci/smux/issues/67)
- [quic-go Streams Docs](https://quic-go.net/docs/quic/streams/)
- [QUIC vs TCP (arXiv)](https://arxiv.org/html/2504.10054v2)
- [Turbo Tunnel Protocol Evaluation](https://github.com/net4people/bbs/issues/14)
- [QUIC Tunneling in Go](https://mranv.pages.dev/posts/quic-tunneling-go-implementation/)
- [Fastly: QUIC vs TCP Efficiency](https://www.fastly.com/blog/measuring-quic-vs-tcp-computational-efficiency)
- [sing-box Mux Config](https://sing-box.sagernet.org/configuration/shared/multiplex/)
+28 -36
View File
@@ -4,20 +4,20 @@ go 1.26.3
require (
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137
github.com/andybalholm/brotli v1.0.6
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d
github.com/andybalholm/brotli v1.2.2
github.com/gin-contrib/cors v1.7.2
github.com/gin-gonic/gin v1.12.0
github.com/go-gost/core v0.6.0
github.com/go-gost/go-shadowsocks2 v0.1.3
github.com/go-gost/core v0.6.1
github.com/go-gost/go-shadowsocks2 v0.1.4
github.com/go-gost/gosocks4 v0.1.0
github.com/go-gost/gosocks5 v0.5.0
github.com/go-gost/plugin v0.5.0
github.com/go-gost/plugin v0.8.1
github.com/go-gost/quic-dissector v0.1.0
github.com/go-gost/relay v0.7.0
github.com/go-gost/tls-dissector v0.3.1
github.com/go-redis/redis/v8 v8.11.5
github.com/gobwas/glob v0.2.3
github.com/goccy/go-yaml v1.19.2
github.com/golang/snappy v0.0.4
github.com/google/shlex v0.0.0-20191202100458-e7afc7fbc510
github.com/google/uuid v1.6.0
@@ -29,12 +29,11 @@ require (
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pion/dtls/v3 v3.1.4
github.com/pires/go-proxyproto v0.8.1
github.com/prometheus/client_golang v1.19.1
github.com/prometheus/client_golang v1.21.1
github.com/quic-go/quic-go v0.60.0
github.com/quic-go/webtransport-go v0.11.1
github.com/refraction-networking/utls v1.8.2
github.com/rs/xid v1.3.0
github.com/shadowsocks/go-shadowsocks2 v0.1.5
github.com/shirou/gopsutil/v3 v3.24.5
github.com/songgao/water v0.0.0-20200317203138-2b4b6d7c09d8
github.com/spf13/viper v1.19.0
@@ -43,41 +42,39 @@ require (
github.com/tidwall/sjson v1.2.5
github.com/vishvananda/netlink v1.1.1-0.20211118161826-650dca95af54
github.com/vishvananda/netns v0.0.4
github.com/vulcand/predicate v1.2.0
github.com/xjasonlyu/tun2socks/v2 v2.6.0
github.com/xtaci/kcp-go/v5 v5.6.5
github.com/xtaci/smux v1.5.31
github.com/xtaci/kcp-go/v5 v5.6.72
github.com/xtaci/smux v1.5.57
github.com/xtaci/tcpraw v1.2.25
github.com/yl2chen/cidranger v1.0.2
github.com/zalando/go-keyring v0.2.4
golang.org/x/crypto v0.54.0
golang.org/x/crypto v0.55.0
golang.org/x/exp v0.0.0-20241210194714-1829a127f884
golang.org/x/net v0.57.0
golang.org/x/net v0.58.0
golang.org/x/sys v0.47.0
golang.org/x/text v0.40.0
golang.org/x/time v0.12.0
golang.org/x/text v0.41.0
golang.org/x/time v0.14.0
golang.zx2c4.com/wireguard v0.0.0-20250521234502-f333402bd9cb
google.golang.org/grpc v1.82.1
google.golang.org/grpc v1.83.2
google.golang.org/protobuf v1.36.11
gopkg.in/natefinch/lumberjack.v2 v2.2.1
gopkg.in/yaml.v3 v3.0.1
gvisor.dev/gvisor v0.0.0-20250523182742-eede7a881b20
)
require (
github.com/bytedance/gopkg v0.1.3 // indirect
github.com/go-ole/go-ole v1.2.6 // indirect
github.com/goccy/go-yaml v1.19.2 // indirect
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 // indirect
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 // indirect
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c // indirect
github.com/shoenig/go-m1cpu v0.1.6 // indirect
github.com/sirupsen/logrus v1.9.3 // indirect
github.com/tidwall/match v1.1.1 // indirect
github.com/tidwall/pretty v1.2.0 // indirect
github.com/tklauser/go-sysconf v0.3.12 // indirect
github.com/tklauser/numcpus v0.6.1 // indirect
github.com/tklauser/go-sysconf v0.3.16 // indirect
github.com/tklauser/numcpus v0.11.0 // indirect
github.com/yusufpapurcu/wmi v1.2.4 // indirect
go.mongodb.org/mongo-driver/v2 v2.5.0 // indirect
go.mongodb.org/mongo-driver/v2 v2.7.0 // indirect
gopkg.in/yaml.v3 v3.0.1 // indirect
)
require (
@@ -102,27 +99,25 @@ require (
github.com/godbus/dbus/v5 v5.1.0 // indirect
github.com/google/btree v1.1.3 // indirect
github.com/google/gopacket v1.1.19 // indirect
github.com/gravitational/trace v1.1.16-0.20220114165159-14a9a7dd6aaf // indirect
github.com/hashicorp/hcl v1.0.0 // indirect
github.com/jonboulle/clockwork v0.2.2 // indirect
github.com/json-iterator/go v1.1.12 // indirect
github.com/klauspost/cpuid/v2 v2.3.0 // indirect
github.com/klauspost/reedsolomon v1.11.8 // indirect
github.com/klauspost/cpuid/v2 v2.4.0 // indirect
github.com/klauspost/reedsolomon v1.12.0 // indirect
github.com/leodido/go-urn v1.4.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/magiconair/properties v1.8.10 // indirect
github.com/mattn/go-isatty v0.0.22 // indirect
github.com/mitchellh/mapstructure v1.5.0
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/onsi/gomega v1.34.2 // indirect
github.com/pelletier/go-toml/v2 v2.2.4 // indirect
github.com/pelletier/go-toml/v2 v2.4.2 // indirect
github.com/pion/logging v0.2.4 // indirect
github.com/pion/transport/v4 v4.0.1 // indirect
github.com/pkg/errors v0.9.1 // indirect
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 // indirect
github.com/prometheus/client_model v0.6.0 // indirect
github.com/prometheus/common v0.48.0 // indirect
github.com/prometheus/procfs v0.12.0 // indirect
github.com/prometheus/client_model v0.6.2 // indirect
github.com/prometheus/common v0.62.0 // indirect
github.com/prometheus/procfs v0.15.1 // indirect
github.com/quic-go/qpack v0.6.0 // indirect
github.com/riobard/go-bloom v0.0.0-20200614022211-cdc8013cb5b3 // indirect
github.com/sagikazarmark/locafero v0.4.0 // indirect
@@ -132,18 +127,15 @@ require (
github.com/spf13/cast v1.6.0 // indirect
github.com/spf13/pflag v1.0.5 // indirect
github.com/subosito/gotenv v1.6.0 // indirect
github.com/templexxx/cpu v0.1.1 // indirect
github.com/templexxx/xorsimd v0.4.3 // indirect
github.com/tjfoc/gmsm v1.4.1 // indirect
github.com/twitchyliquid64/golang-asm v0.15.1 // indirect
github.com/ugorji/go/codec v1.3.1 // indirect
github.com/zeebo/blake3 v0.2.4 // indirect
go.uber.org/multierr v1.11.0 // indirect
golang.org/x/arch v0.22.0 // indirect
golang.org/x/mod v0.37.0 // indirect
golang.org/x/mod v0.38.0 // indirect
golang.org/x/sync v0.22.0
golang.org/x/term v0.45.0 // indirect
golang.org/x/tools v0.47.0 // indirect
golang.org/x/tools v0.48.0 // indirect
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 // indirect
google.golang.org/genproto/googleapis/rpc v0.0.0-20260723215102-3fe39f3c1018 // indirect
gopkg.in/ini.v1 v1.67.0 // indirect
+68 -91
View File
@@ -4,10 +4,8 @@ github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137 h1:s6gZFSlWYmbqAu
github.com/alecthomas/units v0.0.0-20211218093645-b94a6e3cc137/go.mod h1:OMCwj8VM1Kc9e19TLln2VL61YJF0x1XFtfdL4JdbSyE=
github.com/alessio/shellescape v1.4.1 h1:V7yhSDDn8LP4lc4jS8pFkt0zCnzVJlG5JXy9BVKJUX0=
github.com/alessio/shellescape v1.4.1/go.mod h1:PZAiSCk0LJaZkiCSkPv8qIobYglO3FPpyFjDCtHLS30=
github.com/andybalholm/brotli v1.0.6 h1:Yf9fFpf49Zrxb9NlQaluyE92/+X7UVHlhMNJN2sxfOI=
github.com/andybalholm/brotli v1.0.6/go.mod h1:fO7iG3H7G2nSZ7m0zPUDn85XEX2GTukHGRSepvi9Eig=
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d h1:Byv0BzEl3/e6D5CLfI0j/7hiIEtvGVFPCZ7Ei2oq8iQ=
github.com/asaskevich/govalidator v0.0.0-20210307081110-f21760c49a8d/go.mod h1:WaHUgvxTVq04UNunO+XhnAqY/wQc+bxr74GqbsZ/Jqw=
github.com/andybalholm/brotli v1.2.2 h1:HzTuoo2ErYQqf5qvcJInB8uvqSVxRttzkFexPWtnceM=
github.com/andybalholm/brotli v1.2.2/go.mod h1:rzTDkvFWvIrjDXZHkuS16NPggd91W3kUSvPlQ1pLaKY=
github.com/beorn7/perks v1.0.1 h1:VlbKKnNfV8bJzeqoa4cOKqO6bYr3WgKZxO8Z16+hsOM=
github.com/beorn7/perks v1.0.1/go.mod h1:G2ZrVWU2WbWT9wwq4/hrbKbnv/1ERSJQ0ibhJ6rlkpw=
github.com/bytedance/gopkg v0.1.3 h1:TPBSwH8RsouGCBcMBktLt1AymVo2TVsBVCY4b6TnZ/M=
@@ -36,7 +34,6 @@ github.com/dgryski/go-rendezvous v0.0.0-20200823014737-9f7001d12a5f/go.mod h1:cu
github.com/dunglas/httpsfv v1.1.0 h1:Jw76nAyKWKZKFrpMMcL76y35tOpYHqQPzHQiwDvpe54=
github.com/dunglas/httpsfv v1.1.0/go.mod h1:zID2mqw9mFsnt7YC3vYQ9/cjq30q41W+1AnDwH8TiMg=
github.com/envoyproxy/go-control-plane v0.9.0/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.1-0.20191026205805-5f8ba28d4473/go.mod h1:YTl/9mNaCwkRvm6d1a2C3ymFceY/DCBVvsKhRF0iEA4=
github.com/envoyproxy/go-control-plane v0.9.4/go.mod h1:6rpuAdCZL397s3pYoYcLgu1mIlRU8Am5FuJP05cCM98=
github.com/envoyproxy/protoc-gen-validate v0.1.0/go.mod h1:iSmxcyjqTsJpI2R4NaDN7+kN2VEUnK/pcBlmesArF7c=
github.com/frankban/quicktest v1.14.6 h1:7Xjx+VpznH+oBnejlPUj8oUpdxnVs4f8XU8WnHkI4W8=
@@ -51,16 +48,18 @@ github.com/gin-contrib/sse v1.1.0 h1:n0w2GMuUpWDVp7qSpvze6fAu9iRxJY4Hmj6AmBOU05w
github.com/gin-contrib/sse v1.1.0/go.mod h1:hxRZ5gVpWMT7Z0B0gSNYqqsSCNIJMjzvm6fqCz9vjwM=
github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8=
github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc=
github.com/go-gost/core v0.6.0 h1:Y7jG5bI0QxdzDlMOyEtAYxCz+0GxAHKmaei667pRPWc=
github.com/go-gost/core v0.6.0/go.mod h1:WGI43jOka7FAsSAwi/fSMaqxdR+E339ycb4NBGlFr6A=
github.com/go-gost/go-shadowsocks2 v0.1.3 h1:6CUZLp+mTWXnKP2aK8/Z9ZP+ERMX9gSbywmPu4kGX/A=
github.com/go-gost/go-shadowsocks2 v0.1.3/go.mod h1:866zFNNI3He6Wef1M/IvAjTal74WhcfKfBgRpTlkKys=
github.com/go-gost/core v0.6.1 h1:mBBvZpxIbrspuRsksj7YLHEiBGEc+sQBV3wDs4rX/AE=
github.com/go-gost/core v0.6.1/go.mod h1:WGI43jOka7FAsSAwi/fSMaqxdR+E339ycb4NBGlFr6A=
github.com/go-gost/go-shadowsocks2 v0.1.4 h1:n2Po4TDKdLp1PsvhSiWFB/6S4e/YKZfsKJkA0PUa168=
github.com/go-gost/go-shadowsocks2 v0.1.4/go.mod h1:866zFNNI3He6Wef1M/IvAjTal74WhcfKfBgRpTlkKys=
github.com/go-gost/gosocks4 v0.1.0 h1:eAzev6qw4fzkFQKC9uCHLVNnnPdHyqCggbnfNN80Pmk=
github.com/go-gost/gosocks4 v0.1.0/go.mod h1:hzVjwijJuZR1pp3GqpTj+AKcSGrx68RlWTrQMFMYBP0=
github.com/go-gost/gosocks5 v0.5.0 h1:YE37l1MJwde8diIQdynStqogMotG5enoTdborhA5yic=
github.com/go-gost/gosocks5 v0.5.0/go.mod h1:1G6I7HP7VFVxveGkoK8mnprnJqSqJjdcASKsdUn4Pp4=
github.com/go-gost/plugin v0.5.0 h1:zK3Ezcv+z6bz6aUhaFZcZiJc42w44ohybzrj5tnWzcM=
github.com/go-gost/plugin v0.5.0/go.mod h1:oN23l+yGDCIP9G3KnDl/I/0zVGOobZUDCB2Z5yYYXts=
github.com/go-gost/plugin v0.8.0 h1:KL4vLck/yqY3+TSm6RMc07IuZe9nJIVzoGH0/or0ei4=
github.com/go-gost/plugin v0.8.0/go.mod h1:48pqi8/zS5OhEEoFETYZCqu2sR59DX3QxG5SjGmPWcU=
github.com/go-gost/plugin v0.8.1 h1:mZpLbzRZosHocaiZ4dYYqX8waLiavy/rsEWwqx5aluo=
github.com/go-gost/plugin v0.8.1/go.mod h1:48pqi8/zS5OhEEoFETYZCqu2sR59DX3QxG5SjGmPWcU=
github.com/go-gost/quic-dissector v0.1.0 h1:zOaw2XjKxMf6vVC1z4BHDKGHbCs4zrYU1Rxaz5d0TQU=
github.com/go-gost/quic-dissector v0.1.0/go.mod h1:ar+I40Izq9ZT2k/aNnLFGEMvdeSOBLLKxhF/i3FSnQQ=
github.com/go-gost/relay v0.7.0 h1:J8e3Sba6DtBJQotXY5j5EaZNWwtv6Z9CoCFQsnrHNcE=
@@ -124,34 +123,32 @@ github.com/google/uuid v1.6.0 h1:NIvaJDMOsjHA8n1jAhLSgzrAzy1Hgr+hNrb57e+94F0=
github.com/google/uuid v1.6.0/go.mod h1:TIyPZe4MgqvfeYDBFedMoGGpEw/LqOeaOT+nhxU+yHo=
github.com/gorilla/websocket v1.5.3 h1:saDtZ6Pbx/0u+bgYQ3q96pZgCzfhKXGPqt7kZ72aNNg=
github.com/gorilla/websocket v1.5.3/go.mod h1:YR8l580nyteQvAITg2hZ9XVh4b55+EU/adAjf1fMHhE=
github.com/gravitational/trace v1.1.16-0.20220114165159-14a9a7dd6aaf h1:C1GPyPJrOlJlIrcaBBiBpDsqZena2Ks8spa5xZqr1XQ=
github.com/gravitational/trace v1.1.16-0.20220114165159-14a9a7dd6aaf/go.mod h1:zXqxTI6jXDdKnlf8s+nT+3c8LrwUEy3yNpO4XJL90lA=
github.com/hashicorp/hcl v1.0.0 h1:0Anlzjpi4vEasTeNFn2mLJgTSwt0+6sfsiTG8qcWGx4=
github.com/hashicorp/hcl v1.0.0/go.mod h1:E5yfLk+7swimpb2L/Alb/PJmXilQ/rhwaUYs4T20WEQ=
github.com/hashicorp/yamux v0.1.1 h1:yrQxtgseBDrq9Y652vSRDvsKCJKOUD+GzTS4Y0Y8pvE=
github.com/hashicorp/yamux v0.1.1/go.mod h1:CtWFDAQgb7dxtzFs4tWbplKIe2jSi3+5vKbgIO0SLnQ=
github.com/jonboulle/clockwork v0.2.2 h1:UOGuzwb1PwsrDAObMuhUnj0p5ULPj8V/xJ7Kx9qUBdQ=
github.com/jonboulle/clockwork v0.2.2/go.mod h1:Pkfl5aHPm1nk2H9h0bjmnJD/BcgbGXUBGnn1kMkgxc8=
github.com/json-iterator/go v1.1.12 h1:PV8peI4a0ysnczrg+LtxykD8LfKY9ML6u2jnxaEnrnM=
github.com/json-iterator/go v1.1.12/go.mod h1:e30LSqwooZae/UwlEbR2852Gd8hjQvJoHmT4TnhNGBo=
github.com/klauspost/compress v1.19.0 h1:sXLILfc9jV2QYWkzFOPWStmcUVH2RHEB1JCdY2oVvCQ=
github.com/klauspost/compress v1.19.0/go.mod h1:cwPg85FWrGar70rWktvGQj8/hthj3wpl0PGDogxkrSQ=
github.com/klauspost/cpuid/v2 v2.3.0 h1:S4CRMLnYUhGeDFDqkGriYKdfoFlDnMtqTiI/sFzhA9Y=
github.com/klauspost/cpuid/v2 v2.3.0/go.mod h1:hqwkgyIinND0mEev00jJYCxPNVRVXFQeu1XKlok6oO0=
github.com/klauspost/reedsolomon v1.11.8 h1:s8RpUW5TK4hjr+djiOpbZJB4ksx+TdYbRH7vHQpwPOY=
github.com/klauspost/reedsolomon v1.11.8/go.mod h1:4bXRN+cVzMdml6ti7qLouuYi32KHJ5MGv0Qd8a47h6A=
github.com/klauspost/cpuid/v2 v2.4.0 h1:S6Hrbc7+ywsr0r+RLapfGBHfyefhCTwEh3A0tV913Dw=
github.com/klauspost/cpuid/v2 v2.4.0/go.mod h1:19jmZ9mjzoF//ddRSUsv0zfBTJWh3QJh9FNxZTMrGxU=
github.com/klauspost/reedsolomon v1.12.0 h1:I5FEp3xSwVCcEh3F5A7dofEfhXdF/bWhQWPH+XwBFno=
github.com/klauspost/reedsolomon v1.12.0/go.mod h1:EPLZJeh4l27pUGC3aXOjheaoh1I9yut7xTURiW3LQ9Y=
github.com/kr/pretty v0.3.1 h1:flRD4NNwYAUpkphVc1HcthR4KEIFJ65n8Mw5qdRn3LE=
github.com/kr/pretty v0.3.1/go.mod h1:hoEshYVHaxMs3cyo3Yncou5ZscifuDolrwPKZanG3xk=
github.com/kr/text v0.2.0 h1:5Nx0Ya0ZqY2ygV366QzturHI13Jq95ApcVaJBhpS+AY=
github.com/kr/text v0.2.0/go.mod h1:eLer722TekiGuMkidMxC/pM04lWEeraHUUmBw8l2grE=
github.com/kylelemons/godebug v1.1.0 h1:RPNrshWIDI6G2gRW9EHilWtl7Z6Sb1BR0xunSBf0SNc=
github.com/kylelemons/godebug v1.1.0/go.mod h1:9/0rRGxNHcop5bhtWyNeEfOS8JIWk580+fNqagV/RAw=
github.com/leodido/go-urn v1.4.0 h1:WT9HwE9SGECu3lg4d/dIA+jxlljEa1/ffXKmRjqdmIQ=
github.com/leodido/go-urn v1.4.0/go.mod h1:bvxc+MVxLKB4z00jd1z+Dvzr47oO32F/QSNjSBOlFxI=
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0 h1:6E+4a0GO5zZEnZ81pIr0yLvtUWk2if982qA3F3QD6H4=
github.com/lufia/plan9stats v0.0.0-20211012122336-39d0f177ccd0/go.mod h1:zJYVVT2jmtg6P3p1VtQj7WsuWi/y4VnjVBn7F8KPB3I=
github.com/magiconair/properties v1.8.7 h1:IeQXZAiQcpL9mgcAe1Nu6cX9LLw6ExEHKjN0VQdvPDY=
github.com/magiconair/properties v1.8.7/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-isatty v0.0.20 h1:xfD0iDuEKnDkl03q4limB+vH+GxLEtL/jb4xVJSWWEY=
github.com/mattn/go-isatty v0.0.20/go.mod h1:W+V8PltTTMOvKvAeJH7IuucS94S2C6jfK/D7dTCTo3Y=
github.com/magiconair/properties v1.8.10 h1:s31yESBquKXCV9a/ScB3ESkOjUYYv+X0rg8SYxI99mE=
github.com/magiconair/properties v1.8.10/go.mod h1:Dhd985XPs7jluiymwWYZ0G4Z61jb3vdS329zhj2hYo0=
github.com/mattn/go-isatty v0.0.22 h1:j8l17JJ9i6VGPUFUYoTUKPSgKe/83EYU2zBC7YNKMw4=
github.com/mattn/go-isatty v0.0.22/go.mod h1:ZXfXG4SQHsB/w3ZeOYbR0PrPwLy+n6xiMrJlRFqopa4=
github.com/miekg/dns v1.1.61 h1:nLxbwF3XxhwVSm8g9Dghm9MHPaUZuqhPiGL+675ZmEs=
github.com/miekg/dns v1.1.61/go.mod h1:mnAarhS3nWaW+NVP2wTkYVIZyHNJ098SJZUki3eykwQ=
github.com/mitchellh/go-homedir v1.1.0 h1:lukF9ziXFxDFPkA1vsr5zpc1XuPDn/wFntq5mG+4E0Y=
@@ -163,6 +160,8 @@ github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd h1:TRLaZ9cD/w
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd/go.mod h1:6dJC0mAP4ikYIbvyc7fijjWJddQyLn8Ig3JB5CqoB9Q=
github.com/modern-go/reflect2 v1.0.2 h1:xBagoLtFs94CBntxluKeaWgTMpvLxC4ur3nMaC9Gz0M=
github.com/modern-go/reflect2 v1.0.2/go.mod h1:yWuevngMOJpCy52FWWMvUC8ws7m/LJsjYzDa0/r8luk=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822 h1:C3w9PqII01/Oq1c1nUAm88MOHcQC9l5mIlSMApZMrHA=
github.com/munnerz/goautoneg v0.0.0-20191010083416-a7dc8b61c822/go.mod h1:+n7T8mK8HuQTcFwEeznm/DIxMOiR9yIdICNftLE1DvQ=
github.com/nxadm/tail v1.4.8 h1:nPr65rt6Y5JFSKQO7qToXr7pePgD6Gwiw05lkbyAQTE=
github.com/nxadm/tail v1.4.8/go.mod h1:+ncqLTQzXmGhMZNUePPaPqPvBxHAIsmXswZKocGu+AU=
github.com/onsi/ginkgo v1.16.5 h1:8xi0RTUf59SOSfEtZMvwTvXYMzG4gV23XVHOZiXNtnE=
@@ -171,8 +170,8 @@ github.com/onsi/gomega v1.34.2 h1:pNCwDkzrsv7MS9kpaQvVb1aVLahQXyJ/Tv5oAZMI3i8=
github.com/onsi/gomega v1.34.2/go.mod h1:v1xfxRgk0KIsG+QOdm7p8UosrOzPYRo60fd3B/1Dukc=
github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaRUnok+kx1WdO15EQc=
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pelletier/go-toml/v2 v2.4.2 h1:M2fKKbmyvI+hGId/D0W64qDBMVhJnNR10O5gIbMc//Q=
github.com/pelletier/go-toml/v2 v2.4.2/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pion/dtls/v3 v3.1.4 h1:QhvtMflMfu9Kf0RcDC5BJBle4caPskByrKQR6uuYqpY=
github.com/pion/dtls/v3 v3.1.4/go.mod h1:cr/qotLISUw/9C1m83ZPNZtj9WnXkYLpfCptPqbkInc=
github.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8=
@@ -188,15 +187,15 @@ github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2 h1:Jamvg5psRI
github.com/pmezard/go-difflib v1.0.1-0.20181226105442-5d4384ee4fb2/go.mod h1:iKH77koFhYxTK1pcRnkKkqfTogsbg7gZNVY4sRDYZ/4=
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c h1:ncq/mPwQF4JjgDlrVEn3C11VoGHZN7m8qihwgMEtzYw=
github.com/power-devops/perfstat v0.0.0-20210106213030-5aafc221ea8c/go.mod h1:OmDBASR4679mdNQnz2pUhc2G8CO2JrUAVFDRBDP/hJE=
github.com/prometheus/client_golang v1.19.1 h1:wZWJDwK+NameRJuPGDhlnFgx8e8HN3XHQeLaYJFJBOE=
github.com/prometheus/client_golang v1.19.1/go.mod h1:mP78NwGzrVks5S2H6ab8+ZZGJLZUq1hoULYBAYBw1Ho=
github.com/prometheus/client_golang v1.21.1 h1:DOvXXTqVzvkIewV/CDPFdejpMCGeMcbGCQ8YOmu+Ibk=
github.com/prometheus/client_golang v1.21.1/go.mod h1:U9NM32ykUErtVBxdvD3zfi+EuFkkaBvMb09mIfe0Zgg=
github.com/prometheus/client_model v0.0.0-20190812154241-14fe0d1b01d4/go.mod h1:xMI15A0UPsDsEKsMN9yxemIoYk6Tm2C1GtYGdfGttqA=
github.com/prometheus/client_model v0.6.0 h1:k1v3CzpSRUTrKMppY35TLwPvxHqBu0bYgxZzqGIgaos=
github.com/prometheus/client_model v0.6.0/go.mod h1:NTQHnmxFpouOD0DpvP4XujX3CdOAGQPoaGhyTchlyt8=
github.com/prometheus/common v0.48.0 h1:QO8U2CdOzSn1BBsmXJXduaaW+dY/5QLjfB8svtSzKKE=
github.com/prometheus/common v0.48.0/go.mod h1:0/KsvlIEfPQCQ5I2iNSAWKPZziNCvRs5EC6ILDTlAPc=
github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=
github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
github.com/prometheus/client_model v0.6.2 h1:oBsgwpGs7iVziMvrGhE53c/GrLUsZdHnqNwqPLxwZyk=
github.com/prometheus/client_model v0.6.2/go.mod h1:y3m2F6Gdpfy6Ut/GBsUqTWZqCUvMVzSfMLjcu6wAwpE=
github.com/prometheus/common v0.62.0 h1:xasJaQlnWAeyHdUBeGjXmutelfJHWMRr+Fg4QszZ2Io=
github.com/prometheus/common v0.62.0/go.mod h1:vyBcEuLSvWos9B1+CyL7JZ2up+uFzXhkqml0W5zIY1I=
github.com/prometheus/procfs v0.15.1 h1:YagwOFzUgYfKKHX6Dr+sHT7km/hxC76UB0learggepc=
github.com/prometheus/procfs v0.15.1/go.mod h1:fB45yRUv8NstnjriLhBQLuOUt+WW4BsoGhij/e3PBqk=
github.com/quic-go/go-ossfuzz-seeds v0.1.0 h1:APacT+iIaNF6fd8AGEiN3bT/Jtkd2jz4v4TzM7MFjy0=
github.com/quic-go/go-ossfuzz-seeds v0.1.0/go.mod h1:3IOHRbJIc+L6YKMwfDtJAM9Vj9k0YY4muhuyUYk5tbk=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
@@ -217,17 +216,12 @@ github.com/sagikazarmark/locafero v0.4.0 h1:HApY1R9zGo4DBgr7dqsTH/JJxLTTsOt7u6ke
github.com/sagikazarmark/locafero v0.4.0/go.mod h1:Pe1W6UlPYUk/+wc/6KFhbORCfqzgYEpgQ3O5fPuL3H4=
github.com/sagikazarmark/slog-shim v0.1.0 h1:diDBnUNK9N/354PgrxMywXnAwEr1QZcOr6gto+ugjYE=
github.com/sagikazarmark/slog-shim v0.1.0/go.mod h1:SrcSrq8aKtyuqEI1uvTDTK1arOWRIczQRv+GVI1AkeQ=
github.com/shadowsocks/go-shadowsocks2 v0.1.5 h1:PDSQv9y2S85Fl7VBeOMF9StzeXZyK1HakRm86CUbr28=
github.com/shadowsocks/go-shadowsocks2 v0.1.5/go.mod h1:AGGpIoek4HRno4xzyFiAtLHkOpcoznZEkAccaI/rplM=
github.com/shirou/gopsutil/v3 v3.24.5 h1:i0t8kL+kQTvpAYToeuiVk3TgDeKOFioZO3Ztz/iZ9pI=
github.com/shirou/gopsutil/v3 v3.24.5/go.mod h1:bsoOS1aStSs9ErQ1WWfxllSeS1K5D+U30r2NfcubMVk=
github.com/shoenig/go-m1cpu v0.1.6 h1:nxdKQNcEB6vzgA2E2bvzKIYRuNj7XNJ4S/aRSwKzFtM=
github.com/shoenig/go-m1cpu v0.1.6/go.mod h1:1JJMcUBvfNwpq05QDQVAnx3gUHr9IYF7GNg9SUEw2VQ=
github.com/shoenig/test v0.6.4 h1:kVTaSd7WLz5WZ2IaoM0RSzRsUD+m8wRR+5qvntpn4LU=
github.com/shoenig/test v0.6.4/go.mod h1:byHiCGXqrVaflBLAMq/srcZIHynQPQgeyvkvXnjqq0k=
github.com/sirupsen/logrus v1.7.0/go.mod h1:yWOB1SBYBC5VeMP7gHvWumXLIWorT60ONWic61uBYv0=
github.com/sirupsen/logrus v1.9.3 h1:dueUQJ1C2q9oE3F7wvmSGAaVtTmUizReu6fjN8uqzbQ=
github.com/sirupsen/logrus v1.9.3/go.mod h1:naHLuLoDiP4jHNo9R0sCBMtWGeIprob74mVsIT4qYEQ=
github.com/songgao/water v0.0.0-20200317203138-2b4b6d7c09d8 h1:TG/diQgUe0pntT/2D9tmUCz4VNwm9MfrtPr0SU2qSX8=
github.com/songgao/water v0.0.0-20200317203138-2b4b6d7c09d8/go.mod h1:P5HUIBuIWKbyjl083/loAegFkfbFNx5i2qEP4CNbm7E=
github.com/sourcegraph/conc v0.3.0 h1:OQTbbt6P72L20UqAkXXuLOj79LfEanQ+YQFNpLA9ySo=
@@ -245,11 +239,8 @@ github.com/stretchr/objx v0.4.0/go.mod h1:YvHI0jy2hoMjB+UWwv71VJQ9isScKT/TqJzVSS
github.com/stretchr/objx v0.5.0/go.mod h1:Yh+to48EsGEfYuaHDzXPcE3xhTkx73EhmCGUpEOglKo=
github.com/stretchr/objx v0.5.2 h1:xuMeJ0Sdp5ZMRXx/aWO6RZxdr3beISkG5/G/aIRr3pY=
github.com/stretchr/objx v0.5.2/go.mod h1:FRsXN1f5AsAjCGJKqEizvkpNtU+EGNCLh3NxZ/8L+MA=
github.com/stretchr/testify v1.2.2/go.mod h1:a8OnRcib4nhh0OaRAV+Yts87kKdq0PP7pXfy6kDkUVs=
github.com/stretchr/testify v1.3.0/go.mod h1:M5WIy9Dh21IEIfnGCwXGc5bZfKNJtfHm1UVUgZn+9EI=
github.com/stretchr/testify v1.4.0/go.mod h1:j7eGeouHqKxXV5pUuKE4zz7dFj8WfuZ+81PSLYec5m4=
github.com/stretchr/testify v1.6.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.0/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.7.1/go.mod h1:6Fq8oRcR53rry900zMqJjRRixrwX3KX962/h/Wwjteg=
github.com/stretchr/testify v1.8.0/go.mod h1:yNjHg4UonilssWZ8iaSj1OCr/vHnekPRkoO+kdMU+MU=
github.com/stretchr/testify v1.8.4/go.mod h1:sz/lmYIOXD/1dqDmKjjqLyZ2RngseejIcXlSw2iwfAo=
@@ -258,10 +249,6 @@ github.com/stretchr/testify v1.11.1 h1:7s2iGBzp5EwR7/aIZr8ao5+dra3wiQyKjjFuvgVKu
github.com/stretchr/testify v1.11.1/go.mod h1:wZwfW3scLgRK+23gO65QZefKpKQRnfz6sD981Nm4B6U=
github.com/subosito/gotenv v1.6.0 h1:9NlTDc1FTs4qu0DDq7AEtTPNw6SVm7uBMsUCUjABIf8=
github.com/subosito/gotenv v1.6.0/go.mod h1:Dk4QP5c2W3ibzajGcXpNraDfq2IrhjMIvMSWPKKo0FU=
github.com/templexxx/cpu v0.1.1 h1:isxHaxBXpYFWnk2DReuKkigaZyrjs2+9ypIdGP4h+HI=
github.com/templexxx/cpu v0.1.1/go.mod h1:w7Tb+7qgcAlIyX4NhLuDKt78AHA5SzPmq0Wj6HiEnnk=
github.com/templexxx/xorsimd v0.4.3 h1:9AQTFHd7Bhk3dIT7Al2XeBX5DWOvsUPZCuhyAtNbHjU=
github.com/templexxx/xorsimd v0.4.3/go.mod h1:oZQcD6RFDisW2Am58dSAGwwL6rHjbzrlu25VDqfWkQg=
github.com/tidwall/gjson v1.14.2/go.mod h1:/wbyibRr2FHMks5tjHJ5F8dMZh3AcwJEMf5vlfC0lxk=
github.com/tidwall/gjson v1.19.0 h1:xwxm7n691Uf3u5OFjzngavjGTh55KX5q/9w9xHW88JU=
github.com/tidwall/gjson v1.19.0/go.mod h1:V37/opeE/JbLUOfH0QTXiNez2l0RUjYUhpT4szFQAfc=
@@ -273,10 +260,10 @@ github.com/tidwall/sjson v1.2.5 h1:kLy8mja+1c9jlljvWTlSazM7cKDRfJuR/bOJhcY5NcY=
github.com/tidwall/sjson v1.2.5/go.mod h1:Fvgq9kS/6ociJEDnK0Fk1cpYF4FIW6ZF7LAe+6jwd28=
github.com/tjfoc/gmsm v1.4.1 h1:aMe1GlZb+0bLjn+cKTPEvvn9oUEBlJitaZiiBwsbgho=
github.com/tjfoc/gmsm v1.4.1/go.mod h1:j4INPkHWMrhJb38G+J6W4Tw0AbuN8Thu3PbdVYhVcTE=
github.com/tklauser/go-sysconf v0.3.12 h1:0QaGUFOdQaIVdPgfITYzaTegZvdCjmYO52cSFAEVmqU=
github.com/tklauser/go-sysconf v0.3.12/go.mod h1:Ho14jnntGE1fpdOqQEEaiKRpvIavV0hSfmBq8nJbHYI=
github.com/tklauser/numcpus v0.6.1 h1:ng9scYS7az0Bk4OZLvrNXNSAO2Pxr1XXRAPyjhIx+Fk=
github.com/tklauser/numcpus v0.6.1/go.mod h1:1XfjsgE2zo8GVw7POkMbHENHzVg3GzmoZ9fESEdAacY=
github.com/tklauser/go-sysconf v0.3.16 h1:frioLaCQSsF5Cy1jgRBrzr6t502KIIwQ0MArYICU0nA=
github.com/tklauser/go-sysconf v0.3.16/go.mod h1:/qNL9xxDhc7tx3HSRsLWNnuzbVfh3e7gh/BmM179nYI=
github.com/tklauser/numcpus v0.11.0 h1:nSTwhKH5e1dMNsCdVBukSZrURJRoHbSEQjdEbY+9RXw=
github.com/tklauser/numcpus v0.11.0/go.mod h1:z+LwcLq54uWZTX0u/bGobaV34u6V7KNlTZejzM6/3MQ=
github.com/twitchyliquid64/golang-asm v0.15.1 h1:SU5vSMR7hnwNxj24w34ZyCi/FmDZTkS4MhqMhdFk5YI=
github.com/twitchyliquid64/golang-asm v0.15.1/go.mod h1:a1lVb/DtPvCB8fslRZhAngC2+aY1QWCk3Cedj/Gdt08=
github.com/ugorji/go/codec v1.3.1 h1:waO7eEiFDwidsBN6agj1vJQ4AG7lh2yqXyOXqhgQuyY=
@@ -286,18 +273,18 @@ github.com/vishvananda/netlink v1.1.1-0.20211118161826-650dca95af54/go.mod h1:tw
github.com/vishvananda/netns v0.0.0-20200728191858-db3c7e526aae/go.mod h1:DD4vA1DwXk04H54A1oHXtwZmA0grkVMdPxx/VGLCah0=
github.com/vishvananda/netns v0.0.4 h1:Oeaw1EM2JMxD51g9uhtC0D7erkIjgmj8+JZc26m1YX8=
github.com/vishvananda/netns v0.0.4/go.mod h1:SpkAiCQRtJ6TvvxPnOSyH3BMl6unz3xZlaprSwhNNJM=
github.com/vulcand/predicate v1.2.0 h1:uFsW1gcnnR7R+QTID+FVcs0sSYlIGntoGOTb3rQJt50=
github.com/vulcand/predicate v1.2.0/go.mod h1:VipoNYXny6c8N381zGUWkjuuNHiRbeAZhE7Qm9c+2GA=
github.com/xjasonlyu/tun2socks/v2 v2.6.0 h1:gI9saJT3XgH4e6v9jBuHRLwK7l3aN9YFWec/SsDTDx4=
github.com/xjasonlyu/tun2socks/v2 v2.6.0/go.mod h1:35AwqxIxnMkfBfT0UJ1Lku7PZm2ZiZJ8sxHyp0gt1yw=
github.com/xtaci/kcp-go/v5 v5.6.5 h1:oxGZNobj3OddrLzwdJYnR/waNgwrL98u02u0DWNHE3k=
github.com/xtaci/kcp-go/v5 v5.6.5/go.mod h1:Qy3Zf2tWTdFdEs0E8JvhrX+39r5UDZoYac8anvud7/Q=
github.com/xtaci/kcp-go/v5 v5.6.72 h1:FLaQPalgpufJYQRk0OK+gErEhXGLUPjv6FSRPrFR8Lk=
github.com/xtaci/kcp-go/v5 v5.6.72/go.mod h1:9O3D8WR+cyyUjGiTILYfg17vn72otWuXK2AFfqIe6CM=
github.com/xtaci/lossyconn v0.0.0-20190602105132-8df528c0c9ae h1:J0GxkO96kL4WF+AIT3M4mfUVinOCPgf2uUWYFUzN0sM=
github.com/xtaci/lossyconn v0.0.0-20190602105132-8df528c0c9ae/go.mod h1:gXtu8J62kEgmN++bm9BVICuT/e8yiLI2KFobd/TRFsE=
github.com/xtaci/smux v1.5.31 h1:3ha7sHtH46h85Iv7MfQogxasuRt1KPRhoFB3S4rmHgU=
github.com/xtaci/smux v1.5.31/go.mod h1:OMlQbT5vcgl2gb49mFkYo6SMf+zP3rcjcwQz7ZU7IGY=
github.com/xtaci/smux v1.5.57 h1:N72VbGoSYxgcm6mPOYX0QzEZNVD3UI/JlVvAtXF+WrY=
github.com/xtaci/smux v1.5.57/go.mod h1:IGQ9QYrBphmb/4aTnLEcJby0TNr3NV+OslIOMrX825Q=
github.com/xtaci/tcpraw v1.2.25 h1:VDlqo0op17JeXBM6e2G9ocCNLOJcw9mZbobMbJjo0vk=
github.com/xtaci/tcpraw v1.2.25/go.mod h1:dKyZ2V75s0cZ7cbgJYdxPvms7af0joIeOyx1GgJQbLk=
github.com/xyproto/randomstring v1.0.5 h1:YtlWPoRdgMu3NZtP45drfy1GKoojuR7hmRcnhZqKjWU=
github.com/xyproto/randomstring v1.0.5/go.mod h1:rgmS5DeNXLivK7YprL0pY+lTuhNQW3iGxZ18UQApw/E=
github.com/yl2chen/cidranger v1.0.2 h1:lbOWZVCG1tCRX4u24kuM1Tb4nHqWkDxwLdoS+SevawU=
github.com/yl2chen/cidranger v1.0.2/go.mod h1:9U1yz7WPYDwf0vpNWFaeRh0bjwz5RVgRy/9UEQfHl0g=
github.com/yusufpapurcu/wmi v1.2.4 h1:zFUKzehAFReQwLys1b/iSMl+JQGSCSjtVqQn9bBrPo0=
@@ -310,20 +297,20 @@ github.com/zeebo/blake3 v0.2.4 h1:KYQPkhpRtcqh0ssGYcKLG1JYvddkEA8QwCM/yBqhaZI=
github.com/zeebo/blake3 v0.2.4/go.mod h1:7eeQ6d2iXWRGF6npfaxl2CU+xy2Fjo2gxeyZGCRUjcE=
github.com/zeebo/pcg v1.0.1 h1:lyqfGeWiv4ahac6ttHs+I5hwtH/+1mrhlCtVNQM2kHo=
github.com/zeebo/pcg v1.0.1/go.mod h1:09F0S9iiKrwn9rlI5yjLkmrug154/YRW6KnnXVDM/l4=
go.mongodb.org/mongo-driver/v2 v2.5.0 h1:yXUhImUjjAInNcpTcAlPHiT7bIXhshCTL3jVBkF3xaE=
go.mongodb.org/mongo-driver/v2 v2.5.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.mongodb.org/mongo-driver/v2 v2.7.0 h1:RO+zqavD2/GCL3cxOMyZhx6R9Irzr8/6gsoqx5tcY/c=
go.mongodb.org/mongo-driver/v2 v2.7.0/go.mod h1:yOI9kBsufol30iFsl1slpdq1I0eHPzybRWdyYUs8K/0=
go.opentelemetry.io/auto/sdk v1.2.1 h1:jXsnJ4Lmnqd11kwkBV2LgLoFMZKizbCi5fNZ/ipaZ64=
go.opentelemetry.io/auto/sdk v1.2.1/go.mod h1:KRTj+aOaElaLi+wW1kO/DZRXwkF4C5xPbEe3ZiIhN7Y=
go.opentelemetry.io/otel v1.43.0 h1:mYIM03dnh5zfN7HautFE4ieIig9amkNANT+xcVxAj9I=
go.opentelemetry.io/otel v1.43.0/go.mod h1:JuG+u74mvjvcm8vj8pI5XiHy1zDeoCS2LB1spIq7Ay0=
go.opentelemetry.io/otel/metric v1.43.0 h1:d7638QeInOnuwOONPp4JAOGfbCEpYb+K6DVWvdxGzgM=
go.opentelemetry.io/otel/metric v1.43.0/go.mod h1:RDnPtIxvqlgO8GRW18W6Z/4P462ldprJtfxHxyKd2PY=
go.opentelemetry.io/otel/sdk v1.43.0 h1:pi5mE86i5rTeLXqoF/hhiBtUNcrAGHLKQdhg4h4V9Dg=
go.opentelemetry.io/otel/sdk v1.43.0/go.mod h1:P+IkVU3iWukmiit/Yf9AWvpyRDlUeBaRg6Y+C58QHzg=
go.opentelemetry.io/otel/sdk/metric v1.43.0 h1:S88dyqXjJkuBNLeMcVPRFXpRw2fuwdvfCGLEo89fDkw=
go.opentelemetry.io/otel/sdk/metric v1.43.0/go.mod h1:C/RJtwSEJ5hzTiUz5pXF1kILHStzb9zFlIEe85bhj6A=
go.opentelemetry.io/otel/trace v1.43.0 h1:BkNrHpup+4k4w+ZZ86CZoHHEkohws8AY+WTX09nk+3A=
go.opentelemetry.io/otel/trace v1.43.0/go.mod h1:/QJhyVBUUswCphDVxq+8mld+AvhXZLhe+8WVFxiFff0=
go.opentelemetry.io/otel v1.44.0 h1:JjwHmHpA4iZ3wBxluu2fbbE7j4kqlE8jXyAyPXH7HqU=
go.opentelemetry.io/otel v1.44.0/go.mod h1:BMgjTHL9WPRlRjL2oZCBTL4whCGtXch2H4BhOPIAyYc=
go.opentelemetry.io/otel/metric v1.44.0 h1:1w0gILTcHdr3YI+ixLyjemwrVnsMURbTZFrSYCdDdmc=
go.opentelemetry.io/otel/metric v1.44.0/go.mod h1:8O7hanEPBNgEMmybD3s2VBKcgWOCsA6tzHBPODAiquo=
go.opentelemetry.io/otel/sdk v1.44.0 h1:nHYwb9lK+fJPU/dnT6s7W7Z8itMWyqrnVfbheVYrZ58=
go.opentelemetry.io/otel/sdk v1.44.0/go.mod h1:Osuydd3Se74nqjAKxid74N5eC+jfEqfTegHRnq58oK0=
go.opentelemetry.io/otel/sdk/metric v1.44.0 h1:3LlKgI+VjbVsjNRFZJZAJ30WjXC5VkNRks6si09iEfI=
go.opentelemetry.io/otel/sdk/metric v1.44.0/go.mod h1:5B5pMARnXxKhltooO4xUuCBorl65a4EpnTalObqOigA=
go.opentelemetry.io/otel/trace v1.44.0 h1:jxF5CsGYCe74MCRx2X4g7WsY/VBKRqqpNvXlX/6gtIk=
go.opentelemetry.io/otel/trace v1.44.0/go.mod h1:oLl1jrMQAVo6v3GAggN+1VH9VIz9iUSvW53sW1Q8PIE=
go.uber.org/mock v0.6.0 h1:hyF9dfmbgIX5EfOdasqLsWD6xqpNZlXblLB/Dbnwv3Y=
go.uber.org/mock v0.6.0/go.mod h1:KiVJ4BqZJaMj4svdfmHM0AUx4NJYO8ZNpPnZn1Z+BBU=
go.uber.org/multierr v1.11.0 h1:blXXJkSxSSfBVBlC76pxqeO+LN3aDfLQo+309xJstO0=
@@ -334,10 +321,8 @@ golang.org/x/crypto v0.0.0-20190308221718-c2843e01d9a2/go.mod h1:djNgcEr1/C05ACk
golang.org/x/crypto v0.0.0-20191011191535-87dc89f01550/go.mod h1:yigFU9vqHzYiE8UmvKecakEJjdnWj3jj499lnFckfCI=
golang.org/x/crypto v0.0.0-20200622213623-75b288015ac9/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201012173705-84dcc777aaee/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20201016220609-9e8e0b390897/go.mod h1:LzIPMQfyMNhhGPhUkYOs5KpL4U8rLKemX1yGLhDgUto=
golang.org/x/crypto v0.0.0-20210220033148-5ea612d1eb83/go.mod h1:jdWPYTVW3xRLrWPugEBEK3UY2ZEsg3UU495nc5E+M+I=
golang.org/x/crypto v0.54.0 h1:YLIA59K4fiNzHzjnZt2tUJQjQtUWfWbeHBqKtk3eScw=
golang.org/x/crypto v0.54.0/go.mod h1:KWL8ny2AZdGR2cWmzeHrp2azQPGogOv+HeQaVEXC2dk=
golang.org/x/crypto v0.55.0 h1:+KWHjbgOaAQ66dh/YlkZKHlz9ZUlq61AFirAR9ntP8M=
golang.org/x/crypto v0.55.0/go.mod h1:uq0V9dE/fzQuJtbnL+2EhWOE63vo164FY8xqEnV9xis=
golang.org/x/exp v0.0.0-20190121172915-509febef88a4/go.mod h1:CJ0aWSM057203Lf6IL+f9T1iT9GByDxfZKAQTCR3kQA=
golang.org/x/exp v0.0.0-20241210194714-1829a127f884 h1:Y/Mj/94zIQQGHVSv1tTtQBDaQaJe62U9bkDZKKyhPCU=
golang.org/x/exp v0.0.0-20241210194714-1829a127f884/go.mod h1:qj5a5QZpwLU2NLQudwIN5koi3beDhSAlJwa67PuM98c=
@@ -346,8 +331,8 @@ golang.org/x/lint v0.0.0-20190227174305-5b3e6a55c961/go.mod h1:wehouNa3lNwaWXcvx
golang.org/x/lint v0.0.0-20190313153728-d0100b6bd8b3/go.mod h1:6SW0HCj/g11FgYtHlgUYUwCkIfeOF89ocIRzGO/8vkc=
golang.org/x/lint v0.0.0-20200302205851-738671d3881b/go.mod h1:3xt1FjdF8hUf6vQPIChWIBhFzV8gjjsPE/fR3IyQdNY=
golang.org/x/mod v0.1.1-0.20191105210325-c90efee705ee/go.mod h1:QqPTAvyqsEbceGzBzNggFXnrqF1CaUcvgkdR5Ot7KZg=
golang.org/x/mod v0.37.0 h1:vF1DjpVEshcIqoEaauuHebaLk1O1forxjxBaVn884JQ=
golang.org/x/mod v0.37.0/go.mod h1:m8S8VeM9r4dzDwjrKO0a1sZP3YjeMamRRlD+fmR2Q/0=
golang.org/x/mod v0.38.0 h1:MECBjubtXD7yj4HrhIUcywNaGeNVUdfVnxmPajOk4yk=
golang.org/x/mod v0.38.0/go.mod h1:V6Xz0pq8TQ3dGqVQ1FVHuelZpAL0uNhSkk9ogYP3c40=
golang.org/x/net v0.0.0-20180724234803-3673e40ba225/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20180826012351-8a410e7b638d/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
golang.org/x/net v0.0.0-20190213061140-3a22650c66bd/go.mod h1:mL1N/T3taQHkDXs73rZJwtUhF3w3ftmwwsq0BUmARs4=
@@ -355,9 +340,8 @@ golang.org/x/net v0.0.0-20190311183353-d8887717615a/go.mod h1:t9HGtf8HONx5eT2rtn
golang.org/x/net v0.0.0-20190404232315-eb5bcb51f2a3/go.mod h1:t9HGtf8HONx5eT2rtn7q6eTqICYqUVnKs3thJo3Qplg=
golang.org/x/net v0.0.0-20190620200207-3b0461eec859/go.mod h1:z5CRVTTTmAJ677TzLLGU+0bjPO0LkuOLi4/5GtJWs/s=
golang.org/x/net v0.0.0-20201010224723-4f7140c49acb/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.0.0-20201031054903-ff519b6c9102/go.mod h1:sp8m0HH+o8qH0wwXwYZr8TS3Oi6o0r6Gce1SSxlDquU=
golang.org/x/net v0.57.0 h1:K5+3DljvIuDG9/Jv9rvyMywYNFCQ9RSUY6OOTTkT+tE=
golang.org/x/net v0.57.0/go.mod h1:KpXc8iv+r3XplLAG/f7Jsf9RPszJzdR0f58q9vGOuEU=
golang.org/x/net v0.58.0 h1:ynWG7rqYi4ccpTEuPZ2QGWHktVEM9DMCj9yzDE0Q7To=
golang.org/x/net v0.58.0/go.mod h1:YwCddHnFlT7eLQqVprV19OnhLGtc5xOKgE0RyqgfWAU=
golang.org/x/oauth2 v0.0.0-20180821212333-d2e6202438be/go.mod h1:N/0e6XlmueqKjAGxoOufVs8QHGRruUQn6yWY3a++T0U=
golang.org/x/sync v0.0.0-20180314180146-1d60e4601c6f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
golang.org/x/sync v0.0.0-20181108010431-42b317875d0f/go.mod h1:RxMgew5VJxzue5/jJTE5uejpjVlOe/izrB70Jof72aM=
@@ -368,34 +352,28 @@ golang.org/x/sys v0.0.0-20180830151530-49385e6e1522/go.mod h1:STP8DvDyc/dI5b8T5h
golang.org/x/sys v0.0.0-20190215142949-d0b11bdaac8a/go.mod h1:STP8DvDyc/dI5b8T5hshtkjS+E42TnysNCUPdjciGhY=
golang.org/x/sys v0.0.0-20190412213103-97732733099d/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20190916202348-b4ddaad3f8a3/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20191026070338-33540a1f6037/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200217220822-9197077df867/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200728102440-3e129f6d46b1/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20200930185726-fdedc70b468f/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20201204225414-ed752295db88/go.mod h1:h1NjWce9XRLGQEsW7wpKNCjG9DtNlClVuFLEZdDNbEs=
golang.org/x/sys v0.0.0-20220715151400-c0bba94af5f8/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.6.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.8.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.11.0/go.mod h1:oPkhp1MJrh7nUepCBck5+mAzfO9JrbApNNgaTdGDITg=
golang.org/x/sys v0.47.0 h1:o7XGOvZQCADBQQ4Y7VNq2dRWQR7JmOUW8Kxx4ZsNgWs=
golang.org/x/sys v0.47.0/go.mod h1:4GL1E5IUh+htKOUEOaiffhrAeqysfVGipDYzABqnCmw=
golang.org/x/term v0.0.0-20201117132131-f5c789dd3221/go.mod h1:Nr5EML6q2oocZ2LXRh80K7BxOlk5/8JxuGnuhpl+muw=
golang.org/x/term v0.45.0 h1:NwWyBmoJCbfTHpxrWoZ9C6/VxOf7ic219I8xZZFdrf0=
golang.org/x/term v0.45.0/go.mod h1:9aqxs0blBcrm/n0L9QW0aRVD+ktan8ssZromtqJC43w=
golang.org/x/text v0.3.0/go.mod h1:NqM8EUOU14njkJ3fqMW+pc6Ldnwhi/IjpwHt7yyuwOQ=
golang.org/x/text v0.3.3/go.mod h1:5Zoc/QRtKVWzQhOtBMvqHzDpF6irO9z98xDceosuGiQ=
golang.org/x/text v0.40.0 h1:Ub2Z6/xjgF1WrYQz2nuITOEegKFtiIy+rieRJ5lHZKs=
golang.org/x/text v0.40.0/go.mod h1:hpnzDAfGV753zIKo+wk3u1bVKCGPbrnF7+7LBF/UHVY=
golang.org/x/time v0.12.0 h1:ScB/8o8olJvc+CQPWrK3fPZNfh7qgwCrY0zJmoEQLSE=
golang.org/x/time v0.12.0/go.mod h1:CDIdPxbZBQxdj6cxyCIdrNogrJKMJ7pr37NYpMcMDSg=
golang.org/x/text v0.41.0 h1:vz/seA0lnX87Othu2f/0L24RcgrXD9/YFTSuGjj3rH8=
golang.org/x/text v0.41.0/go.mod h1:jvf1O8ajNzZqhSrQBPbutR/EB83Cc0CFrezNQIwbb5M=
golang.org/x/time v0.14.0 h1:MRx4UaLrDotUKUdCIqzPC48t1Y9hANFKIRpNx+Te8PI=
golang.org/x/time v0.14.0/go.mod h1:eL/Oa2bBBK0TkX57Fyni+NgnyQQN4LitPmob2Hjnqw4=
golang.org/x/tools v0.0.0-20180917221912-90fa682c2a6e/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190114222345-bf090417da8b/go.mod h1:n7NCudcB/nEzxVGmLbDWY5pfWTLqBcC2KZ6jyYvM4mQ=
golang.org/x/tools v0.0.0-20190226205152-f727befe758c/go.mod h1:9Yl7xja0Znq3iFh3HoIrodX9oNMXvdceNzlUR8zjMvY=
golang.org/x/tools v0.0.0-20190311212946-11955173bddd/go.mod h1:LCzVGOaR6xXOjkQ3onu1FJEFr0SW1gC7cKk1uF8kGRs=
golang.org/x/tools v0.0.0-20190524140312-2c0ae7006135/go.mod h1:RgjU9mgBXZiqYHBnxXauZ1Gv1EHHAz9KjViQ78xBX0Q=
golang.org/x/tools v0.0.0-20200130002326-2f3ba24bd6e7/go.mod h1:TB2adYChydJhpapKDTa4BR/hXlZSLoq2Wpct/0txZ28=
golang.org/x/tools v0.47.0 h1:7Kn5x/d1svx/PzryTsqeoZN4TZwqeH5pGWjefhLi/1Q=
golang.org/x/tools v0.47.0/go.mod h1:dFHnyTvFWY212G+h7ZY4Vsp/K3U4/7W9TyVaAul8uCA=
golang.org/x/tools v0.48.0 h1:3+hClM1aLL5mjMKm5ovokw9epgRXPuu2tILgismM6RE=
golang.org/x/tools v0.48.0/go.mod h1:08xX0orndb/F7jJxGDicx061tyd5pcMto75YMAXr6lk=
golang.org/x/xerrors v0.0.0-20191011141410-1b5146add898/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.org/x/xerrors v0.0.0-20191204190536-9bdfabe68543/go.mod h1:I/5z698sn9Ka8TeJc9MKroUUfqBBauWjQqLJ2OPfmY0=
golang.zx2c4.com/wintun v0.0.0-20230126152724-0fa3db229ce2 h1:B82qJJgjvYKsXS9jeunTOisW56dUokqW/FOteYJJ/yg=
@@ -413,10 +391,9 @@ google.golang.org/genproto/googleapis/rpc v0.0.0-20260723215102-3fe39f3c1018/go.
google.golang.org/grpc v1.19.0/go.mod h1:mqu4LbDTu4XGKhr4mRzUsmM4RtVoemTSY81AxZiDr8c=
google.golang.org/grpc v1.23.0/go.mod h1:Y5yQAOtifL1yxbo5wqy6BxZv8vAUGQwXBOALyacEbxg=
google.golang.org/grpc v1.25.1/go.mod h1:c3i+UQWmh7LiEpx4sFZnkU36qjEYZ0imhYfXVyQciAY=
google.golang.org/grpc v1.27.1/go.mod h1:qbnxyOmOxrQa7FizSgH+ReBfzJrCY1pSN7KXBS8abTk=
google.golang.org/grpc v1.31.0/go.mod h1:N36X2cJ7JwdamYAgDz+s+rVMFjt3numwzf/HckM8pak=
google.golang.org/grpc v1.82.1 h1:NnAxzGRA0677vCa4BUkOAnO5+FfQqVl9iUXeD0IqcGE=
google.golang.org/grpc v1.82.1/go.mod h1:yzTZ1TB1Z3SG+LIYaI+WiE8D5+PZ3ArnrSp8zF3+/ZA=
google.golang.org/grpc v1.83.2 h1:EManeRomTObA0BU7I8vXgg/78uE5MJ9M8B39EX2WscU=
google.golang.org/grpc v1.83.2/go.mod h1:YPI1hK3kDked6iHvgX3tR0y+nX/qpMFKhPgFsokw1S8=
google.golang.org/protobuf v0.0.0-20200109180630-ec00e32a8dfd/go.mod h1:DFci5gLYBciE7Vtevhsrf46CRTquxDuWsQurQQe4oz8=
google.golang.org/protobuf v0.0.0-20200221191635-4d8936d0db64/go.mod h1:kwYJMbMJ01Woi6D6+Kah6886xMZcty6N08ah7+eCXa0=
google.golang.org/protobuf v0.0.0-20200228230310-ab0ca4ff8a60/go.mod h1:cfTl7dwQJ+fmap5saPgwCLgHXTUD7jkjRqWcaiX5VyM=
+8
View File
@@ -6,6 +6,7 @@ import (
"errors"
"fmt"
"net"
"path/filepath"
"sync"
"testing"
"time"
@@ -314,6 +315,13 @@ func TestNewHandler(t *testing.T) {
// ---------------------------------------------------------------------------
func TestInit_DefaultExchanger(t *testing.T) {
// Redirect the system config to a nonexistent path: a resolv.conf on
// the host (or CI runner) would otherwise preempt this fallback with
// "system-N" exchangers and the default would never be reached.
restore := resolvConfPath
resolvConfPath = filepath.Join(t.TempDir(), "resolv.conf")
t.Cleanup(func() { resolvConfPath = restore })
h := newTestHandler()
if err := h.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatalf("Init: %v", err)
+17 -8
View File
@@ -10,12 +10,18 @@ import (
"strings"
)
// resolvConfPath is the file systemNameservers reads. It is a var so tests
// can redirect it: the system DNS configuration is per-environment, and
// tests exercising the Init fallback chain must be hermetic against the
// host's (or CI runner's) resolv.conf.
var resolvConfPath = "/etc/resolv.conf"
// systemNameservers reads the system DNS configuration from /etc/resolv.conf
// and returns the nameserver addresses as URL-like strings suitable for
// exchanger.NewExchanger (e.g., "udp://1.1.1.1:53").
// Returns nil if the file cannot be read or contains no nameserver entries.
func systemNameservers() []string {
f, err := os.Open("/etc/resolv.conf")
f, err := os.Open(resolvConfPath)
if err != nil {
return nil
}
@@ -33,15 +39,18 @@ func systemNameservers() []string {
continue
}
addr := fields[1]
// Verify it's a valid IP address.
if ip := net.ParseIP(addr); ip == nil {
// The port is optional in resolv.conf; default it to 53 when
// absent. Validate the host part only: net.ParseIP rejects a
// host:port string, so validating addr itself would drop every
// nameserver that carries an explicit port.
host, port, err := net.SplitHostPort(addr)
if err != nil || port == "" {
host, port = addr, "53"
}
if ip := net.ParseIP(host); ip == nil {
continue
}
// If no port, default to 53.
if _, _, err := net.SplitHostPort(addr); err != nil {
addr = net.JoinHostPort(addr, "53")
}
servers = append(servers, fmt.Sprintf("udp://%s", addr))
servers = append(servers, fmt.Sprintf("udp://%s", net.JoinHostPort(host, port)))
}
if err := scanner.Err(); err != nil {
return nil
+53
View File
@@ -0,0 +1,53 @@
//go:build !windows
package dns
import (
"os"
"path/filepath"
"testing"
xmd "github.com/go-gost/x/metadata"
)
// TestInit_SystemNameservers covers the middle rung of the Init fallback
// chain: when neither hop nodes nor metadata DNS is configured, nameservers
// from the system config become "system-N" exchangers. It pins the parser
// behavior: comments and other directives are ignored, invalid IPs are
// skipped, a missing port defaults to 53, and an explicit port is kept.
func TestInit_SystemNameservers(t *testing.T) {
path := filepath.Join(t.TempDir(), "resolv.conf")
content := "# a comment\n" +
"; another comment\n" +
"search example.com\n" +
"nameserver not-an-ip\n" +
"nameserver 1.1.1.1\n" +
"nameserver 8.8.8.8:5353\n" +
"nameserver ::1\n" +
"options ndots:5\n"
if err := os.WriteFile(path, []byte(content), 0o644); err != nil {
t.Fatal(err)
}
restore := resolvConfPath
resolvConfPath = path
t.Cleanup(func() { resolvConfPath = restore })
h := newTestHandler()
if err := h.Init(xmd.NewMetadata(nil)); err != nil {
t.Fatalf("Init: %v", err)
}
if len(h.exchangers) != 3 {
t.Fatalf("len(exchangers) = %d, want 3", len(h.exchangers))
}
if got, want := h.exchangers["system-0"].String(), "udp://1.1.1.1:53"; got != want {
t.Errorf("system-0 = %q, want %q", got, want)
}
if got, want := h.exchangers["system-1"].String(), "udp://8.8.8.8:5353"; got != want {
t.Errorf("system-1 = %q, want %q", got, want)
}
if got, want := h.exchangers["system-2"].String(), "udp://[::1]:53"; got != want {
t.Errorf("system-2 = %q, want %q", got, want)
}
}
+5
View File
@@ -1,5 +1,10 @@
package dns
// resolvConfPath is unused on Windows — systemNameservers never reads a file
// here — but it stays declared so tests that redirect it (to exercise the
// Init fallback chain) compile on every platform.
var resolvConfPath string
// systemNameservers returns nil on Windows because reading the system DNS
// configuration requires the Windows IP Helper API. Callers should fall
// back to the hardcoded default when this returns nil.
+12 -38
View File
@@ -72,45 +72,19 @@ func (h *forwardHandler) sniffingDial(ctx context.Context, network, address stri
}
// handleSniffedProtocol dispatches a sniffed connection to the protocol-specific
// sniffer (HandleHTTP or HandleTLS). It returns (true, err) when the protocol was
// sniffer registered for proto. It returns (true, err) when the protocol was
// handled and (false, nil) when the caller should fall through to raw forwarding.
func (h *forwardHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS, sniffing.ProtoRedis:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithHTTPKeepalive(h.md.httpKeepalive),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
if proto == sniffing.ProtoTLS {
return true, sniffer.HandleTLS(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
return true, sniffer.HandleRedis(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
return forwarder.Dispatch(h.sniffer.Build(), ctx, conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithHop(h.getHop()),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithHTTPKeepalive(h.md.httpKeepalive),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+9
View File
@@ -93,6 +93,15 @@ func (h *forwardHandler) handleRawForwarding(ctx context.Context, conn net.Conn,
// metadata when set, then falling back to hop selection.
func (h *forwardHandler) selectTarget(ctx context.Context, proto string) *chain.Node {
if host := mdutil.GetString(ictx.MetadataFromContext(ctx), "host"); host != "" {
// Route the tunneled hostname through the hop so a node with a matching
// filter.host is selected (the node's addr is the real target). If the
// hop has no matching node (e.g. no filter configured), fall back to
// dialing the hostname directly — the pre-regression behavior.
if curHop := h.getHop(); curHop != nil {
if node := curHop.Select(ctx, hop.ProtocolSelectOption(proto), hop.HostSelectOption(host)); node != nil {
return node
}
}
return &chain.Node{Addr: host}
}
if curHop := h.getHop(); curHop != nil {
+46 -6
View File
@@ -201,7 +201,52 @@ func TestHandleRawForwarding_MarkerReset(t *testing.T) {
}
}
func TestHandleRawForwarding_ContextHost(t *testing.T) {
// TestHandleRawForwarding_ContextHostHop: a tunneled hostname with a hop whose
// node matches the host filter routes to that node's addr (the documented
// filter.host client routing). The node addr is the real target.
func TestHandleRawForwarding_ContextHostHop(t *testing.T) {
server, client := net.Pipe()
defer server.Close()
defer client.Close()
var gotHost string
h := newInitdHandler(withRouter(&mockRouter{
opts: &chain.RouterOptions{},
dialFn: func(ctx context.Context, network, address string) (net.Conn, error) {
return server, nil
},
}))
h.Forward(&mockHop{
selectFn: func(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
var so hop.SelectOptions
for _, o := range opts {
o(&so)
}
gotHost = so.Host
return &chain.Node{Name: "example", Addr: "10.0.0.2:80"}
},
})
ctx := ictx.ContextWithMetadata(context.Background(), xmd.NewMetadata(map[string]any{
"host": "example.local:80",
}))
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(ctx, conn, ro, nopLog(), "tcp", "")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if gotHost != "example.local:80" {
t.Errorf("expected host passed to hop selection, got '%s'", gotHost)
}
if ro.Host != "10.0.0.2:80" {
t.Errorf("expected node addr to be dialed, got '%s'", ro.Host)
}
}
// TestHandleRawForwarding_ContextHostNoHop: when there is no hop, the tunneled
// hostname is dialed directly (fallback preserving the pre-regression behavior).
func TestHandleRawForwarding_ContextHostNoHop(t *testing.T) {
server, client := net.Pipe()
defer server.Close()
defer client.Close()
@@ -212,11 +257,6 @@ func TestHandleRawForwarding_ContextHost(t *testing.T) {
return server, nil
},
}))
h.Forward(&mockHop{
selectFn: func(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
return &chain.Node{Addr: "10.0.0.2:80"}
},
})
ctx := ictx.ContextWithMetadata(context.Background(), xmd.NewMetadata(map[string]any{
"host": "10.0.0.99:8080",
}))
+12 -38
View File
@@ -72,45 +72,19 @@ func (h *forwardHandler) sniffingDial(ctx context.Context, network, address stri
}
// handleSniffedProtocol dispatches a sniffed connection to the protocol-specific
// sniffer (HandleHTTP or HandleTLS). It returns (true, err) when the protocol was
// sniffer registered for proto. It returns (true, err) when the protocol was
// handled and (false, nil) when the caller should fall through to raw forwarding.
func (h *forwardHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS, sniffing.ProtoRedis:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithHTTPKeepalive(h.md.httpKeepalive),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
if proto == sniffing.ProtoTLS {
return true, sniffer.HandleTLS(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
return true, sniffer.HandleRedis(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
return forwarder.Dispatch(h.sniffer.Build(), ctx, conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithHop(h.getHop()),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithHTTPKeepalive(h.md.httpKeepalive),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+13 -30
View File
@@ -78,13 +78,15 @@ func (h *httpHandler) handleConnect(ctx context.Context, conn net.Conn, ro *xrec
}
if h.md.sniffing {
snifferHandled, err = h.sniffAndHandle(ctx, conn, cc, ro, log)
snifferHandled, err = h.sniffAndHandle(ictx.ContextWithSession(ctx, nil), conn, cc, ro, log)
if snifferHandled {
ro.Time = time.Time{}
return err
}
}
ictx.SessionFromContext(ctx).Start(*ro)
start := time.Now()
log.Infof("%s <-> %s", conn.RemoteAddr(), addr)
xnet.Pipe(ctx, conn, cc, xnet.WithReadTimeout(h.md.idleTimeout))
@@ -128,35 +130,14 @@ func (h *httpHandler) sniffAndHandle(ctx context.Context, conn net.Conn, cc net.
sniffer := h.sniffer.Build()
conn = xnet.NewReadWriteConn(br, conn, conn)
switch proto {
case sniffing.ProtoHTTP:
return true, sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoTLS:
return true, sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoRedis:
return true, sniffer.HandleRedis(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
return false, nil
return sniffing.Dispatch(sniffer, ctx, "tcp", conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
// dial establishes an upstream connection through the proxy chain router.
@@ -235,6 +216,7 @@ type SnifferBuilder struct {
Websocket bool
WebsocketSampleRate float64
Recorder recorder.Recorder
SessionRecorder *xrecorder.SessionRecorder
RecorderOptions *recorder.Options
Certificate *x509.Certificate
PrivateKey crypto.PrivateKey
@@ -255,6 +237,7 @@ func (b *SnifferBuilder) Build() *sniffing.Sniffer {
Websocket: b.Websocket,
WebsocketSampleRate: b.WebsocketSampleRate,
Recorder: b.Recorder,
SessionRecorder: b.SessionRecorder,
RecorderOptions: b.RecorderOptions,
Certificate: b.Certificate,
PrivateKey: b.PrivateKey,
+22 -12
View File
@@ -210,6 +210,7 @@ import (
"github.com/go-gost/core/recorder"
xbypass "github.com/go-gost/x/bypass"
xctx "github.com/go-gost/x/ctx"
ictx "github.com/go-gost/x/internal/ctx"
xnet "github.com/go-gost/x/internal/net"
xhttp "github.com/go-gost/x/internal/net/http"
"github.com/go-gost/x/internal/util/httpcache"
@@ -231,16 +232,17 @@ func init() {
// parsed metadata, handler options, and runtime state such as the traffic
// limiter, recorder, certificate pool, and the upstream HTTP transport.
type httpHandler struct {
md metadata // parsed configuration
options handler.Options // handler options from the service config
auth *Authenticator // auth + probe resistance (constructed in Init)
sniffer *SnifferBuilder // builds sniffing.Sniffer per connection
stats *stats_util.HandlerStats // per-client stats, created when Observer is set
limiter traffic.TrafficLimiter // per-client traffic shaper (cached)
cancel context.CancelFunc // cancels the observeStats goroutine
recorder recorder.RecorderObject // first matching service-handler recorder
certPool tls_util.CertPool // in-memory cert pool for MITM TLS termination
transport http.RoundTripper // upstream HTTP transport (injectable for tests)
md metadata // parsed configuration
options handler.Options // handler options from the service config
auth *Authenticator // auth + probe resistance (constructed in Init)
sniffer *SnifferBuilder // builds sniffing.Sniffer per connection
stats *stats_util.HandlerStats // per-client stats, created when Observer is set
limiter traffic.TrafficLimiter // per-client traffic shaper (cached)
cancel context.CancelFunc // cancels the observeStats goroutine
recorder recorder.RecorderObject // first matching service-handler recorder
sessionRecorder *xrecorder.SessionRecorder // reports live sessions on an interval when recorder.period is set.
certPool tls_util.CertPool // in-memory cert pool for MITM TLS termination
transport http.RoundTripper // upstream HTTP transport (injectable for tests)
}
// NewHandler creates a new HTTP handler and applies the given options.
@@ -295,10 +297,15 @@ func (h *httpHandler) Init(md md.Metadata) error {
Log: h.options.Logger,
}
h.sessionRecorder = xrecorder.NewSessionRecorder(h.recorder.Recorder, xrecorder.SessionRecorderOptions{
Period: h.md.recorderPeriod,
})
h.sniffer = &SnifferBuilder{
Websocket: h.md.sniffingWebsocket,
WebsocketSampleRate: h.md.sniffingWebsocketSampleRate,
Recorder: h.recorder.Recorder,
SessionRecorder: h.sessionRecorder,
RecorderOptions: h.recorder.Options,
Certificate: h.md.certificate,
PrivateKey: h.md.privateKey,
@@ -364,6 +371,9 @@ func (h *httpHandler) Handle(ctx context.Context, conn net.Conn, opts ...handler
pStats := xstats.Stats{}
conn = stats_wrapper.WrapConn(conn, &pStats)
session := h.sessionRecorder.NewSession(ctx, &pStats)
ctx = ictx.ContextWithSession(ctx, session)
defer func() {
if err != nil {
ro.Err = err.Error()
@@ -371,7 +381,7 @@ func (h *httpHandler) Handle(ctx context.Context, conn net.Conn, opts ...handler
ro.InputBytes = pStats.Get(stats.KindInputBytes)
ro.OutputBytes = pStats.Get(stats.KindOutputBytes)
ro.Duration = time.Since(start)
if err := ro.Record(ctx, h.recorder.Recorder); err != nil {
if err := session.Finish(ctx, *ro); err != nil {
log.Error("record: %v", err)
}
@@ -539,7 +549,7 @@ func (h *httpHandler) Close() error {
if h.cancel != nil {
h.cancel()
}
return nil
return h.sessionRecorder.Close()
}
// buildHTTPRecorder creates an HTTPRecorderObject from the request metadata.
+4
View File
@@ -23,6 +23,8 @@ const (
// Fields are populated by parseMetadata and read throughout the handler's
// request-processing methods.
type metadata struct {
recorderPeriod time.Duration // reports a live session on this interval; 0 = one record per session, written when it ends.
readTimeout time.Duration // deadline for upstream response headers (http.Transport.ResponseHeaderTimeout); 0=15s default, negative=disabled
idleTimeout time.Duration // idle read deadline per Pipe direction during CONNECT/forwarding; 0 or negative = disabled
keepalive bool // enable HTTP keep-alive on the upstream transport
@@ -59,6 +61,8 @@ type metadata struct {
// observerPeriod (5s, min 1s), and proxyAgent ("gost/3.0"). MITM TLS is
// enabled when both mitm.certFile and mitm.keyFile are provided.
func (h *httpHandler) parseMetadata(md mdata.Metadata) error {
h.md.recorderPeriod = mdutil.GetDuration(md, "recorder.period", "recorder.reportPeriod")
h.md.readTimeout = mdutil.GetDuration(md, "readTimeout")
if h.md.readTimeout == 0 {
h.md.readTimeout = 15 * time.Second
+31 -3
View File
@@ -8,13 +8,14 @@ import (
"io"
"net"
"net/http"
"net/http/httptrace"
"net/http/httputil"
"strings"
"time"
"github.com/go-gost/core/bypass"
stats "github.com/go-gost/core/observer/stats"
"github.com/go-gost/core/logger"
stats "github.com/go-gost/core/observer/stats"
xbypass "github.com/go-gost/x/bypass"
ictx "github.com/go-gost/x/internal/ctx"
xio "github.com/go-gost/x/internal/io"
@@ -98,6 +99,18 @@ func (h *httpHandler) proxyRoundTrip(ctx context.Context, rw io.ReadWriteCloser,
}
req.Header.Del("Gost-Record")
// Origin-form requests ("GET / HTTP/1.1" + Host header, e.g. sent by
// nginx proxy_pass) carry their target only in req.Host, which
// normalizeRequest doesn't mirror into the URL (it fills Scheme only,
// leaving URL.Host empty). Subsequent keep-alive requests parsed in
// handleProxy never pass through normalizeRequest at all. The
// http.Transport requires a non-empty URL.Host, otherwise it fails
// with "http: no Host in request URL" (#679).
if req.URL.Host == "" {
req.URL.Scheme = "http"
req.URL.Host = req.Host
}
host := normalizeHostPort(req.Host, "80")
ro.Host = host
ro.Time = time.Now()
@@ -106,6 +119,10 @@ func (h *httpHandler) proxyRoundTrip(ctx context.Context, rw io.ReadWriteCloser,
"host": host,
})
// A keep-alive connection carries several exchanges; each is accounted for
// on its own, under the connection's shared counters.
session := h.sessionRecorder.NewSession(ctx, pStats)
log.Infof("%s <-> %s", ro.RemoteAddr, req.Host)
defer func() {
if err != nil {
@@ -114,7 +131,7 @@ func (h *httpHandler) proxyRoundTrip(ctx context.Context, rw io.ReadWriteCloser,
ro.InputBytes = pStats.Get(stats.KindInputBytes)
ro.OutputBytes = pStats.Get(stats.KindOutputBytes)
ro.Duration = time.Since(ro.Time)
if err := ro.Record(ctx, h.recorder.Recorder); err != nil {
if err := session.Finish(ctx, *ro); err != nil {
log.Errorf("record: %v", err)
}
@@ -190,7 +207,18 @@ func (h *httpHandler) proxyRoundTrip(ctx context.Context, rw io.ReadWriteCloser,
ctx = ictx.ContextWithRecorderObject(ctx, ro)
ctx = ictx.ContextWithLogger(ctx, log)
resp, err := h.transport.RoundTrip(req.WithContext(ctx))
// Report from the moment the upstream connection is in hand, and only for
// this request: a recorder of its own would otherwise inherit the callback
// and overwrite the addresses with its own.
traceCtx := httptrace.WithClientTrace(ctx, &httptrace.ClientTrace{
GotConn: func(info httptrace.GotConnInfo) {
ro.SrcAddr = info.Conn.LocalAddr().String()
ro.DstAddr = info.Conn.RemoteAddr().String()
session.Start(*ro)
},
})
resp, err := h.transport.RoundTrip(req.WithContext(traceCtx))
if reqBody != nil {
ro.HTTP.Request.Body = reqBody.Content()

Some files were not shown because too many files have changed in this diff Show More