Compare commits

...
45 Commits
Author SHA1 Message Date
ginuerzh 93bd2ea760 fix(hop): parenthesize host OR and normalize wildcard in filter->matcher rule
filterToMatcherRule produced `Host(x) || Host(.x) && Proto(...) && ...`,
but the matcher DSL (vulcand/predicate) binds && tighter than ||, so the
apex Host(x) branch bypassed the protocol/path conjuncts — a request to
the apex with a non-matching protocol/path matched where the legacy
filter (AND of host/protocol/path) rejected it.

Also normalize `*.example.com` to `.example.com` before the leading-dot
branch: legacy ParseNode did this so both forms matched apex+subdomains;
without it, `*.example.com` reached Host() as a wildcard matching only
subdomains, dropping apex coverage that `.example.com` still had.

- Parenthesize the host OR when protocol/path conjuncts follow.
- Strip leading `*` + ensure leading dot, mirroring legacy normalization.
- Update TestFilterToMatcherRule expectations; add dot/star combined cases.
- Add TestFilterToMatcherRule_Precedence: runtime guard that apex+wrong
  proto/path is rejected and apex/subdomain+valid is accepted, for both
  `.example.com` and `*.example.com` forms.
2026-08-26 21:02:50 +08:00
ginuerzh 9292e0af3e fix(tunnel, forward): route tunneled hostname through hop filter.host
The tunnel bind conns (bindConn/bindUDPConn) now expose their metadata (the
tunneled hostname) via the xctx.Context interface, so the service accept loop
forwards it into the handler context. handleRawForwarding selects a hop node by
that hostname when a hop is configured, so a node with a matching filter.host is
picked; otherwise it falls back to dialing the hostname directly. Fixes
reverse-proxy-tunnel TCP routing (gost#898) where the docs example failed with
"node not available" because the mux path never set the host metadata.
2026-08-26 20:28:37 +08:00
ginuerzh 6ad7269644 refactor(hop): normalize deprecated node filter into matcher at parse time
Fold the deprecated per-node filter (host/protocol/path) into an equivalent
matcher DSL rule during config parsing, so hop.Select keeps a single
filtering path (routing.Matcher). Legacy filter configs stay accepted.

- filterToMatcherRule() maps host->Host(), protocol->Proto(), path->PathPrefix();
  a leading-dot host emits `Host(x) || Host(.x)` to preserve the legacy
  apex+subdomain coverage. Converted rules get priority=-1 to keep the
  always-through-selector behavior of the legacy path.
- Remove isEligible/checkHost/checkProtocol/checkPath from x/hop.
  core/ chain.NodeFilterSettings is left untouched (dead but harmless).
- Migrate hop tests to matchers; add TestFilterToMatcherRule.
2026-08-26 20:26:26 +08:00
ginuerzh 4ea4efaba9 fix(forwarder, sniffing): assert BasicProxy recording via captured output
Since 486a7766 HandleHTTP records on an internal clone of the passed
recorder object, TestHandleHTTP_BasicProxy's assertions on the original
ro were always empty, failing every run. Assert SrcAddr/DstAddr on the
captured recorder payload instead, and drain the response body before
closing the net.Pipe so HandleHTTP completes the write and exits cleanly.
2026-08-25 22:17:05 +08:00
ginuerzh faeede7881 feat(forwarder): delete JSON body fields via json-: rewrite match
Add a json-: match prefix to http.rewriteRequestBody/rewriteResponseBody
that deletes the matched JSON field (sjson.DeleteBytes) instead of
replacing it, keeping the existing value-regex conditional semantics.
2026-08-25 21:54:27 +08:00
ginuerzh a38a701dc4 refactor(sniffing): register protocol handlers via init() registries
Replace the 5 duplicated switch statements that dispatch sniffed
connections (forward/local, forward/remote, http CONNECT, sni, relay)
with per-package registries keyed by proto string. Protocol handlers
self-register via init(), following the house registration pattern, so
adding a protocol no longer requires touching every dispatch site.

Unify the two HandleOptions types (forwarder lacked dialTLS; sniffing
lacked hop/node/httpKeepalive) into one exported struct in sniffing;
forwarder aliases it. Unknown protocols fall through to raw forwarding
exactly as before; SSH remains detection-only. redirect/tcp's special
fallback-replay dispatch is intentionally left untouched.

Adds Dispatch helpers and 8 registry tests (register/get/duplicate/
unknown/known) in both packages. Net -151 lines.
2026-08-25 21:35:45 +08:00
ginuerzh 0132cefaec fix(api): save config back to the loaded file path (gost/x#123)
POST /config without an explicit path= always wrote gost.yaml/gost.json to
the process CWD, silently leaving the config loaded via -C untouched. Track
the last file read by ReadFile and default the save target to it, deriving
the output format from the file extension so a JSON config is saved back as
JSON. Falls back to the CWD default when no config file was loaded.
2026-08-21 21:43:17 +08:00
ginuerzh 25cd60d42d fix(connector/ss): own the target addr instead of aliasing the pooled buffer (gost#896)
WrapConn stores the target and only writes it on first I/O, but the
connector encoded it into a bufpool buffer that was returned to the pool
when Connect returned. A later Connect could reuse the backing array and
overwrite the bytes, misrouting the first connection. Allocate the target
per connection instead of sharing a pooled buffer.

Regression test drives two Connect calls and asserts the first conn's
wire target survives a second Connect reusing the pool.
2026-08-21 21:42:43 +08:00
ginuerzh f8ca4908f3 fix(hop): skip failed nodes before priority short-circuit, last-resort fallback
Failed nodes (http.failCodes marker) were still selected when the
priority short-circuit fired or when FailFilter saw a single-node tier,
so failover never happened for a high-priority matcher node returning
429. Check IsFailed first in the candidate-pool scan, and when every
candidate is failed return the first node matching the request gates as
a best-effort last resort instead of nil.

Refactors the fail-code check into shared selector.IsFailed (cee264e)
wired through FailFilterSettingsOption (64ea518).
2026-08-18 23:42:08 +08:00
ginuerzh cee264e184 refactor(selector): share IsFailed dead-node check with FailFilter
Extract the per-item failed-state decision (failure count >= maxFails,
last failure within failTimeout, metadata overrides) from FailFilter
into an exported IsFailed helper, and reuse it in the hop's
pre-selection dead-node filter instead of a local duplicate.
2026-08-18 23:21:34 +08:00
ginuerzh 64ea5184f8 fix(hop): drop marked-failed nodes before priority selection
http.failCodes (and probe) marking had no effect when the failing node
had a strictly higher matcher priority: the priority short-circuit
returned the top node directly, and FailFilter no-ops on a single-node
tier (len(vs) <= 1 guard), so the marked node was always re-selected
and failover never happened.

Pre-filter nodes marked failed (same Count >= maxFails && within
failTimeout criteria as FailFilter) out of the candidate pool before
priority selection. Single-node hops keep their lone node (safety net),
and an all-failed pool returns nil as before.

Wired maxFails/failTimeout from the hop selector config into the hop
via FailFilterSettingsOption.
2026-08-18 23:15:59 +08:00
ginuerzh fc5fc052e4 chore(deps): bump go-shadowsocks2 to v0.1.4
Pulls in the classic-AEAD UDP [addr][payload] response-framing fix that
restores shadowsocks UDP interop with third-party servers/clients
(go-shadowsocks2, shadowsocks-libev). Fixes go-gost/gost#894.
2026-08-18 22:17:20 +08:00
ginuerzh e6aeba8701 fix(listener/redirect/udp): set SO_REUSEADDR on tproxy listener socket (gost#413)
Match the dialUDP socket so per-packet tproxy sockets can bind a
destination port overlapping this listener without "address already in use".
2026-08-18 21:55:12 +08:00
ginuerzh cc02cff00c fix(logger): deduplicate keys across WithFields chains (gost#895)
slog.Logger.With does not deduplicate keys, so chained WithFields calls
(e.g. service -> listener/handler, each setting "kind") emitted duplicate
JSON keys. slogLogger now rebuilds from a pristine base logger with
accumulated attrs deduplicated by key, last write wins.
2026-08-18 21:49:11 +08:00
ginuerzh 18df26e9bc Merge branch 'master' of github.com:go-gost/x 2026-08-17 23:05:14 +08:00
Vincent Yang a9be7e6dce fix(masque): repair server CONNECT-UDP relay (#122)
The MASQUE handler's CONNECT-UDP path was broken in two ways, so no
standards-compliant client (e.g. Surge) could establish UDP tunnels:

1. Missing Capsule-Protocol response header. HTTPStream() flushes the
   response internally and commits the headers, but it was called before
   Header().Set("Capsule-Protocol", "?1"), so the mandatory RFC 9298
   header was silently dropped and clients rejected the tunnel.

2. WriteTo on a pre-connected socket. A direct UDP dial via the router
   returns a *net.UDPConn that satisfies net.PacketConn but is
   pre-connected; the relay called WriteTo on it, which fails with "use
   of WriteTo with pre-connected connection", so no packet reached the
   target.

Set the Capsule-Protocol header before hijacking the stream (dialing the
target first, matching handleConnectTCP), and always wrap the
router-returned conn so writes go through Conn.Write. Verified with a
standards-compliant HTTP/3 MASQUE client: CONNECT-UDP now returns
Capsule-Protocol: ?1 and datagrams relay end to end; CONNECT-TCP
unaffected.
2026-08-17 23:05:00 +08:00
ginuerzhandgithub 0e0eab0c52 fix(cmd): restore unix socket port-forward semantics
Commit b3b5986 crammed the URL path into the listen/connect address and
disabled forward mode for path-based protocols, breaking the
`unix://a.sock/b.sock` (listen on a, forward to b) port-forward form and
producing an invalid `a.sockb.sock` address.

Distinguish the two URL forms by host presence:
- three-slash (unix:///abs.sock, empty host) → path is the absolute socket
  address, no forward target;
- two-slash (unix://a.sock/b.sock) → host is the socket and the path is a
  forward target (service) or ignored (node).

Add regression tests for both forms.

Co-Authored-By: Claude (claude-opus-4-8[1m])
2026-08-17 22:51:00 +08:00
Dean Chen 98e9651361 cmd: keep absolute unix socket paths for path-based schemes (#121)
Path-based address selection used the dialer/listener type after registry
fallback, so unix remapped to tcp skipped EscapedPath and dropped
addrs like /var/run/docker.sock from three-slash URLs.

Decide path-based protocols from the original scheme components.

Signed-off-by: Dean Chen <862469039@qq.com>
2026-08-17 22:29:17 +08:00
PahaKush e4e36ca095 x: stream websocket reads instead of buffering each frame (#120)
websocketConn.Read called Conn.ReadMessage, which is NextReader plus
io.ReadAll. io.ReadAll starts at 512 bytes and grows by doubling, so every
frame allocated a fresh slice and a chain of intermediate ones: a stream of
64KB frames costs about 2.5 bytes of garbage per byte carried, even though
the caller already supplied a buffer to copy into.

On a relay+wss tunnel moving 750MB this was 2526KB of garbage per MB of
payload and 757 GC cycles in 15 seconds against a 1.5MB live heap. An
allocation profile attributed 99.92% of all process garbage to io.ReadAll
under ReadMessage; a CPU profile spent 15% in memclrNoHeapPointers and ~21%
in the collector, against 8% in AES-GCM.

Use NextReader and copy straight into the caller's buffer. The reader is
kept across calls and dropped at end of message, so a message larger than
the caller's buffer is still delivered across successive reads.

This also fixes a zero-length message surfacing as a (0, nil) read, which
io.Copy counts as no progress and rejects after 100 in a row.

Benchmark on 64KB frames:

    before  39267 ns/op  1668.97 MB/s  138444 B/op  19 allocs/op
    after   19920 ns/op  3289.98 MB/s      80 B/op   3 allocs/op
2026-08-17 22:28:48 +08:00
Niek van der Maas 746811eea5 fix: support SOCKS5 UDP associations over MASQUE (#119)
* fix: support SOCKS5 UDP over MASQUE

* fix: complete MASQUE SOCKS5 proxy support

* fix: isolate MASQUE UDP destinations
2026-08-17 22:28:20 +08:00
ginuerzh 7f72a81864 x: fix SSH authorized_keys parse failing on trailing newline (gost#699)
ParseAuthorizedKeysFile looped over ssh.ParseAuthorizedKey, but x/crypto's
parser splits on '\n' and returns rest='\n' for a trailing newline. The next
iteration called ParseAuthorizedKey('\n'), which skips whitespace and returns
'ssh: no key found', aborting the whole parse. Any normal authorized_keys file
(ends with newline, or has comments/blank lines) failed despite being valid.

Break the loop once rest is exhausted so all keys are parsed.
2026-08-01 18:24:10 +08:00
ginuerzh 2112532e0a x: set TCP_NODELAY on ws/wss/tls transports to cut relay UDP latency
Disable Nagle on the underlying TCP socket for relay+ws, relay+wss, and
relay+tls listeners/dialers. Small latency-sensitive datagrams (e.g.
WireGuard UDP over a relay tunnel) were being coalesced by Nagle until a
peer ACK returned, adding up to one RTT per datagram and inflating
round-trip latency (go-gost/gost#858). wstunnel sets TCP_NODELAY on its
tunnel socket by default and keeps latency flat; gost did not.

Add ApplyNoDelay helper and WrapNoDelayListener (mirrors ApplyKeepalive),
applied on the raw *net.TCPConn before the proxyproto/tls wrapper chain.
2026-08-01 16:59:54 +08:00
ginuerzh b2d901e70f x: add Redis TCP sniffing with protocol-based routing
Add RESP protocol detection to the sniffing framework so plaintext
Redis traffic on a port-reuse listener can be identified and routed by
protocol. Sniff() now peeks 1 byte first to match RESP type bytes
(*+-:$) without blocking, then falls through to the existing 5-byte
TLS/HTTP/SSH detection.

ParseRedisMetadata consumes exactly one RESP message via io.TeeReader
and records command/key in HandlerRecorderObject.Redis. HandleRedis in
both the sniffing (injected-dial) and forwarder (hop-selected) packages
replays the captured first message and pipes the rest, so pipelined
commands beyond the first are forwarded transparently.

Wired into all 5 dispatch sites: sni, forward/local, forward/remote,
http CONNECT, and relay tunnel handlers. forwarder paths route via
resolveRedisNode using ProtocolSelectOption(ProtoRedis), enabling
`protocol: redis` hop selection.

Part of two-phase plan (2026-07-29-sniffing-redis-postgres.md);
PostgreSQL deferred to phase 2.
2026-08-01 16:03:23 +08:00
ginuerzh b106f764a6 x: bump quic-dissector to v0.1.0, remove merged plan docs
Bump quic-dissector dependency from pseudo-version to tagged v0.1.0.
Remove docs/plans that have been implemented and merged.
2026-08-01 16:03:23 +08:00
ginuerzh edb0b5d651 x: extract shared QUIC sniffing helpers, add redirect UDP QUIC sniff
Extract PopulateQUICClientHello/PopulateQUICServerHello into
internal/util/sniffing/quic.go to deduplicate the QUIC ClientHello/ServerHello
population logic that was copy-pasted across forward/local (Handle, handleRawDatagram)
and redirect/udp handlers (3 sites → 2 helpers).

- handler/redirect/udp: add QUIC ClientHello + ServerHello sniff with
  sniffing/sniffing.timeout metadata, populating ro.Proto/ro.TLS for recording.
- handler/forward/local: replace inline QUIC blocks with shared helpers;
  use h.md.bufferSize instead of hardcoded 8192 for sniff reads.
- internal/util/sniffing: add ProtoQUIC constant.

Bump quic-dissector dependency to v0.1.0.
2026-08-01 16:03:23 +08:00
dependabot[bot] e3e41abd66 build(deps): bump github.com/pion/dtls/v3 from 3.1.1 to 3.1.4 (#118)
Bumps [github.com/pion/dtls/v3](https://github.com/pion/dtls) from 3.1.1 to 3.1.4.
- [Release notes](https://github.com/pion/dtls/releases)
- [Commits](https://github.com/pion/dtls/compare/v3.1.1...v3.1.4)

---
updated-dependencies:
- dependency-name: github.com/pion/dtls/v3
  dependency-version: 3.1.4
  dependency-type: direct:production
...

Signed-off-by: dependabot[bot] <support@github.com>
Co-authored-by: dependabot[bot] <49699333+dependabot[bot]@users.noreply.github.com>
2026-08-01 15:59:34 +08:00
Anivice IvesandYour Name 221144fab3 fixed the issue where the pull request to exit and subsequent application writes to fail immediately in gost (#117)
Co-authored-by: Your Name <you@example.com>
2026-08-01 15:59:08 +08:00
ginuerzh 6b1c6757f6 x: record cache-hit status in HTTP recorder object 2026-07-29 23:02:18 +08:00
ginuerzh 3157436140 x: add Redis backend for cache
Adds a RedisCache config block and redisCache implementation for the
generic cache.Cache interface, enabling distributed HTTP response
caching across gost instances.

go-redis/redis/v8 was already a dependency (used by recorder and
internal loader) — no new imports needed.
2026-07-29 22:39:19 +08:00
ginuerzh 0ab5363f60 x: distinguish client vs server errors in service handler loop
The service Serve loop counted every handler error — including client-side
disconnects (io.EOF, net.ErrClosed, context cancellation) and policy
decisions (rate limiting, bypass) — as server handler errors, inflating
the error counter and log noise. isServerError() now gates the error-level
log, error counter, and stats increment; non-server errors are logged at
debug level only.
2026-07-29 21:44:12 +08:00
ginuerzh 88670baeae x: don't overwrite ro.Host when TLS SNI is empty
The previous empty-SNI fix clobbered ro.Host with the listener's local
address. For handlers that set ro.Host before HandleTLS (SOCKS5 via the
CONNECT address, SS via target), that destroyed the real destination used
for recording and bypass. Now ro.Host is only set when SNI is present;
the caller's pre-set value is left intact otherwise.
2026-07-29 21:42:56 +08:00
ginuerzh 226420a622 x: allow empty TLS SNI in sniffing handlers
Clients connecting to HTTPS by raw IP (e.g. https://1.1.1.1) don't
send SNI per RFC 6066. Both HandleTLS methods errored immediately
on empty SNI, closing the connection. Now they fall through to the
normal routing path — hop selection in the forwarder, or the caller's
dial closure in the sniffing sniffer (e.g. HTTP CONNECT) — which
handles the empty host gracefully or fails naturally.
2026-07-28 23:52:29 +08:00
ginuerzh 3cd4fad4bf x: close forwarder hop probe goroutines on service shutdown
Same leak class as the previous chainGroup fix: a forwarder built from
a hopGroup or inline nodes starts probe goroutines at parse time, but
nothing closed them — every SIGHUP reload leaked them permanently.

- parseForwarder returns an io.Closer for service-owned hops (hop
  group or inline nodes); registry-owned named hops return nil since
  the registry closes them on unregister
- ParseService registers the closer via ClosersOption and closes it
  when construction fails partway
- hopGroup.Close no longer closes the underlying hops: they come from
  the HopRegistry and are shared with other referents; closing them
  here would kill shared state
2026-07-28 23:45:40 +08:00
ginuerzh 291fa48fae x: add -race test coverage for chainGroup matcher/probe/close paths
Adds tests for the new chainGroup code paths the matcher+probe review flagged
as untested: matcher filtering in selectChain, probe goroutine cleanup on
Close(), and probe marking the entry marker on cmd failure. All run clean under
`CGO_ENABLED=1 go test -race`.
2026-07-28 23:36:08 +08:00
ginuerzh 6e9ff675b5 x: extract duplicated cmd-probe logic into probe.RunCmdProbe
chainGroup and hopGroup had verbatim copies of the cmd-probe block (timeout
default, CmdProber run, marker Mark/Reset, log). Move it to
internal/probe.RunCmdProbe(cfg, marker, log) so both call sites share one
implementation (cleanup5 from the chainGroup review).
2026-07-28 23:31:15 +08:00
ginuerzh 506edffc93 x: skip eligible allocation when no matchers; always run FailFilter
In selectChain: precompute whether any entry has a matcher and, when none do,
pass entries straight to the selector instead of allocating a new slice on every
Route() call. Drop the single-eligible-entry shortcut so the selector's
FailFilter always runs and can exclude probe-failed chains (design6).

Combined fix for efficiency4 (hot-path allocation) and design6 (single-match
bypass of FailFilter/probe markers) from the chainGroup review.
2026-07-28 23:26:54 +08:00
ginuerzh 6248d4c0f8 x: derive probe goroutine context from lifecycle ctx (bug2)
probeEntry used context.Background() for Route/Dial, so an in-flight probe
could not be cancelled by Close()/reload. Thread the lifecycle ctx through
probeEntry so a stuck route aborts when the group is torn down.

Also promote mapstructure to a direct dependency (now imported by config.go).
2026-07-28 23:25:28 +08:00
ginuerzh 50f4a6b096 x: close chainGroup probe goroutines on service shutdown
chainGroup.Close() existed but was never called: the group is wrapped
in a chain.Router held by the listener/handler, and service.Close()
only closed the handler, observer, and listener. Every SIGHUP reload
leaked the old config's probe goroutines permanently.

Wire the close path end to end:
- Router.Close() closes its chain when it implements io.Closer
- service ClosersOption registers io.Closers called on shutdown
- ParseService passes both routers via ClosersOption, and closes
  them on construction failure so probes started early don't leak
2026-07-28 23:14:27 +08:00
ginuerzhandClaude 30ba6d64a5 x: add matcher + probe to chainGroup with independent entry marker
chainGroup now selects chains in two stages: a matcher filter (by host/network)
builds an eligible pool, then the group selector applies FailFilter + strategy.
Each entry carries its own failure marker so chain-level probe marking is
independent of node-level failCodes markers (probe Reset() can't undo a node
failCodes Mark()).

Config gains ChainGroupEntry (chain/matcher/probe) replacing the old []string,
with a mapstructure decode hook preserving backward-compatible "chains: [a, b]"
string lists and time.Duration string parsing. Probe goroutines run per-entry
and mark the entry on failure.

Co-Authored-By: Claude <noreply@anthropic.com>
2026-07-28 21:20:34 +08:00
ginuerzh 72db96d4cd x: add hopGroup for conditional hop-level selection with probe/failCodes marker separation
hopGroup wraps named hops with per-entry matcher + probe, implementing
hop.Hop for drop-in compatibility with the existing Forwarder.Forward().

Core design: hopEntry.marker (probe, hop-level) is independent of
node.Marker() (failCodes, node-level) so probe Reset() cannot undo a
failCodes Mark() — the two mechanisms no longer conflict on the single
chain.Node marker.

- hop/group.go: HopEntry + hopGroup + probe infrastructure
- config/config.go: ForwardHopGroupConfig, ForwardHopConfig types
- config/parsing/service/parse.go: parseHopGroup, HopGroup > Hop > Name > Nodes precedence
- config/parsing/node/parse.go: export parseProbeConfig → ParseProbeConfig
2026-07-27 22:08:34 +08:00
ginuerzh 8edf82af9b hop: fix backup-priority interaction — primary beats backup regardless of priority
Stage 3 shortcut now gates on isBackupNode(maxPriNode) instead of
hasBackup — the old check disabled priority short-circuit whenever any
backup existed anywhere in the pool, even when the winning node was a
primary.

Stage 4 tier loop gains hasAnyPrimary scan + backup-tier skip — a
lower-priority primary always beats a higher-priority backup tier.
Nil-selector path now returns the first node of the first non-skipped
tier instead of unconditionally returning maxPriNode.
2026-07-27 22:06:37 +08:00
ginuerzh 839f2a280b chore: bump deps (tls-dissector v0.3.1, quic-go v0.60.0, webtransport-go v0.11.1) 2026-07-26 22:58:52 +08:00
ginuerzh c23a2f1e53 fix(redirect/tcp): sniffingFallback with empty SNI falls through to raw forwarding
When sniffing detects TLS but the ClientHello has no SNI, HandleTLS
errors with 'tls: sni is empty, closing connection'. Previously this
error was returned unconditionally even with sniffingFallback enabled.

Now, bytes consumed by the sniffer are captured via io.TeeReader and
replayed through io.MultiReader when sniffingFallback is true, allowing
the connection to fall through to raw forwarding to the original dstAddr.
2026-07-26 22:27:44 +08:00
ginuerzh 0ffc000d0d fix(forwarder/sniffer): update ho.recorderObject after ro copy to prevent nil HTTP panic
The HandleHTTP ro clone (486a776) only updated the local ro variable, but
dial() reads ho.recorderObject directly, getting the original with nil HTTP
field, causing nil pointer dereference at ro.HTTP.StatusCode access.
2026-07-26 22:09:55 +08:00
ginuerzh 486a77665d fix: prevent double-recording when HTTP response cache is enabled
The cache-hit path (serveCachedResponse) recorded on the same HandlerRecorderObject
as the calling handler's deferred Record(), producing two records per request.
Copy ro at HandleHTTP entry so the sniffer's internal recording stays isolated
from the caller's defer. Also zero ro.Time when the sniffer takes over, so the
handler's defer Record() skips the now-redundant connection-level record.

Fixes 10 files across sniffer and all 8 handlers that support sniffed HTTP forwarding.
2026-07-26 20:11:36 +08:00
85 changed files with 4288 additions and 1358 deletions
+29 -11
View File
@@ -5,6 +5,8 @@ import (
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"github.com/gin-gonic/gin"
"github.com/go-gost/core/observer/stats"
@@ -127,10 +129,11 @@ func getConfig(ctx *gin.Context) {
// swagger:parameters saveConfigRequest
type saveConfigRequest struct {
// output format, one of yaml|json, default is yaml.
// output format, one of yaml|json, default is derived from the file extension.
// in: query
Format string `form:"format" json:"format"`
// file path, default is gost.yaml|gost.json in current working directory.
// file path, default is the loaded config file, or gost.yaml|gost.json in
// the current working directory when no config file was loaded.
// in: query
Path string `form:"path" json:"path"`
}
@@ -155,16 +158,31 @@ func saveConfig(ctx *gin.Context) {
var req saveConfigRequest
ctx.ShouldBindQuery(&req)
file := "gost.yaml"
switch req.Format {
case "json":
file = "gost.json"
default:
req.Format = "yaml"
file := req.Path
if file == "" {
// Default to the config file that was loaded (via -C), falling back to
// the current working directory when the config came from elsewhere.
file = config.ConfigFile()
}
if req.Path != "" {
file = req.Path
if file == "" {
file = "gost.yaml"
if req.Format == "json" {
file = "gost.json"
}
}
switch strings.ToLower(req.Format) {
case "json":
req.Format = "json"
case "yaml":
req.Format = "yaml"
default:
// Derive the format from the file extension so a loaded JSON config
// is saved back as JSON (ReadFile forces format from extension).
if strings.ToLower(filepath.Ext(file)) == ".json" {
req.Format = "json"
} else {
req.Format = "yaml"
}
}
f, err := os.Create(file)
+119
View File
@@ -0,0 +1,119 @@
package cache
import (
"context"
"time"
"github.com/go-gost/core/cache"
"github.com/go-gost/core/logger"
"github.com/go-redis/redis/v8"
)
// defaultKeyPrefix is prepended to cache keys when RedisOptions.KeyPrefix is empty.
const defaultKeyPrefix = "gost:cache:"
// RedisOptions configures a Redis cache.
type RedisOptions struct {
// Addr is the Redis server address (host:port).
Addr string
// DB is the Redis database number.
DB int
// Username for Redis authentication. Optional.
Username string
// Password for Redis authentication. Optional.
Password string
// DefaultTTL is applied to entries stored without an explicit TTL.
// A non-positive value means entries never expire by default.
DefaultTTL time.Duration
// KeyPrefix is prepended to all cache keys. Defaults to "gost:cache:".
KeyPrefix string
// Logger is used for debug logging. Optional.
Logger logger.Logger
}
// redisCache is a cache.Cache backed by a Redis server.
//
// Redis expires keys natively, so an expired entry is a true miss here — the
// interface's serve-stale contract (returning expired entries from Get) does
// not apply to this backend.
type redisCache struct {
client *redis.Client
defaultTTL time.Duration
keyPrefix string
logger logger.Logger
}
// NewRedisCache creates a Redis-backed cache. Close releases the underlying
// client connections.
func NewRedisCache(opts RedisOptions) cache.Cache {
keyPrefix := opts.KeyPrefix
if keyPrefix == "" {
keyPrefix = defaultKeyPrefix
}
return &redisCache{
client: redis.NewClient(&redis.Options{
Addr: opts.Addr,
Username: opts.Username,
Password: opts.Password,
DB: opts.DB,
}),
defaultTTL: opts.DefaultTTL,
keyPrefix: keyPrefix,
logger: opts.Logger,
}
}
// Get returns the entry for key, or cache.ErrNotFound if the key is absent
// (including keys Redis has already expired).
func (c *redisCache) Get(ctx context.Context, key string) (*cache.Entry, error) {
k := c.keyPrefix + key
pipe := c.client.Pipeline()
getCmd := pipe.Get(ctx, k)
ttlCmd := pipe.PTTL(ctx, k)
if _, err := pipe.Exec(ctx); err != nil {
if err == redis.Nil {
return nil, cache.ErrNotFound
}
return nil, err
}
entry := &cache.Entry{
Data: []byte(getCmd.Val()),
}
if ttl := ttlCmd.Val(); ttl > 0 {
entry.Expiration = time.Now().Add(ttl)
}
return entry, nil
}
// Set stores data under key. A non-positive TTL (after applying the default)
// stores the entry without expiration.
func (c *redisCache) Set(ctx context.Context, key string, data []byte, opts ...cache.SetOption) error {
var options cache.SetOptions
for _, opt := range opts {
opt(&options)
}
ttl := options.TTL
if ttl <= 0 {
ttl = c.defaultTTL
}
if ttl < 0 {
ttl = 0 // redis: 0 = no expiration
}
return c.client.Set(ctx, c.keyPrefix+key, data, ttl).Err()
}
// Delete removes key from the cache. Deleting an absent key is a no-op.
func (c *redisCache) Delete(ctx context.Context, key string) error {
return c.client.Del(ctx, c.keyPrefix+key).Err()
}
// Close releases the Redis client. Implements io.Closer so the registry
// closes the client on hot reload.
func (c *redisCache) Close() error {
return c.client.Close()
}
+185 -8
View File
@@ -31,12 +31,15 @@ package chain
import (
"context"
"io"
"time"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/metadata"
"github.com/go-gost/core/routing"
"github.com/go-gost/core/selector"
"github.com/go-gost/x/internal/probe"
)
var (
@@ -160,14 +163,20 @@ func (c *Chain) Route(ctx context.Context, network, address string, opts ...chai
}
type chainGroup struct {
chains []chain.Chainer
selector selector.Selector[chain.Chainer]
entries []*ChainEntry
selector selector.Selector[chain.Chainer]
logger logger.Logger
cancelFunc context.CancelFunc
}
// NewChainGroup creates a chain group that selects one Chainer from the
// given list using the configured selector (round-robin by default).
func NewChainGroup(chains ...chain.Chainer) *chainGroup {
return &chainGroup{chains: chains}
entries := make([]*ChainEntry, len(chains))
for i, c := range chains {
entries[i] = NewChainEntry(c, nil, nil)
}
return &chainGroup{entries: entries}
}
func (p *chainGroup) WithSelector(s selector.Selector[chain.Chainer]) *chainGroup {
@@ -175,17 +184,185 @@ func (p *chainGroup) WithSelector(s selector.Selector[chain.Chainer]) *chainGrou
return p
}
// WithGroupEntries sets the chain entries for the group (replaces defaults)
// and starts probe goroutines for entries that have probe configurations.
func (p *chainGroup) WithGroupEntries(entries ...*ChainEntry) *chainGroup {
// Cancel any previous probe goroutines before replacing entries.
if p.cancelFunc != nil {
p.cancelFunc()
}
p.entries = entries
ctx, cancel := context.WithCancel(context.Background())
p.cancelFunc = cancel
for _, e := range entries {
if e.probe != nil {
go p.runEntryProbe(ctx, e)
}
}
return p
}
// WithGroupLogger sets the logger for the group.
func (p *chainGroup) WithGroupLogger(log logger.Logger) *chainGroup {
p.logger = log
return p
}
func (p *chainGroup) Route(ctx context.Context, network, address string, opts ...chain.RouteOption) chain.Route {
if chain := p.next(ctx); chain != nil {
return chain.Route(ctx, network, address, opts...)
if c := p.selectChain(ctx, network, address, opts...); c != nil {
return c.Route(ctx, network, address, opts...)
}
return nil
}
func (p *chainGroup) next(ctx context.Context) chain.Chainer {
if p == nil || len(p.chains) == 0 {
func (p *chainGroup) selectChain(ctx context.Context, network, address string, opts ...chain.RouteOption) chain.Chainer {
if p == nil || len(p.entries) == 0 {
return nil
}
return p.selector.Select(ctx, p.chains...)
var options chain.RouteOptions
for _, opt := range opts {
opt(&options)
}
// When no entry has a matcher, the eligible pool equals all entries; the
// selector must still run (FailFilter excludes probe-failed chains), so
// skip the per-call allocation and pass entries directly.
anyMatcher := false
for _, e := range p.entries {
if e.matcher != nil {
anyMatcher = true
break
}
}
if !anyMatcher {
eligible := make([]chain.Chainer, len(p.entries))
for i, e := range p.entries {
eligible[i] = e
}
return p.selectFrom(ctx, eligible...)
}
// Stage 1: matcher filter — build eligible pool.
req := routing.Request{
Network: network,
Host: options.Host,
}
eligible := make([]chain.Chainer, 0, len(p.entries))
for _, e := range p.entries {
if e.matcher != nil {
if !e.matcher.Match(&req) {
continue
}
if p.logger != nil {
p.logger.Debugf("chain entry matched request %s %s", req.Network, req.Host)
}
}
eligible = append(eligible, e) // *ChainEntry as chain.Chainer, FailFilter reads e.Marker()
}
if len(eligible) == 0 {
return nil
}
// Stage 2: group selector — FailFilter + strategy.
return p.selectFrom(ctx, eligible...)
}
func (p *chainGroup) selectFrom(ctx context.Context, eligible ...chain.Chainer) chain.Chainer {
if p.selector != nil {
return p.selector.Select(ctx, eligible...)
}
return eligible[0]
}
// Close stops all probe goroutines.
func (p *chainGroup) Close() error {
if p.cancelFunc != nil {
p.cancelFunc()
}
return nil
}
// runEntryProbe periodically probes a chain and marks
// the chain entry (not the chain's nodes) on failure.
func (p *chainGroup) runEntryProbe(ctx context.Context, e *ChainEntry) {
cfg := e.probe
interval := cfg.Interval
if interval <= 0 {
interval = 30 * time.Second
}
p.probeEntry(ctx, e, cfg) // first probe immediately
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
p.probeEntry(ctx, e, cfg)
case <-ctx.Done():
return
}
}
}
func (p *chainGroup) probeEntry(ctx context.Context, e *ChainEntry, cfg *chain.ProbeConfig) {
// cmd probe runs a shell command — no chain routing needed.
if cfg.Type == chain.ProbeTypeCmd {
probe.RunCmdProbe(cfg, e.marker, p.logger)
return
}
// TCP/HTTP probe: route through the chain to probe target.
addr := cfg.Addr
if addr == "" {
e.marker.Mark()
if p.logger != nil {
p.logger.Debug("chain entry probe: no probe address configured")
}
return
}
// Route through the probe lifecycle context so cancellation (reload/close)
// aborts an in-flight probe instead of blocking forever on a stuck route.
route := e.chainer.Route(ctx, "tcp", addr)
if route == nil {
e.marker.Mark()
if p.logger != nil {
p.logger.Debug("chain entry probe: no route available")
}
return
}
timeout := cfg.Timeout
if timeout <= 0 {
timeout = 10 * time.Second
}
probeCtx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
conn, err := route.Dial(probeCtx, "tcp", addr)
if err != nil {
e.marker.Mark()
if p.logger != nil {
p.logger.Debugf("chain entry probe dial %s failed: %v", addr, err)
}
return
}
defer conn.Close()
if cfg.Type == chain.ProbeTypeHTTP {
if err := probe.NewHTTPProber(cfg).Probe(conn); err != nil {
e.marker.Mark()
if p.logger != nil {
p.logger.Debugf("chain entry probe http %s failed: %v", addr, err)
}
return
}
}
e.marker.Reset()
}
+77 -5
View File
@@ -15,6 +15,7 @@ import (
"github.com/go-gost/core/logger"
"github.com/go-gost/core/metadata"
"github.com/go-gost/core/recorder"
"github.com/go-gost/core/routing"
xlogger "github.com/go-gost/x/logger"
xmetadata "github.com/go-gost/x/metadata"
"github.com/stretchr/testify/assert"
@@ -787,7 +788,7 @@ func TestNewChainGroup(t *testing.T) {
ch2 := NewChain("ch2")
g := NewChainGroup(ch1, ch2)
require.NotNil(t, g)
assert.Len(t, g.chains, 2)
assert.Len(t, g.entries, 2)
}
func TestChainGroup_Route_NoChains(t *testing.T) {
@@ -1758,12 +1759,83 @@ func TestChainGroup_WithSelector(t *testing.T) {
assert.Len(t, rt.Nodes(), 1)
}
func TestChainGroup_Next_NilReceiver(t *testing.T) {
func TestChainGroup_SelectChain_NilReceiver(t *testing.T) {
var g *chainGroup
assert.Nil(t, g.next(context.Background()))
assert.Nil(t, g.selectChain(context.Background(), "", ""))
}
func TestChainGroup_Next_EmptyChains(t *testing.T) {
func TestChainGroup_SelectChain_EmptyChains(t *testing.T) {
g := NewChainGroup()
assert.Nil(t, g.next(context.Background()))
assert.Nil(t, g.selectChain(context.Background(), "", ""))
}
// hostMatcher matches requests whose Host equals the configured value.
type hostMatcher struct {
host string
}
func (m *hostMatcher) Match(req *routing.Request) bool {
return req != nil && req.Host == m.host
}
func TestChainGroup_SelectChain_MatcherFilter(t *testing.T) {
matchChain := NewChain("match")
skipChain := NewChain("skip")
entries := []*ChainEntry{
NewChainEntry(matchChain, &hostMatcher{host: "a.example.com"}, nil),
NewChainEntry(skipChain, &hostMatcher{host: "b.example.com"}, nil),
}
g := NewChainGroup().WithGroupEntries(entries...)
// Only the entry matching "a.example.com" should be eligible.
got := g.selectChain(context.Background(), "tcp", "a.example.com:80", chain.WithHostRouteOption("a.example.com"))
require.NotNil(t, got)
// selectChain returns the *ChainEntry (so the FailFilter can read its marker),
// not the inner *Chain.
ce, ok := got.(*ChainEntry)
require.True(t, ok)
assert.Equal(t, matchChain, ce.chainer)
// Non-matching host → no eligible entry → nil.
assert.Nil(t, g.selectChain(context.Background(), "tcp", "x.example.com:80", chain.WithHostRouteOption("x.example.com")))
}
func TestChainGroup_Close_StopsProbeGoroutines(t *testing.T) {
ch := NewChain("ch")
// cmd probe with a short interval; the entry marker count is the observable
// signal that the probe goroutine is (or is not) still running.
entry := NewChainEntry(ch, nil, &chain.ProbeConfig{
Type: chain.ProbeTypeCmd,
Command: "true",
Interval: 10 * time.Millisecond,
})
g := NewChainGroup().WithGroupEntries(entry)
// Let the probe tick a few times, then Close().
time.Sleep(50 * time.Millisecond)
require.NoError(t, g.Close())
before := entry.Marker().Count()
time.Sleep(60 * time.Millisecond)
after := entry.Marker().Count()
// A succeeding cmd probe ("true") never marks, so the count must stay
// stable after Close(): the goroutine has stopped ticking.
assert.Equal(t, before, after, "probe goroutine should stop ticking after Close()")
}
func TestChainGroup_Probe_MarksEntryOnCmdFailure(t *testing.T) {
ch := NewChain("ch")
entry := NewChainEntry(ch, nil, &chain.ProbeConfig{
Type: chain.ProbeTypeCmd,
Command: "false", // always exits non-zero → probe failure
Interval: time.Second,
})
g := NewChainGroup().WithGroupEntries(entry)
// First probe runs immediately; give it time to mark.
time.Sleep(50 * time.Millisecond)
assert.NotZero(t, entry.Marker().Count(), "failing cmd probe should mark the entry")
require.NoError(t, g.Close())
}
+44
View File
@@ -0,0 +1,44 @@
package chain
import (
"context"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/routing"
"github.com/go-gost/core/selector"
)
// ChainEntry wraps a chain.Chainer with an optional matcher and an independent
// failure marker for chain-level probe tracking. The entry's marker is
// separate from any node-level markers inside the chain so that probe
// Reset() cannot undo a node-level failCodes Mark().
type ChainEntry struct {
chainer chain.Chainer
matcher routing.Matcher
probe *chain.ProbeConfig
marker selector.Marker
}
// NewChainEntry creates a ChainEntry. matcher may be nil (unconditional
// catch-all). probe may be nil (no health check for this entry).
func NewChainEntry(c chain.Chainer, m routing.Matcher, probe *chain.ProbeConfig) *ChainEntry {
return &ChainEntry{
chainer: c,
matcher: m,
probe: probe,
marker: selector.NewFailMarker(),
}
}
// Route implements chain.Chainer so the entry can be passed directly to the
// group selector. The selector's FailFilter reads Marker() from the entry,
// not from the underlying chain — this keeps probe-driven marking independent
// of any node-level markers.
func (e *ChainEntry) Route(ctx context.Context, network, address string, opts ...chain.RouteOption) chain.Route {
return e.chainer.Route(ctx, network, address, opts...)
}
// Marker implements selector.Markable.
func (e *ChainEntry) Marker() selector.Marker {
return e.marker
}
+14
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"context"
"fmt"
"io"
"net"
"strings"
"time"
@@ -49,6 +50,19 @@ func (r *Router) Options() *chain.RouterOptions {
return &r.options
}
// Close releases resources held by the router's chain. A chain that owns
// background goroutines (e.g. a chainGroup running liveness probes) is closed
// here so its goroutines stop when the service shuts down or is reloaded.
func (r *Router) Close() error {
if r == nil {
return nil
}
if c, ok := r.options.Chain.(io.Closer); ok {
return c.Close()
}
return nil
}
// Dial establishes a connection to the target address through the configured
// chain. It resolves the address, records the dial event, retries up to
// Retries+1 times, and returns the resulting connection.
+38 -21
View File
@@ -397,15 +397,24 @@ func buildServiceConfig(url *url.URL) ([]*config.ServiceConfig, error) {
listener = schemes[1]
}
// For path-based protocols (Unix socket listeners and serial), use path as address
// Decide path-based from the scheme before registry remaps listener to tcp.
isPathBasedProtocol := listener == "unix" || listener == "runix" || listener == "serial"
for _, s := range schemes {
if s == "unix" || s == "runix" || s == "serial" {
isPathBasedProtocol = true
break
}
}
// A path-based protocol with an empty host (three-slash URL like
// unix:///var/run/sock) uses the path as the absolute listen address and
// has no forward target. With a non-empty host (two-slash URL like
// unix://sock/other.sock), the host is the listen address and any path is a
// forward target (port-forward to another socket), handled below.
isPathAddr := isPathBasedProtocol && url.Host == ""
var addrs []string
if isPathBasedProtocol {
path := url.EscapedPath()
if url.Host != "" {
addrs = append(addrs, url.Host+strings.TrimPrefix(path, "/"))
} else if path != "" {
if isPathAddr {
if path := url.EscapedPath(); path != "" {
addrs = append(addrs, path)
} else {
addrs = append(addrs, url.Host)
@@ -436,8 +445,10 @@ func buildServiceConfig(url *url.URL) ([]*config.ServiceConfig, error) {
var nodes []*config.ForwardNodeConfig
// forward mode
// For path-based protocols, path is address not forward target
if remotes := strings.Trim(url.EscapedPath(), "/"); remotes != "" && !isPathBasedProtocol {
// A path-based protocol with an empty host (e.g. unix:///abs.sock) treats
// the path as the listen address, not a forward target. With a host
// (e.g. unix://a.sock/b.sock) the path is a forward target.
if remotes := strings.Trim(url.EscapedPath(), "/"); remotes != "" && !isPathAddr {
i := 0
for _, addr := range strings.Split(remotes, ",") {
addrs := xnet.AddrPortRange(addr).Addrs()
@@ -640,6 +651,18 @@ func buildNodeConfig(url *url.URL, m map[string]any) (*config.NodeConfig, error)
dialer = schemes[1]
}
// Path-based address logic must use the requested scheme, not the
// registry fallback (unix/serial often remap to http/tcp when only
// the CLI registry is partially populated, or in unit tests).
isPathBasedProtocol := dialer == "unix" || dialer == "serial" ||
connector == "unix" || connector == "serial"
for _, s := range schemes {
if s == "unix" || s == "serial" || s == "runix" {
isPathBasedProtocol = true
break
}
}
md := mdx.NewMetadata(m)
if c := registry.ConnectorRegistry().Get(connector); c == nil {
@@ -725,20 +748,14 @@ func buildNodeConfig(url *url.URL, m map[string]any) (*config.NodeConfig, error)
nodeMd[k] = v
}
// For path-based protocols (Unix socket dialers and serial), use path as address
// For path-based protocols (Unix socket dialers and serial), use path as address.
// Three-slash URLs (empty host) give the absolute socket path; two-slash URLs
// use the host as the socket. The path on a host form is not part of the node
// address (it is a forward target for services, ignored for nodes).
var nodeAddr string
isPathBasedProtocol := dialer == "unix" || dialer == "serial"
if isPathBasedProtocol {
path := url.EscapedPath()
if url.Host != "" {
// Two slashes: host + path = relative path
nodeAddr = url.Host + strings.TrimPrefix(path, "/")
} else if path != "" {
// Three slashes: path is absolute (keep leading slash)
nodeAddr = path
} else {
nodeAddr = url.Host
}
if isPathBasedProtocol && url.Host == "" {
// Three slashes: path is absolute (keep leading slash)
nodeAddr = url.EscapedPath()
} else {
nodeAddr = url.Host
}
+62 -6
View File
@@ -1765,10 +1765,10 @@ func TestBuildNodeConfig_TLSWithCert(t *testing.T) {
}
}
func TestBuildNodeConfig_UnixFallsBackToTCP(t *testing.T) {
func TestBuildNodeConfig_UnixAbsolutePath(t *testing.T) {
// http+unix scheme: connector=http, dialer=unix.
// Since unix is not in the test registry, dialer falls back to tcp,
// and the path-based address logic (dialer=="unix") is skipped.
// Dialer falls back to tcp when unix is not in the test registry, but
// the absolute socket path from a three-slash URL must still be kept.
rawURL := "http+unix:///var/run/socket"
u, _ := url.Parse(rawURL)
@@ -1780,12 +1780,68 @@ func TestBuildNodeConfig_UnixFallsBackToTCP(t *testing.T) {
if node.Dialer.Type != "tcp" {
t.Errorf("Dialer.Type = %q, want tcp (unix not in test registry)", node.Dialer.Type)
}
// Addr is url.Host ("") since no host component in three-slash unix URL
if node.Addr != "" {
t.Errorf("Addr = %q, want empty", node.Addr)
if node.Addr != "/var/run/socket" {
t.Errorf("Addr = %q, want /var/run/socket", node.Addr)
}
}
func TestBuildNodeConfig_UnixRelativeHost(t *testing.T) {
u, _ := url.Parse("unix://docker.sock")
node, err := buildNodeConfig(u, map[string]any{})
if err != nil {
t.Fatalf("buildNodeConfig: %v", err)
}
if node.Addr != "docker.sock" {
t.Errorf("Addr = %q, want docker.sock", node.Addr)
}
}
func TestBuildServiceConfig_UnixAbsolutePath(t *testing.T) {
// three-slash: path is the absolute socket listen address, no forward target
svcs, err := buildServiceConfig(mustURL("unix:///var/run/gost.sock"))
if err != nil {
t.Fatalf("buildServiceConfig: %v", err)
}
if len(svcs) != 1 {
t.Fatalf("len(svcs) = %d, want 1", len(svcs))
}
if svcs[0].Addr != "/var/run/gost.sock" {
t.Errorf("Addr = %q, want /var/run/gost.sock", svcs[0].Addr)
}
if svcs[0].Forwarder != nil {
t.Errorf("Forwarder = %+v, want nil (no forward target)", svcs[0].Forwarder)
}
}
func TestBuildServiceConfig_UnixPortForward(t *testing.T) {
// two-slash: host is the listen socket, path is a forward target socket
svcs, err := buildServiceConfig(mustURL("unix://gost.sock/gost2.sock"))
if err != nil {
t.Fatalf("buildServiceConfig: %v", err)
}
if len(svcs) != 1 {
t.Fatalf("len(svcs) = %d, want 1", len(svcs))
}
if svcs[0].Addr != "gost.sock" {
t.Errorf("Addr = %q, want gost.sock", svcs[0].Addr)
}
if svcs[0].Forwarder == nil || len(svcs[0].Forwarder.Nodes) == 0 {
t.Fatalf("Forwarder = %+v, want a forward target node", svcs[0].Forwarder)
}
if got := svcs[0].Forwarder.Nodes[0].Addr; got != "gost2.sock" {
t.Errorf("forward Addr = %q, want gost2.sock", got)
}
}
func mustURL(raw string) *url.URL {
u, err := url.Parse(raw)
if err != nil {
panic(err)
}
return u
}
func TestBuildServiceConfig_RelayTLSForward(t *testing.T) {
// relay+tls scheme: handler=relay, listener=tls.
// relay not in test registry → handler falls back to auto, listener falls back to tcp.
+80 -10
View File
@@ -4,10 +4,13 @@ import (
"encoding/json"
"io"
"path/filepath"
"reflect"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/mitchellh/mapstructure"
"github.com/spf13/viper"
"gopkg.in/yaml.v3"
)
@@ -26,8 +29,21 @@ func init() {
var (
global = &Config{}
globalMux sync.RWMutex
// configFile remembers the last file loaded via ReadFile so the API can
// save back to the same location (issue go-gost/x#123).
configFile atomic.Value // string
)
// ConfigFile returns the path of the last config file loaded via ReadFile,
// or an empty string when the config came from stdin, a URL, or inline JSON.
func ConfigFile() string {
if s, ok := configFile.Load().(string); ok {
return s
}
return ""
}
func Global() *Config {
globalMux.RLock()
defer globalMux.RUnlock()
@@ -322,11 +338,22 @@ type RewriterConfig struct {
Plugin *PluginConfig `yaml:",omitempty" json:"plugin,omitempty"`
}
// CacheConfig defines a named cache store. Only the Memory backend is
// currently supported; other backends (Redis, file, plugin) are follow-ups.
// CacheConfig defines a named cache store. Memory and Redis backends are
// supported; other backends (file, plugin) are follow-ups.
type CacheConfig struct {
Name string `json:"name"`
Memory *MemoryCache `yaml:",omitempty" json:"memory,omitempty"`
Redis *RedisCache `yaml:",omitempty" json:"redis,omitempty"`
}
// RedisCache configures the Redis cache backend.
type RedisCache struct {
Addr string `json:"addr"`
DB int `yaml:",omitempty" json:"db,omitempty"`
Username string `yaml:",omitempty" json:"username,omitempty"`
Password string `yaml:",omitempty" json:"password,omitempty"`
TTL time.Duration `yaml:",omitempty" json:"ttl,omitempty"`
Key string `yaml:",omitempty" json:"key,omitempty"` // key prefix, default "gost:cache:"
}
// MemoryCache configures the in-memory cache backend.
@@ -382,9 +409,21 @@ type ForwarderConfig struct {
// Deprecated: use hop instead
Name string `yaml:",omitempty" json:"name,omitempty"`
// the referenced hop name
Hop string `yaml:",omitempty" json:"hop,omitempty"`
Selector *SelectorConfig `yaml:",omitempty" json:"selector,omitempty"`
Nodes []*ForwardNodeConfig `json:"nodes"`
Hop string `yaml:",omitempty" json:"hop,omitempty"`
Selector *SelectorConfig `yaml:",omitempty" json:"selector,omitempty"`
Nodes []*ForwardNodeConfig `json:"nodes"`
HopGroup *ForwardHopGroupConfig `yaml:"hopGroup,omitempty" json:"hopGroup,omitempty"`
}
type ForwardHopGroupConfig struct {
Hops []*ForwardHopConfig `yaml:",omitempty" json:"hops,omitempty"`
Selector *SelectorConfig `yaml:",omitempty" json:"selector,omitempty"`
}
type ForwardHopConfig struct {
Hop string `yaml:",omitempty" json:"hop,omitempty"`
Matcher *NodeMatcherConfig `yaml:",omitempty" json:"matcher,omitempty"`
Probe *ProbeConfig `yaml:",omitempty" json:"probe,omitempty"`
}
type ForwardNodeConfig struct {
@@ -624,9 +663,39 @@ type ChainConfig struct {
Metadata map[string]any `yaml:",omitempty" json:"metadata,omitempty"`
}
type ChainGroupEntry struct {
Chain string `yaml:",omitempty" json:"chain,omitempty"`
Matcher *NodeMatcherConfig `yaml:",omitempty" json:"matcher,omitempty"`
Probe *ProbeConfig `yaml:",omitempty" json:"probe,omitempty"`
}
// gostDecodeHook is a composite mapstructure decode hook that handles custom
// type conversions needed by GOST config fields. viper's internal unmarshalling
// never calls yaml.Unmarshaler / json.Unmarshaler on leaf types, so the hook is
// the only way to support these.
func gostDecodeHook() mapstructure.DecodeHookFunc {
return func(f, t reflect.Type, data any) (any, error) {
// Plain string → ChainGroupEntry: backward compat for "chains: [a, b]".
if t == reflect.TypeOf(ChainGroupEntry{}) {
if s, ok := data.(string); ok {
return ChainGroupEntry{Chain: s}, nil
}
return data, nil
}
// String → time.Duration: accept "10s", "1m", etc. in YAML/JSON.
if t == reflect.TypeOf(time.Duration(0)) {
if s, ok := data.(string); ok {
return time.ParseDuration(s)
}
return data, nil
}
return data, nil
}
}
type ChainGroupConfig struct {
Chains []string `yaml:",omitempty" json:"chains,omitempty"`
Selector *SelectorConfig `yaml:",omitempty" json:"selector,omitempty"`
Chains []*ChainGroupEntry `yaml:",omitempty" json:"chains,omitempty"`
Selector *SelectorConfig `yaml:",omitempty" json:"selector,omitempty"`
}
type HopConfig struct {
@@ -707,7 +776,7 @@ func (c *Config) Load() error {
return err
}
return v.Unmarshal(c)
return v.Unmarshal(c, viper.DecodeHook(gostDecodeHook()))
}
func (c *Config) Read(r io.Reader, configType string) error {
@@ -716,7 +785,7 @@ func (c *Config) Read(r io.Reader, configType string) error {
return err
}
return v.Unmarshal(c)
return v.Unmarshal(c, viper.DecodeHook(gostDecodeHook()))
}
func (c *Config) ReadFile(file string) error {
@@ -725,7 +794,8 @@ func (c *Config) ReadFile(file string) error {
if err := v.ReadInConfig(); err != nil {
return err
}
return v.Unmarshal(c)
configFile.Store(file)
return v.Unmarshal(c, viper.DecodeHook(gostDecodeHook()))
}
func (c *Config) Write(w io.Writer, format string) error {
+15 -3
View File
@@ -9,9 +9,9 @@ import (
"github.com/go-gost/x/config"
)
// ParseCache converts a CacheConfig into a cache.Cache. Only the memory
// backend is currently supported. Returns nil when cfg is nil or no backend
// is configured.
// ParseCache converts a CacheConfig into a cache.Cache. Memory and Redis
// backends are supported. Returns nil when cfg is nil or no backend is
// configured.
func ParseCache(cfg *config.CacheConfig) cache.Cache {
if cfg == nil {
return nil
@@ -28,5 +28,17 @@ func ParseCache(cfg *config.CacheConfig) cache.Cache {
})
}
if cfg.Redis != nil {
return xcache.NewRedisCache(xcache.RedisOptions{
Addr: cfg.Redis.Addr,
DB: cfg.Redis.DB,
Username: cfg.Redis.Username,
Password: cfg.Redis.Password,
DefaultTTL: cfg.Redis.TTL,
KeyPrefix: cfg.Redis.Key,
Logger: logger.Default(),
})
}
return nil
}
+12
View File
@@ -15,6 +15,7 @@ import (
selector_parser "github.com/go-gost/x/config/parsing/selector"
xhop "github.com/go-gost/x/hop"
hop_plugin "github.com/go-gost/x/hop/plugin"
xs "github.com/go-gost/x/selector"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
"github.com/go-gost/x/metadata"
@@ -158,10 +159,21 @@ func ParseHop(cfg *config.HopConfig, log logger.Logger) (hop.Hop, error) {
sel = selector_parser.DefaultNodeSelector()
}
maxFails, timeout := xs.DefaultMaxFails, xs.DefaultFailTimeout
if cfg.Selector != nil {
if cfg.Selector.MaxFails > 0 {
maxFails = cfg.Selector.MaxFails
}
if cfg.Selector.FailTimeout > 0 {
timeout = cfg.Selector.FailTimeout
}
}
opts := []xhop.Option{
xhop.NameOption(cfg.Name),
xhop.NodeOption(nodes...),
xhop.SelectorOption(sel),
xhop.FailFilterSettingsOption(maxFails, timeout),
xhop.BypassOption(xbypass.BypassGroup(bypass_parser.List(cfg.Bypass, cfg.Bypasses...)...)),
xhop.ReloadPeriodOption(cfg.Reload),
xhop.LoggerOption(log.WithFields(map[string]any{
+103 -50
View File
@@ -36,6 +36,50 @@ const DefaultMatcherBodySize = 1 << 20 // 1MB
// when a node opts in to body matching.
const MaxMatcherBodySize = 10 << 20 // 10MB
// filterToMatcherRule converts a deprecated NodeFilterConfig (host/protocol/
// path) into an equivalent matcher DSL rule. Empty fields are omitted; an
// empty result means the node is unconditional. Path maps to PathPrefix
// (the legacy filter is a prefix match, not exact).
func filterToMatcherRule(filter *config.NodeFilterConfig) string {
if filter == nil {
return ""
}
var parts []string
// Normalize the legacy host forms first: `*.example.com` and
// `.example.com` both meant "apex AND subdomains" in the legacy filter.
// Strip the wildcard so the leading-dot branch below handles both.
host := filter.Host
if strings.HasPrefix(host, "*") {
host = host[1:]
if !strings.HasPrefix(host, ".") {
host = "." + host
}
}
if host != "" {
// A leading-dot host matches the apex AND its subdomains in the
// legacy filter (HasSuffix on the dot-stripped form), but matcher
// Host(.x) only matches subdomains. Emit both to preserve apex
// coverage. Parenthesize the OR when other conjuncts follow, since
// && binds tighter than || in the matcher DSL.
if strings.HasPrefix(host, ".") {
h := "Host(`" + host[1:] + "`) || Host(`" + host + "`)"
if filter.Protocol != "" || filter.Path != "" {
h = "(" + h + ")"
}
parts = append(parts, h)
} else {
parts = append(parts, "Host(`"+host+"`)")
}
}
if prot := filter.Protocol; prot != "" {
parts = append(parts, "Proto(`"+prot+"`)")
}
if path := filter.Path; path != "" {
parts = append(parts, "PathPrefix(`"+path+"`)")
}
return strings.Join(parts, " && ")
}
// ParseNode converts a NodeConfig into a *chain.Node. It resolves the
// connector and dialer from their registries, applies TLS settings, extracts
// metadata-driven options (so_mark, interface, netns, proxy protocol), sets up
@@ -48,8 +92,16 @@ func parseBodyRewrites(vs []config.HTTPBodyRewriteConfig, log logger.Logger) []c
var pattern *regexp.Regexp
var rewriteType string
if js, ok := strings.CutPrefix(v.Match, "json:"); ok {
// json:<path> or json:<path>=<value-regex>
js, ok := strings.CutPrefix(v.Match, "json:")
del := false
if !ok {
// json-:<path> deletes the matched JSON field instead of replacing it.
if js, ok = strings.CutPrefix(v.Match, "json-:"); ok {
del = true
}
}
if ok {
// json:<path>[=<value-regex>] or json-:<path>[=<value-regex>]
path, valRegex, _ := strings.Cut(js, "=")
if valRegex == "" {
valRegex = ".*"
@@ -60,7 +112,11 @@ func parseBodyRewrites(vs []config.HTTPBodyRewriteConfig, log logger.Logger) []c
log.Warnf("invalid JSON value regex %q for path %q: %v", valRegex, path, err)
continue
}
rewriteType = "json:" + path
if del {
rewriteType = "json-:" + path
} else {
rewriteType = "json:" + path
}
} else {
if v.Match != "" {
pattern, _ = regexp.Compile(v.Match)
@@ -209,56 +265,53 @@ func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.No
chain.NetworkNodeOption(cfg.Network),
}
if filter := cfg.Filter; filter != nil {
// convert *.example.com to .example.com
// convert *example.com to example.com
host := filter.Host
if strings.HasPrefix(host, "*") {
host = host[1:]
if !strings.HasPrefix(host, ".") {
host = "." + host
}
}
var (
rule string
priority int
bodySize int
)
settings := &chain.NodeFilterSettings{
Protocol: filter.Protocol,
Host: host,
Path: filter.Path,
}
opts = append(opts, chain.NodeFilterOption(settings))
// Deprecated filter: normalize into an equivalent matcher DSL rule so the
// runtime keeps a single filtering logic. Explicit -1 priority opts out of
// auto-priority and the priority short-circuit, preserving the legacy
// filter behavior (always through the selector).
if filter := cfg.Filter; filter != nil {
rule = filterToMatcherRule(filter)
priority = -1
bodySize = DefaultMatcherBodySize
}
if cfg.Matcher != nil {
priority := cfg.Matcher.Priority
if rule := strings.TrimSpace(cfg.Matcher.Rule); rule != "" {
if matcher, err := routing.NewMatcher(rule); err == nil {
log.Debugf("new matcher for node %s with rule %s", cfg.Name, cfg.Matcher.Rule)
// Priority 0 means "use default": automatically set to the
// rule length so longer (more specific) rules outrank shorter
// ones. Use a negative priority to opt out of this behavior
// and always go through the selector.
if priority == 0 {
priority = len(cfg.Matcher.Rule)
}
opts = append(opts, chain.MatcherNodeOption(matcher))
} else {
log.Error(err)
priority = -1
}
}
bodySize := cfg.Matcher.BodySize
if bodySize <= 0 {
bodySize = DefaultMatcherBodySize
} else if bodySize > MaxMatcherBodySize {
bodySize = MaxMatcherBodySize
}
opts = append(opts, chain.MatcherBodySizeNodeOption(bodySize))
opts = append(opts, chain.PriorityNodeOption(priority))
priority = cfg.Matcher.Priority
rule = cfg.Matcher.Rule
bodySize = cfg.Matcher.BodySize
}
if rule = strings.TrimSpace(rule); rule != "" {
if matcher, err := routing.NewMatcher(rule); err == nil {
log.Debugf("new matcher for node %s with rule %s", cfg.Name, rule)
// Priority 0 means "use default": automatically set to the
// rule length so longer (more specific) rules outrank shorter
// ones. Use a negative priority to opt out of this behavior
// and always go through the selector.
if priority == 0 {
priority = len(rule)
}
opts = append(opts, chain.MatcherNodeOption(matcher))
} else {
log.Error(err)
priority = -1
}
}
if bodySize <= 0 {
bodySize = DefaultMatcherBodySize
} else if bodySize > MaxMatcherBodySize {
bodySize = MaxMatcherBodySize
}
opts = append(opts, chain.MatcherBodySizeNodeOption(bodySize))
opts = append(opts, chain.PriorityNodeOption(priority))
if cfg.HTTP != nil {
settings := &chain.HTTPNodeSettings{
Host: cfg.HTTP.Host,
@@ -319,16 +372,16 @@ func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.No
node := chain.NewNode(cfg.Name, cfg.Addr, opts...)
if cfg.Probe != nil {
if pc := parseProbeConfig(cfg.Probe); pc != nil {
if pc := ParseProbeConfig(cfg.Probe); pc != nil {
xchain.StartNodeProbe(node, pc, nodeLogger)
}
}
return node, nil
}
// parseProbeConfig converts a config.ProbeConfig into a chain.ProbeConfig.
// ParseProbeConfig converts a config.ProbeConfig into a chain.ProbeConfig.
// Returns nil when the config is invalid (e.g. empty addr).
func parseProbeConfig(cfg *config.ProbeConfig) *chain.ProbeConfig {
func ParseProbeConfig(cfg *config.ProbeConfig) *chain.ProbeConfig {
if cfg == nil || (cfg.Addr == "" && cfg.Type != "cmd") {
return nil
}
+35
View File
@@ -501,3 +501,38 @@ func TestParseNode_DeprecatedFilterFields(t *testing.T) {
t.Fatal("expected non-nil node")
}
}
func TestFilterToMatcherRule(t *testing.T) {
tests := []struct {
name string
filter *config.NodeFilterConfig
want string
}{
{"nil", nil, ""},
{"all empty", &config.NodeFilterConfig{}, ""},
{"host only", &config.NodeFilterConfig{Host: "example.com"}, "Host(`example.com`)"},
{"dot host", &config.NodeFilterConfig{Host: ".example.com"},
"Host(`example.com`) || Host(`.example.com`)"},
{"star host", &config.NodeFilterConfig{Host: "*.example.com"},
"Host(`example.com`) || Host(`.example.com`)"},
{"protocol only", &config.NodeFilterConfig{Protocol: "ssh"}, "Proto(`ssh`)"},
{"path only", &config.NodeFilterConfig{Path: "/api"}, "PathPrefix(`/api`)"},
{"combined", &config.NodeFilterConfig{
Host: ".example.com",
Protocol: "http",
Path: "/api",
}, "(Host(`example.com`) || Host(`.example.com`)) && Proto(`http`) && PathPrefix(`/api`)"},
{"star combined", &config.NodeFilterConfig{
Host: "*.example.com",
Protocol: "http",
Path: "/api",
}, "(Host(`example.com`) || Host(`.example.com`)) && Proto(`http`) && PathPrefix(`/api`)"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := filterToMatcherRule(tt.filter); got != tt.want {
t.Errorf("filterToMatcherRule(%v) = %q, want %q", tt.filter, got, tt.want)
}
})
}
}
+35
View File
@@ -0,0 +1,35 @@
package node
import (
"testing"
"github.com/go-gost/x/config"
coreRouting "github.com/go-gost/core/routing"
"github.com/go-gost/x/routing"
)
func TestFilterToMatcherRule_Precedence(t *testing.T) {
for _, fc := range []struct{ name, host string }{
{"dot", ".example.com"},
{"star", "*.example.com"},
} {
t.Run(fc.name, func(t *testing.T) {
rule := filterToMatcherRule(&config.NodeFilterConfig{
Host: fc.host, Protocol: "http", Path: "/api",
})
m, err := routing.NewMatcher(rule)
if err != nil {
t.Fatalf("parse %q: %v", rule, err)
}
if m.Match(&coreRouting.Request{Host: "example.com", Protocol: "https", Path: "/other"}) {
t.Errorf("rule %q matched apex+https+/other (should reject)", rule)
}
if !m.Match(&coreRouting.Request{Host: "example.com", Protocol: "http", Path: "/api/v1"}) {
t.Errorf("rule %q rejected apex+http+/api/v1 (should accept)", rule)
}
if !m.Match(&coreRouting.Request{Host: "www.example.com", Protocol: "http", Path: "/api"}) {
t.Errorf("rule %q rejected www+http+/api (should accept)", rule)
}
})
}
}
+166 -23
View File
@@ -2,6 +2,7 @@ package service
import (
"fmt"
"io"
"runtime"
"strings"
"time"
@@ -12,6 +13,7 @@ import (
"github.com/go-gost/core/hop"
"github.com/go-gost/core/listener"
"github.com/go-gost/core/rewriter"
"github.com/go-gost/core/routing"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/observer/stats"
"github.com/go-gost/core/recorder"
@@ -28,8 +30,12 @@ import (
bypass_parser "github.com/go-gost/x/config/parsing/bypass"
hop_parser "github.com/go-gost/x/config/parsing/hop"
logger_parser "github.com/go-gost/x/config/parsing/logger"
node_parser "github.com/go-gost/x/config/parsing/node"
selector_parser "github.com/go-gost/x/config/parsing/selector"
xhop "github.com/go-gost/x/hop"
tls_util "github.com/go-gost/x/internal/util/tls"
xrouting "github.com/go-gost/x/routing"
xs "github.com/go-gost/x/selector"
quota_wrapper "github.com/go-gost/x/limiter/quota/wrapper"
cache_limiter "github.com/go-gost/x/limiter/traffic/cache"
"github.com/go-gost/x/metadata"
@@ -199,13 +205,32 @@ func ParseService(cfg *config.ServiceConfig) (service.Service, error) {
}
if !ignoreChain {
routerOpts = append(routerOpts,
chain.ChainRouterOption(chainGroup(cfg.Listener.Chain, cfg.Listener.ChainGroup)),
chain.ChainRouterOption(chainGroup(cfg.Listener.Chain, cfg.Listener.ChainGroup, log)),
)
}
lnRouter := xchain.NewRouter(routerOpts...)
// The routers may hold a chainGroup running probe goroutines, and the
// forwarder hop may run probe goroutines of its own; make sure they are
// closed if service construction fails after this point.
var hRouter *xchain.Router
var fwdCloser io.Closer
success := false
defer func() {
if !success {
lnRouter.Close()
if hRouter != nil {
hRouter.Close()
}
if fwdCloser != nil {
fwdCloser.Close()
}
}
}()
listenOpts := []listener.Option{
listener.AddrOption(cfg.Addr),
listener.RouterOption(xchain.NewRouter(routerOpts...)),
listener.RouterOption(lnRouter),
listener.AutherOption(auther),
listener.AuthOption(auth_parser.Info(cfg.Listener.Auth)),
listener.TLSConfigOption(tlsConfig),
@@ -375,14 +400,15 @@ func ParseService(cfg *config.ServiceConfig) (service.Service, error) {
}
if !ignoreChain {
routerOpts = append(routerOpts,
chain.ChainRouterOption(chainGroup(cfg.Handler.Chain, cfg.Handler.ChainGroup)),
chain.ChainRouterOption(chainGroup(cfg.Handler.Chain, cfg.Handler.ChainGroup, log)),
)
}
var h handler.Handler
if rf := registry.HandlerRegistry().Get(cfg.Handler.Type); rf != nil {
hRouter = xchain.NewRouter(routerOpts...)
h = rf(
handler.RouterOption(xchain.NewRouter(routerOpts...)),
handler.RouterOption(hRouter),
handler.AutherOption(auther),
handler.AuthOption(auth_parser.Info(cfg.Handler.Auth)),
handler.BypassOption(xbypass.BypassGroup(bypass_parser.List(cfg.Bypass, cfg.Bypasses...)...)),
@@ -402,10 +428,11 @@ func ParseService(cfg *config.ServiceConfig) (service.Service, error) {
}
if forwarder, ok := h.(handler.Forwarder); ok {
hop, err := parseForwarder(cfg.Forwarder, log)
hop, closer, err := parseForwarder(cfg.Forwarder, log)
if err != nil {
return nil, err
}
fwdCloser = closer
forwarder.Forward(hop)
}
@@ -418,6 +445,10 @@ func ParseService(cfg *config.ServiceConfig) (service.Service, error) {
return nil, err
}
closers := []io.Closer{lnRouter, hRouter}
if fwdCloser != nil {
closers = append(closers, fwdCloser)
}
s := xservice.NewService(cfg.Name, ln, h,
xservice.AdmissionOption(xadmission.AdmissionGroup(admissions...)),
xservice.PreUpOption(preUp),
@@ -430,15 +461,31 @@ func ParseService(cfg *config.ServiceConfig) (service.Service, error) {
xservice.ObserverPeriodOption(observerPeriod),
xservice.LoggerOption(serviceLogger),
xservice.LabelsOption(labels),
xservice.ClosersOption(closers...),
)
success = true
serviceLogger.Infof("listening on %s/%s", s.Addr().String(), s.Addr().Network())
return s, nil
}
func parseForwarder(cfg *config.ForwarderConfig, log logger.Logger) (hop.Hop, error) {
// parseForwarder builds the forwarder hop for a service. The returned
// io.Closer is non-nil only when the hop is owned by the service (inline
// nodes or a hop group) and must be closed on service shutdown; hops from
// the HopRegistry are closed by the registry on unregister.
func parseForwarder(cfg *config.ForwarderConfig, log logger.Logger) (hop.Hop, io.Closer, error) {
if cfg == nil {
return nil, nil
return nil, nil, nil
}
// HopGroup takes precedence over Hop / Name / Nodes.
if cfg.HopGroup != nil {
group, err := parseHopGroup(cfg.HopGroup, log)
if err != nil || group == nil {
return nil, nil, err
}
closer, _ := group.(io.Closer)
return group, closer, nil
}
hopName := cfg.Hop
@@ -446,7 +493,7 @@ func parseForwarder(cfg *config.ForwarderConfig, log logger.Logger) (hop.Hop, er
hopName = cfg.Name
}
if hopName != "" {
return registry.HopRegistry().Get(hopName), nil
return registry.HopRegistry().Get(hopName), nil, nil
}
hc := config.HopConfig{
@@ -492,32 +539,128 @@ func parseForwarder(cfg *config.ForwarderConfig, log logger.Logger) (hop.Hop, er
Metadata: node.Metadata,
})
}
return hop_parser.ParseHop(&hc, log)
// Inline nodes build a per-service hop, owned (and closed) by the service.
h, err := hop_parser.ParseHop(&hc, log)
if err != nil || h == nil {
return nil, nil, err
}
closer, _ := h.(io.Closer)
return h, closer, nil
}
func chainGroup(name string, group *config.ChainGroupConfig) chain.Chainer {
var chains []chain.Chainer
var sel selector.Selector[chain.Chainer]
if c := registry.ChainRegistry().Get(name); c != nil {
chains = append(chains, c)
func parseHopGroup(cfg *config.ForwardHopGroupConfig, log logger.Logger) (hop.Hop, error) {
if cfg == nil {
return nil, nil
}
if group != nil {
for _, s := range group.Chains {
if c := registry.ChainRegistry().Get(s); c != nil {
chains = append(chains, c)
var entries []*xhop.HopEntry
for _, hc := range cfg.Hops {
if hc == nil {
continue
}
h := registry.HopRegistry().Get(hc.Hop)
if h == nil {
log.Warnf("hop %q not found in hopGroup, skipping", hc.Hop)
continue
}
var m routing.Matcher
if hc.Matcher != nil && hc.Matcher.Rule != "" {
var err error
m, err = xrouting.NewMatcher(hc.Matcher.Rule)
if err != nil {
log.Warnf("hop %q: bad matcher rule %q: %v, skipping", hc.Hop, hc.Matcher.Rule, err)
continue
}
}
sel = selector_parser.ParseChainSelector(group.Selector)
entries = append(entries, xhop.NewHopEntry(h, m, node_parser.ParseProbeConfig(hc.Probe)))
}
if len(chains) == 0 {
if len(entries) == 0 {
return nil, nil
}
var hopSelector selector.Selector[hop.Hop]
if cfg.Selector != nil {
var strategy selector.Strategy[hop.Hop]
switch cfg.Selector.Strategy {
case "round", "rr":
strategy = xs.RoundRobinStrategy[hop.Hop]()
case "random", "rand":
strategy = xs.RandomStrategy[hop.Hop]()
case "fifo", "ha":
strategy = xs.FIFOStrategy[hop.Hop]()
case "hash":
strategy = xs.HashStrategy[hop.Hop]()
default:
strategy = xs.RoundRobinStrategy[hop.Hop]()
}
hopSelector = xs.NewSelector(
strategy,
xs.FailFilter[hop.Hop](cfg.Selector.MaxFails, cfg.Selector.FailTimeout),
xs.BackupFilter[hop.Hop](),
)
}
if hopSelector == nil {
hopSelector = xs.NewSelector(
xs.RoundRobinStrategy[hop.Hop](),
xs.FailFilter[hop.Hop](xs.DefaultMaxFails, xs.DefaultFailTimeout),
xs.BackupFilter[hop.Hop](),
)
}
return xhop.NewHopGroup(
xhop.WithEntriesOption(entries...),
xhop.WithGroupSelectorOption(hopSelector),
xhop.WithGroupLoggerOption(log.WithFields(map[string]any{
"kind": "hopGroup",
"service": "forwarder",
})),
), nil
}
func chainGroup(name string, group *config.ChainGroupConfig, log logger.Logger) chain.Chainer {
var entries []*xchain.ChainEntry
if c := registry.ChainRegistry().Get(name); c != nil {
entries = append(entries, xchain.NewChainEntry(c, nil, nil))
}
if group != nil {
for _, ge := range group.Chains {
c := registry.ChainRegistry().Get(ge.Chain)
if c == nil {
log.Warnf("chain %q not found in chainGroup, skipping", ge.Chain)
continue
}
var m routing.Matcher
if ge.Matcher != nil && ge.Matcher.Rule != "" {
var err error
m, err = xrouting.NewMatcher(ge.Matcher.Rule)
if err != nil {
log.Warnf("chain %q: bad matcher rule %q: %v, skipping", ge.Chain, ge.Matcher.Rule, err)
continue
}
}
entries = append(entries, xchain.NewChainEntry(c, m, node_parser.ParseProbeConfig(ge.Probe)))
}
}
if len(entries) == 0 {
return nil
}
var sel selector.Selector[chain.Chainer]
if group != nil {
sel = selector_parser.ParseChainSelector(group.Selector)
}
if sel == nil {
sel = selector_parser.DefaultChainSelector()
}
return xchain.NewChainGroup(chains...).
WithSelector(sel)
return xchain.NewChainGroup().
WithGroupEntries(entries...).
WithSelector(sel).
WithGroupLogger(log)
}
+328
View File
@@ -0,0 +1,328 @@
package masque
import (
"context"
"errors"
"net"
"os"
"sync"
"time"
"github.com/go-gost/core/logger"
)
const udpAssociationBufferSize = 64 * 1024
var errMissingDestination = errors.New("masque: destination address required")
type udpAssociationDialFunc func(ctx context.Context, addr net.Addr) (net.PacketConn, error)
type udpAssociationResult struct {
data []byte
addr net.Addr
}
type udpAssociationTunnel struct {
conn net.PacketConn
}
type udpAssociationConn struct {
ctx context.Context
cancel context.CancelFunc
localAddr net.Addr
dial udpAssociationDialFunc
closeIdle func() error
log logger.Logger
timeout time.Duration
closed chan struct{}
closeOnce sync.Once
closeErr error
results chan udpAssociationResult
mu sync.Mutex
tunnels map[string]*udpAssociationTunnel
readDeadline time.Time
readDeadlineChanged chan struct{}
}
func newUDPAssociationConn(
ctx context.Context,
localAddr net.Addr,
timeout time.Duration,
dial udpAssociationDialFunc,
closeIdle func() error,
log logger.Logger,
) *udpAssociationConn {
ctx, cancel := context.WithCancel(context.WithoutCancel(ctx))
c := &udpAssociationConn{
ctx: ctx,
cancel: cancel,
localAddr: localAddr,
dial: dial,
closeIdle: closeIdle,
log: log,
timeout: timeout,
closed: make(chan struct{}),
results: make(chan udpAssociationResult, 32),
tunnels: make(map[string]*udpAssociationTunnel),
readDeadlineChanged: make(chan struct{}),
}
return c
}
func (c *udpAssociationConn) ReadFrom(b []byte) (n int, addr net.Addr, err error) {
for {
select {
case <-c.closed:
return 0, nil, net.ErrClosed
default:
}
c.mu.Lock()
deadline := c.readDeadline
deadlineChanged := c.readDeadlineChanged
c.mu.Unlock()
var (
timer *time.Timer
timeout <-chan time.Time
)
if !deadline.IsZero() {
remaining := time.Until(deadline)
if remaining <= 0 {
return 0, nil, os.ErrDeadlineExceeded
}
timer = time.NewTimer(remaining)
timeout = timer.C
}
select {
case result := <-c.results:
if timer != nil {
timer.Stop()
}
return copy(b, result.data), result.addr, nil
case <-deadlineChanged:
if timer != nil {
timer.Stop()
}
continue
case <-timeout:
return 0, nil, os.ErrDeadlineExceeded
case <-c.closed:
if timer != nil {
timer.Stop()
}
return 0, nil, net.ErrClosed
}
}
}
func (c *udpAssociationConn) WriteTo(b []byte, addr net.Addr) (int, error) {
if addr == nil {
return 0, errMissingDestination
}
tunnel, err := c.tunnel(addr)
if err != nil {
if c.isClosed() {
return 0, net.ErrClosed
}
c.logError(addr, err)
return len(b), nil
}
n, err := tunnel.conn.WriteTo(b, addr)
if err == nil {
return n, nil
}
if c.isClosed() {
return 0, net.ErrClosed
}
if c.removeTunnel(addr.String(), tunnel) {
c.logError(addr, err)
}
return len(b), nil
}
func (c *udpAssociationConn) tunnel(addr net.Addr) (*udpAssociationTunnel, error) {
key := addr.String()
c.mu.Lock()
select {
case <-c.closed:
c.mu.Unlock()
return nil, net.ErrClosed
default:
}
if tunnel := c.tunnels[key]; tunnel != nil {
c.mu.Unlock()
return tunnel, nil
}
c.mu.Unlock()
ctx := c.ctx
if c.timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, c.timeout)
defer cancel()
}
conn, err := c.dial(ctx, addr)
if err != nil {
return nil, err
}
if conn == nil {
return nil, errors.New("masque: nil UDP tunnel")
}
tunnel := &udpAssociationTunnel{conn: conn}
c.mu.Lock()
select {
case <-c.closed:
c.mu.Unlock()
conn.Close()
return nil, net.ErrClosed
default:
}
if current := c.tunnels[key]; current != nil {
c.mu.Unlock()
conn.Close()
return current, nil
}
c.tunnels[key] = tunnel
c.mu.Unlock()
go c.readTunnel(key, tunnel)
return tunnel, nil
}
func (c *udpAssociationConn) readTunnel(key string, tunnel *udpAssociationTunnel) {
buf := make([]byte, udpAssociationBufferSize)
for {
n, addr, err := tunnel.conn.ReadFrom(buf)
if err != nil {
if c.isClosed() {
return
}
if c.removeTunnel(key, tunnel) {
c.logError(key, err)
}
return
}
data := append([]byte(nil), buf[:n]...)
select {
case c.results <- udpAssociationResult{data: data, addr: addr}:
case <-c.closed:
return
}
}
}
func (c *udpAssociationConn) isClosed() bool {
select {
case <-c.closed:
return true
default:
return false
}
}
func (c *udpAssociationConn) logError(addr any, err error) {
if c.log != nil {
c.log.Warnf("masque: UDP tunnel %v: %v", addr, err)
}
}
func (c *udpAssociationConn) removeTunnel(key string, tunnel *udpAssociationTunnel) bool {
c.mu.Lock()
removed := c.tunnels[key] == tunnel
if c.tunnels[key] == tunnel {
delete(c.tunnels, key)
}
c.mu.Unlock()
tunnel.conn.Close()
return removed
}
func (c *udpAssociationConn) Read(b []byte) (int, error) {
n, _, err := c.ReadFrom(b)
return n, err
}
func (c *udpAssociationConn) Write(b []byte) (int, error) {
return 0, errMissingDestination
}
func (c *udpAssociationConn) Close() error {
c.closeOnce.Do(func() {
close(c.closed)
c.cancel()
c.mu.Lock()
close(c.readDeadlineChanged)
tunnels := make([]*udpAssociationTunnel, 0, len(c.tunnels))
for _, tunnel := range c.tunnels {
tunnels = append(tunnels, tunnel)
}
clear(c.tunnels)
c.mu.Unlock()
var errs []error
for _, tunnel := range tunnels {
if err := tunnel.conn.Close(); err != nil {
errs = append(errs, err)
}
}
if c.closeIdle != nil {
if err := c.closeIdle(); err != nil {
errs = append(errs, err)
}
}
c.closeErr = errors.Join(errs...)
})
return c.closeErr
}
func (c *udpAssociationConn) LocalAddr() net.Addr {
return c.localAddr
}
func (c *udpAssociationConn) RemoteAddr() net.Addr {
return &net.UDPAddr{}
}
func (c *udpAssociationConn) SetDeadline(t time.Time) error {
if err := c.SetReadDeadline(t); err != nil {
return err
}
return c.SetWriteDeadline(t)
}
func (c *udpAssociationConn) SetReadDeadline(t time.Time) error {
c.mu.Lock()
defer c.mu.Unlock()
select {
case <-c.closed:
return net.ErrClosed
default:
}
c.readDeadline = t
close(c.readDeadlineChanged)
c.readDeadlineChanged = make(chan struct{})
return nil
}
func (c *udpAssociationConn) SetWriteDeadline(t time.Time) error {
select {
case <-c.closed:
return net.ErrClosed
default:
}
return nil
}
var (
_ net.Conn = (*udpAssociationConn)(nil)
_ net.PacketConn = (*udpAssociationConn)(nil)
)
+321
View File
@@ -0,0 +1,321 @@
package masque
import (
"context"
"errors"
"net"
"sync"
"testing"
"time"
)
func TestUDPAssociationConnRoutesByDestination(t *testing.T) {
type dialCall struct {
addr net.Addr
conn *fakePacketConn
}
var (
mu sync.Mutex
calls []dialCall
)
dial := func(ctx context.Context, addr net.Addr) (net.PacketConn, error) {
conn := newFakePacketConn()
mu.Lock()
calls = append(calls, dialCall{addr: addr, conn: conn})
mu.Unlock()
return conn, nil
}
ctx, cancel := context.WithCancel(context.Background())
conn := newUDPAssociationConn(ctx, &net.UDPAddr{}, 0, dial, nil, nil)
cancel() // The route dial context ends before the UDP association is used.
t.Cleanup(func() { conn.Close() })
addr1 := &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 53}
addr2 := &net.UDPAddr{IP: net.ParseIP("8.8.8.8"), Port: 53}
if _, err := conn.WriteTo([]byte("first"), addr1); err != nil {
t.Fatalf("first WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("second"), addr1); err != nil {
t.Fatalf("second WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("third"), addr2); err != nil {
t.Fatalf("third WriteTo failed: %v", err)
}
mu.Lock()
gotCalls := append([]dialCall(nil), calls...)
mu.Unlock()
if len(gotCalls) != 2 {
t.Fatalf("expected one tunnel per destination, got %d", len(gotCalls))
}
if gotCalls[0].addr.String() != addr1.String() || gotCalls[1].addr.String() != addr2.String() {
t.Fatalf("unexpected tunnel destinations: %s, %s", gotCalls[0].addr, gotCalls[1].addr)
}
if got := gotCalls[0].conn.writtenData(); len(got) != 2 || string(got[0]) != "first" || string(got[1]) != "second" {
t.Fatalf("unexpected writes for first destination: %q", got)
}
if got := gotCalls[1].conn.writtenData(); len(got) != 1 || string(got[0]) != "third" {
t.Fatalf("unexpected writes for second destination: %q", got)
}
replyAddr := &net.UDPAddr{IP: net.ParseIP("127.0.0.1"), Port: 1}
gotCalls[1].conn.queueRead([]byte("reply"), replyAddr)
buf := make([]byte, 32)
n, addr, err := conn.ReadFrom(buf)
if err != nil {
t.Fatalf("ReadFrom failed: %v", err)
}
if string(buf[:n]) != "reply" {
t.Fatalf("unexpected reply: %q", buf[:n])
}
if addr.String() != replyAddr.String() {
t.Fatalf("expected reply from %s, got %s", replyAddr, addr)
}
}
func TestUDPAssociationConnClose(t *testing.T) {
upstream := newFakePacketConn()
idleClosed := 0
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) { return upstream, nil },
func() error {
idleClosed++
return nil
},
nil,
)
if _, err := conn.WriteTo([]byte("request"), &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 53}); err != nil {
t.Fatalf("WriteTo failed: %v", err)
}
if err := conn.Close(); err != nil {
t.Fatalf("Close failed: %v", err)
}
if err := conn.Close(); err != nil {
t.Fatalf("second Close failed: %v", err)
}
if idleClosed != 1 {
t.Fatalf("expected idle stream to be closed once, got %d", idleClosed)
}
if !upstream.isClosed() {
t.Fatal("expected destination tunnel to be closed")
}
if _, err := conn.WriteTo([]byte("request"), &net.UDPAddr{}); !errors.Is(err, net.ErrClosed) {
t.Fatalf("expected net.ErrClosed after Close, got %v", err)
}
}
func TestUDPAssociationConnTunnelFailureIsIsolated(t *testing.T) {
var (
mu sync.Mutex
calls []*fakePacketConn
)
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) {
pc := newFakePacketConn()
mu.Lock()
calls = append(calls, pc)
mu.Unlock()
return pc, nil
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
addr1 := &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443}
addr2 := &net.UDPAddr{IP: net.ParseIP("8.8.8.8"), Port: 443}
if _, err := conn.WriteTo([]byte("first"), addr1); err != nil {
t.Fatalf("first WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("second"), addr2); err != nil {
t.Fatalf("second WriteTo failed: %v", err)
}
mu.Lock()
first, second := calls[0], calls[1]
mu.Unlock()
first.queueReadError(errors.New("stream reset"))
select {
case <-first.closed:
case <-time.After(time.Second):
t.Fatal("failed tunnel was not removed")
}
second.queueRead([]byte("reply"), addr2)
buf := make([]byte, 32)
n, addr, err := conn.ReadFrom(buf)
if err != nil {
t.Fatalf("healthy tunnel failed after peer reset: %v", err)
}
if string(buf[:n]) != "reply" || addr.String() != addr2.String() {
t.Fatalf("unexpected healthy tunnel reply %q from %v", buf[:n], addr)
}
if _, err := conn.WriteTo([]byte("retry"), addr1); err != nil {
t.Fatalf("retry WriteTo failed: %v", err)
}
mu.Lock()
callCount := len(calls)
mu.Unlock()
if callCount != 3 {
t.Fatalf("expected failed destination to redial, got %d dials", callCount)
}
}
func TestUDPAssociationConnDialTimeout(t *testing.T) {
dialErr := make(chan error, 1)
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
20*time.Millisecond,
func(ctx context.Context, _ net.Addr) (net.PacketConn, error) {
<-ctx.Done()
dialErr <- ctx.Err()
return nil, ctx.Err()
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
payload := []byte("request")
n, err := conn.WriteTo(payload, &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443})
if err != nil || n != len(payload) {
t.Fatalf("timed out datagram was not dropped cleanly: n=%d err=%v", n, err)
}
if err := <-dialErr; !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("expected dial deadline, got %v", err)
}
}
func TestUDPAssociationConnDialDoesNotHoldAssociationLock(t *testing.T) {
started := make(chan struct{})
release := make(chan struct{})
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) {
close(started)
<-release
return newFakePacketConn(), nil
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
writeDone := make(chan struct{})
go func() {
conn.WriteTo([]byte("request"), &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443})
close(writeDone)
}()
<-started
deadlineDone := make(chan error, 1)
go func() {
deadlineDone <- conn.SetReadDeadline(time.Now().Add(time.Second))
}()
select {
case err := <-deadlineDone:
if err != nil {
t.Fatalf("SetReadDeadline failed: %v", err)
}
case <-time.After(time.Second):
t.Fatal("dial held the association lock")
}
close(release)
<-writeDone
}
type fakePacket struct {
data []byte
addr net.Addr
err error
}
type fakePacketConn struct {
reads chan fakePacket
closed chan struct{}
closeOnce sync.Once
mu sync.Mutex
writes [][]byte
}
func newFakePacketConn() *fakePacketConn {
return &fakePacketConn{
reads: make(chan fakePacket, 1),
closed: make(chan struct{}),
}
}
func (c *fakePacketConn) queueRead(data []byte, addr net.Addr) {
c.reads <- fakePacket{data: append([]byte(nil), data...), addr: addr}
}
func (c *fakePacketConn) queueReadError(err error) {
c.reads <- fakePacket{err: err}
}
func (c *fakePacketConn) writtenData() [][]byte {
c.mu.Lock()
defer c.mu.Unlock()
result := make([][]byte, len(c.writes))
for i, data := range c.writes {
result[i] = append([]byte(nil), data...)
}
return result
}
func (c *fakePacketConn) isClosed() bool {
select {
case <-c.closed:
return true
default:
return false
}
}
func (c *fakePacketConn) ReadFrom(b []byte) (int, net.Addr, error) {
select {
case packet := <-c.reads:
if packet.err != nil {
return 0, nil, packet.err
}
return copy(b, packet.data), packet.addr, nil
case <-c.closed:
return 0, nil, net.ErrClosed
}
}
func (c *fakePacketConn) WriteTo(b []byte, _ net.Addr) (int, error) {
select {
case <-c.closed:
return 0, net.ErrClosed
default:
}
c.mu.Lock()
c.writes = append(c.writes, append([]byte(nil), b...))
c.mu.Unlock()
return len(b), nil
}
func (c *fakePacketConn) Close() error {
c.closeOnce.Do(func() { close(c.closed) })
return nil
}
func (c *fakePacketConn) LocalAddr() net.Addr { return &net.UDPAddr{} }
func (c *fakePacketConn) SetDeadline(time.Time) error { return nil }
func (c *fakePacketConn) SetReadDeadline(time.Time) error { return nil }
func (c *fakePacketConn) SetWriteDeadline(time.Time) error { return nil }
+82 -15
View File
@@ -10,6 +10,7 @@ import (
"net/url"
"strconv"
"strings"
"sync"
"time"
"github.com/go-gost/core/connector"
@@ -19,6 +20,7 @@ import (
masque_dialer "github.com/go-gost/x/dialer/http3/masque"
masque_util "github.com/go-gost/x/internal/util/masque"
"github.com/go-gost/x/registry"
"github.com/quic-go/quic-go/http3"
)
func init() {
@@ -73,6 +75,16 @@ func (c *masqueConnector) Connect(ctx context.Context, conn net.Conn, network, a
return nil, fmt.Errorf("%w: %s", ErrUnsupportedNetwork, network)
}
func newTCPConnectRequest(address string) *http.Request {
return &http.Request{
Method: http.MethodConnect,
URL: &url.URL{},
Host: address,
Header: http.Header{},
ProtoMajor: 3,
}
}
func (c *masqueConnector) connectTCP(ctx context.Context, conn net.Conn, address string, log logger.Logger) (net.Conn, error) {
log.Debugf("connect-tcp %s", address)
@@ -103,13 +115,7 @@ func (c *masqueConnector) connectTCP(ctx context.Context, conn net.Conn, address
// Create standard HTTP/3 CONNECT request (RFC 9114)
// No :protocol pseudo-header for standard CONNECT
req := &http.Request{
Method: http.MethodConnect,
URL: &url.URL{},
Host: address, // Target address goes in :authority
Header: http.Header{},
Proto: "HTTP/3.0",
}
req := newTCPConnectRequest(address)
// Add proxy authentication if configured
if c.options.Auth != nil {
@@ -172,9 +178,69 @@ func (c *masqueConnector) connectUDP(ctx context.Context, conn net.Conn, address
return nil, ErrInvalidConnection
}
// Get pre-opened stream from dialer (stream opening happens there for dead connection detection)
reqStream := masqueConn.GetRequestStream()
proxyHost := masqueConn.GetHost()
if address == "" {
return c.connectUDPAssociation(ctx, masqueConn, log), nil
}
return c.connectUDPTarget(
ctx,
conn.LocalAddr(),
masqueConn.GetHost(),
masqueConn.GetRequestStream(),
address,
nil,
log,
)
}
func (c *masqueConnector) connectUDPAssociation(ctx context.Context, conn *masque_dialer.MasqueConn, log logger.Logger) net.Conn {
var (
mu sync.Mutex
firstStream = conn.GetRequestStream()
)
nextStream := func(ctx context.Context) (*http3.RequestStream, error) {
mu.Lock()
defer mu.Unlock()
if firstStream != nil {
stream := firstStream
firstStream = nil
return stream, nil
}
return conn.OpenRequestStream(ctx)
}
closeIdle := func() error {
mu.Lock()
defer mu.Unlock()
if firstStream == nil {
return nil
}
err := firstStream.Close()
firstStream = nil
return err
}
dial := func(ctx context.Context, addr net.Addr) (net.PacketConn, error) {
stream, err := nextStream(ctx)
if err != nil {
return nil, err
}
return c.connectUDPTarget(ctx, conn.LocalAddr(), conn.GetHost(), stream, addr.String(), addr, log)
}
return newUDPAssociationConn(ctx, conn.LocalAddr(), c.md.connectTimeout, dial, closeIdle, log)
}
func (c *masqueConnector) connectUDPTarget(
ctx context.Context,
localAddr net.Addr,
proxyHost string,
reqStream *http3.RequestStream,
address string,
remoteAddr net.Addr,
log logger.Logger,
) (*masque_util.DatagramConn, error) {
// Apply connect timeout to the actual stream
if c.md.connectTimeout > 0 {
@@ -259,14 +325,15 @@ func (c *masqueConnector) connectUDP(ctx context.Context, conn net.Conn, address
// Get the underlying HTTP/3 stream for datagrams
stream := reqStream
// Resolve target address
raddr, err := net.ResolveUDPAddr("udp", address)
if err != nil {
return nil, err
if remoteAddr == nil {
remoteAddr, err = net.ResolveUDPAddr("udp", address)
if err != nil {
return nil, err
}
}
// Create datagram connection wrapping the request stream
datagramConn := masque_util.NewDatagramConnFromRequestStream(stream, conn.LocalAddr(), raddr)
datagramConn := masque_util.NewDatagramConnFromRequestStream(stream, localAddr, remoteAddr)
success = true // Prevent defer from closing stream - datagramConn now owns it
return datagramConn, nil
+24
View File
@@ -0,0 +1,24 @@
package masque
import (
"net/http"
"testing"
)
func TestNewTCPConnectRequestUsesStandardConnect(t *testing.T) {
req := newTCPConnectRequest("example.com:443")
if req.Method != http.MethodConnect {
t.Fatalf("expected CONNECT method, got %s", req.Method)
}
if req.Host != "example.com:443" {
t.Fatalf("expected target authority, got %s", req.Host)
}
if req.Proto != "" {
// quic-go treats any non-empty Proto as an extended CONNECT :protocol.
t.Fatalf("standard CONNECT must not set :protocol, got %q", req.Proto)
}
if req.ProtoMajor != 3 {
t.Fatalf("expected HTTP/3 request, got HTTP/%d", req.ProtoMajor)
}
}
+7 -6
View File
@@ -9,7 +9,6 @@ import (
"strings"
"github.com/go-gost/core/common/bufpool"
"github.com/go-gost/core/connector"
md "github.com/go-gost/core/metadata"
"github.com/go-gost/go-shadowsocks2/core"
@@ -97,10 +96,13 @@ func (c *ssConnector) Connect(ctx context.Context, conn net.Conn, network, addre
log.Error(err)
return nil, err
}
rawaddr := bufpool.Get(512)
defer bufpool.Put(rawaddr)
// The wrapped conn stores this target and only writes it on first I/O,
// so it must be owned by this connection, not a shared pooled buffer
// (see go-gost/gost#896).
const maxAddrLen = 1 + 1 + 255 + 2 // SOCKS5 addr: atyp + len + host + port
n, err := addr.Encode(rawaddr)
target := make([]byte, maxAddrLen)
n, err := addr.Encode(target)
if err != nil {
log.Error("encoding addr: ", err)
return nil, err
@@ -111,8 +113,7 @@ func (c *ssConnector) Connect(ctx context.Context, conn net.Conn, network, addre
defer conn.SetDeadline(time.Time{})
}
target := socks.Addr(rawaddr[:n])
conn, err = c.client.WrapConn(conn, target)
conn, err = c.client.WrapConn(conn, socks.Addr(target[:n]))
if err != nil {
return nil, err
}
+81
View File
@@ -0,0 +1,81 @@
package ss
import (
"bytes"
"context"
"io"
"net"
"net/url"
"testing"
"time"
"github.com/go-gost/core/connector"
"github.com/go-gost/go-shadowsocks2/socks"
"github.com/go-gost/x/internal/util/ss/none"
xlogger "github.com/go-gost/x/logger"
mdx "github.com/go-gost/x/metadata"
)
// captureConn records everything written to it, so a test can inspect the
// bytes a connector actually puts on the wire.
type captureConn struct {
buf bytes.Buffer
}
func (c *captureConn) Read(b []byte) (int, error) { return 0, io.EOF }
func (c *captureConn) Write(b []byte) (int, error) { return c.buf.Write(b) }
func (c *captureConn) Close() error { return nil }
func (c *captureConn) LocalAddr() net.Addr { return &net.TCPAddr{} }
func (c *captureConn) RemoteAddr() net.Addr { return &net.TCPAddr{} }
func (c *captureConn) SetDeadline(time.Time) error { return nil }
func (c *captureConn) SetReadDeadline(time.Time) error { return nil }
func (c *captureConn) SetWriteDeadline(time.Time) error { return nil }
// TestConnectTargetNotOverwrittenByPool is a regression test for
// https://github.com/go-gost/gost/issues/896: the connector previously passed
// a slice aliasing the bufpool buffer to WrapConn, and the buffer was returned
// to the pool before the wrapped conn lazily wrote the target on first I/O. A
// second Connect could then overwrite the bytes and the first conn would send
// the second conn's target. The none cipher keeps the wire plaintext so the
// written target is directly assertable.
func TestConnectTargetNotOverwrittenByPool(t *testing.T) {
c := NewConnector(
connector.AuthOption(url.UserPassword("none", "x")),
connector.LoggerOption(xlogger.NewLogger()),
)
if err := c.Init(mdx.NewMetadata(nil)); err != nil {
t.Fatalf("Init: %v", err)
}
ctx := context.Background()
cap1 := &captureConn{}
conn1, err := c.Connect(ctx, cap1, "tcp", "1.2.3.4:80")
if err != nil {
t.Fatalf("first Connect: %v", err)
}
// Second Connect reuses the same 512-byte bufpool buffer that conn1's
// target used to alias (single pool tier, same goroutine).
if _, err := c.Connect(ctx, &captureConn{}, "tcp", "5.6.7.8:443"); err != nil {
t.Fatalf("second Connect: %v", err)
}
if _, err := conn1.Write([]byte("X")); err != nil {
t.Fatalf("first Write: %v", err)
}
// initWriter frames the target as [salt][2-byte length][addr].
data := cap1.buf.Bytes()
payload := data[none.Cipher.SaltSize():]
if len(payload) < 2 {
t.Fatalf("captured payload too short: %d bytes: %x", len(payload), data)
}
n := int(payload[0])<<8 | int(payload[1])
if len(payload) < 2+n {
t.Fatalf("captured payload too short: len=%d, need %d: %x", len(payload), 2+n, data)
}
addr := socks.SplitAddr(payload[2 : 2+n])
if got, want := addr.String(), "1.2.3.4:80"; got != want {
t.Fatalf("first conn wrote target %q, want %q (captured %x)", got, want, data)
}
}
+16
View File
@@ -1,6 +1,7 @@
package tunnel
import (
"context"
"encoding/binary"
"errors"
"fmt"
@@ -11,6 +12,7 @@ import (
"github.com/go-gost/core/common/bufpool"
mdata "github.com/go-gost/core/metadata"
"github.com/go-gost/relay"
ictx "github.com/go-gost/x/internal/ctx"
xrelay "github.com/go-gost/x/internal/util/relay"
)
@@ -94,6 +96,14 @@ func (c *bindConn) Metadata() mdata.Metadata {
return c.md
}
// Context implements xctx.Context, exposing the connection metadata (e.g. the
// tunneled hostname) to the handler's context. The service accept loop derives
// the handler context from a conn that implements this interface, which lets
// reverse-forward handlers (rtcp) route on the hostname carried by the tunnel.
func (c *bindConn) Context() context.Context {
return ictx.ContextWithMetadata(context.Background(), c.md)
}
type bindUDPConn struct {
net.Conn
localAddr net.Addr
@@ -162,6 +172,12 @@ func (c *bindUDPConn) Metadata() mdata.Metadata {
return c.md
}
// Context implements xctx.Context, exposing the connection metadata (e.g. the
// tunneled hostname) to the handler's context. See bindConn.Context.
func (c *bindUDPConn) Context() context.Context {
return ictx.ContextWithMetadata(context.Background(), c.md)
}
type bindAddr struct {
network string
addr string
+45
View File
@@ -0,0 +1,45 @@
package tunnel
import (
"net"
"testing"
mdata "github.com/go-gost/core/metadata"
ictx "github.com/go-gost/x/internal/ctx"
xmd "github.com/go-gost/x/metadata"
mdutil "github.com/go-gost/x/metadata/util"
)
// TestBindConnContext verifies that a bindConn exposes its connection metadata
// (the tunneled hostname) via the xctx.Context interface, so the service accept
// loop forwards it into the handler context for filter.host routing (gost#898).
func TestBindConnContext(t *testing.T) {
md := mdata.Metadata(xmd.NewMetadata(map[string]any{"host": "example.local:80"}))
cn := &bindConn{
Conn: &net.TCPConn{},
localAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0},
remoteAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 80},
md: md,
}
got := mdutil.GetString(ictx.MetadataFromContext(cn.Context()), "host")
if got != "example.local:80" {
t.Errorf("host from Context() = %q, want %q", got, "example.local:80")
}
}
// TestBindUDPConnContext is the same check for the UDP reverse-tunnel conn.
func TestBindUDPConnContext(t *testing.T) {
md := mdata.Metadata(xmd.NewMetadata(map[string]any{"host": "example.local:80"}))
cn := &bindUDPConn{
Conn: &net.TCPConn{},
localAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0},
remoteAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 80},
md: md,
}
got := mdutil.GetString(ictx.MetadataFromContext(cn.Context()), "host")
if got != "example.local:80" {
t.Errorf("host from Context() = %q, want %q", got, "example.local:80")
}
}
+9
View File
@@ -102,6 +102,15 @@ func (c *MasqueConn) GetRequestStream() *http3.RequestStream {
return c.reqStream
}
// OpenRequestStream opens another request stream on the shared HTTP/3 connection.
// CONNECT-UDP associations use one stream per destination.
func (c *MasqueConn) OpenRequestStream(ctx context.Context) (*http3.RequestStream, error) {
if c.clientConn == nil {
return nil, errors.New("masque: connection is closed")
}
return c.clientConn.OpenRequestStream(ctx)
}
// GetHost returns the proxy host.
func (c *MasqueConn) GetHost() string {
return c.host
+2
View File
@@ -52,6 +52,8 @@ func (d *tlsDialer) Dial(ctx context.Context, addr string, opts ...dialer.DialOp
d.log.Error(err)
}
xnet.ApplyNoDelay(conn)
if d.md.keepalive {
xnet.ApplyKeepalive(conn, net.KeepAliveConfig{
Enable: true,
+2
View File
@@ -65,6 +65,8 @@ func (d *wsDialer) Dial(ctx context.Context, addr string, opts ...dialer.DialOpt
d.options.Logger.Error(err)
}
xnet.ApplyNoDelay(conn)
if d.md.tcpKeepalive {
xnet.ApplyKeepalive(conn, net.KeepAliveConfig{
Enable: true,
-145
View File
@@ -1,145 +0,0 @@
# failCodes — HTTP Status-Based Node Fail Marking
## Context
When GOST acts as HTTP reverse proxy (e.g. LLM API gateway), the upstream
backend can return HTTP 5xx/429 errors while the node itself is healthy at
the transport level. The existing Marker + FailFilter only catches dial-time
failures — there is no way to mark a node unhealthy based on HTTP response
codes.
`failCodes` adds a node-level HTTP setting defining which response status
codes trigger node marking. Cooldown is handled by the existing
`failTimeout`/`maxFails` selector config. No Marker interface changes needed.
No retry — the response is always relayed to the client as-is; marking only
affects subsequent node selection.
## Design
### 1. `core/chain/node.go` — `FailCodes` type
```go
// FailCodes lists response status codes that mark the node as failed.
// Codes < 100 are wildcard hundred-level prefixes (e.g. 5 → 5xx).
type FailCodes []int
func (fc FailCodes) Match(statusCode int) bool {
for _, code := range fc {
if (code < 100 && statusCode/100 == code) || statusCode == code {
return true
}
}
return false
}
```
`HTTPNodeSettings` gets one new field:
```go
FailCodes FailCodes
```
### 2. `x/config/config.go` — config struct
```go
// in HTTPNodeConfig:
FailCodes string `yaml:"failCodes,omitempty" json:"failCodes,omitempty"`
```
Plain comma-separated string — simpler than the original array-of-objects
approach, consistent with the "single string value" metadata convention.
### 3. `x/config/parsing/node/parse.go` — parsing
```go
func parseFailCodes(s string, log logger.Logger) chain.FailCodes {
var codes chain.FailCodes
for _, part := range strings.Split(s, ",") {
part = strings.TrimSpace(part)
if len(part) == 3 && strings.HasSuffix(part, "xx") {
if prefix, err := strconv.Atoi(part[:1]); err == nil && prefix > 0 {
codes = append(codes, prefix) // < 100 → wildcard
continue
}
} else if code, err := strconv.Atoi(part); err == nil && code >= 100 {
codes = append(codes, code)
continue
}
log.Warnf("failCodes: invalid token %q", part)
}
return codes
}
```
Wired inside the `if cfg.HTTP != nil` block in `ParseNode`:
```go
if v := strings.TrimSpace(cfg.HTTP.FailCodes); v != "" {
settings.FailCodes = parseFailCodes(v, nodeLogger)
}
```
### 4. `x/internal/util/forwarder/sniffer_http.go` — runtime
In `httpRoundTrip`, after `ro.HTTP.StatusCode = resp.StatusCode`:
```go
// failCodes: a matching upstream status marks the node failed so the
// selector's FailFilter skips it. The response still relays to the client;
// closing the connection makes the next client request re-select a node.
if hts := node.Options().HTTP; hts != nil && hts.FailCodes.Match(resp.StatusCode) {
log.Warnf("failCodes matched status %d for node %s", resp.StatusCode, node.Name)
if marker := node.Marker(); marker != nil {
marker.Mark()
}
shouldClose = true
}
```
No early return — the response writes through to the client normally.
`shouldClose = true` closes the upstream connection after this request so the
next client request triggers a fresh `hop.Select()`.
## YAML Example
```yaml
chains:
- name: llm-chain
hops:
- name: llm-hop
selector:
strategy: round
maxFails: 1
failTimeout: 15s
nodes:
- name: deepseek-proxy
addr: api.deepseek.com:443
connector: {type: http}
http:
failCodes: "429,502,503,504"
```
**Important**: `maxFails: 1` is recommended. The `dial()` function calls
`marker.Reset()` on successful TCP dials, which can wipe HTTP-based marks
before `maxFails` is reached. With `maxFails: 1`, one `Mark()` immediately
excludes the node.
## Scope
- **Implemented**: sniffer-based forwarder path (`forward/local`,
`forward/remote`).
- **Future work**: HTTP handler direct path (`x/handler/http/proxy.go`).
- **No retry**: retry without idempotency knowledge is unsafe — the response
is always relayed to the client for the client to decide.
## What's NOT changed
- `core/selector/` — Marker/FailFilter untouched
- `core/hop/` — Select() untouched
- `x/chain/router.go` — Router.Dial untouched
- `x/handler/http/proxy.go` — httpHandler direct path (future work)
## Verification
```bash
cd core && go build ./... && go vet ./...
cd x && go build ./... && go vet ./...
```
@@ -1,152 +0,0 @@
# Generic Cache Component + HTTP Response Cache (issue #349)
## Context
Issue [#349](https://github.com/go-gost/gost/issues/349) requests an HTTP response cache so GOST can mirror static sites / CDNs (e.g. gh-pages) via its proxy. GOST currently re-forwards every request to upstream — no caching.
Rather than a one-off HTTP cache, we build a **generic cache component** (interface + memory backend + registry/config wiring), then make the HTTP proxy its first consumer. This is justified — not speculative — because two more consumers are committed roadmap:
- **DNS** ([resolver/cache.go](../../internal/util/resolver/cache.go)) already stores `[]byte`-serializable data with near-identical TTL + serve-stale + async-refresh semantics → migratable to the new interface later.
- **Redis / other backends** are planned beyond in-memory.
Scope decisions carried from prior review rounds:
- Interface trimmed to **3 methods** (`Get`/`Set`/`Delete`). 349's 7-method draft dropped `Has`/`Clear`/`Len` (awkward/expensive on Redis); `RefreshTTL` also dropped — its only consumer is DNS async-refresh ([handler.go:343](../../handler/dns/handler.go#L343)), which is not in this PR, and it is expressible as `Get`+`Set(WithTTL)`. Add it in the DNS-migration PR where it has a real caller and a native Redis `EXPIRE` mapping. The HTTP consumer uses `singleflight`, not `RefreshTTL`, for anti-stampede.
- **HTTP consumer lands in this same PR** as the validating consumer (349 had put it in "Phase 4" — wrong; it's the issue's actual ask). DNS migration and Redis backend are explicit follow-ups, not this PR.
- **Named shared store** referenced by name (`services[].cache: <name>`), not the per-chain-node config from the earlier `http-response-cache-for-reverse-proxy.md` draft (which was per-node, ambiguous with a shared store, and modified `core/chain/node.go`).
- Interface lives in **`core/cache/`** (first-class component, consistent with recorder/limiter — maintainer-approved core addition). HTTP policy configured via **handler metadata**, keeping the generic cache as dumb bytes.
## Architecture
```
caches: (named, registry) ──► registry.CacheRegistry().Get(name) ──► handler.CacheOption
│
services[].cache: cache-0 ▼
handler metadata: cache.ttl / cache.status.* / cache.serveStale ─► httpcache.Cache{Store, Policy, sf}
│ held by SnifferBuilder (built once in Init)
▼ Build() copies ref per connection (shared store)
HandleHTTP ──(before dial)── cache hit? ──► serve from cache
└─(miss)── singleflight ─► dial+roundtrip ─► tee response ─► Store
```
**Key correctness points** (from round-1 review of the sniffer):
- The `Sniffer` is built **per connection** (`SnifferBuilder.Build()`), so the cache/singleflight must live on the **shared** `SnifferBuilder` (built once in `Init`), never on `*Sniffer`. The store itself is a registry singleton, so all handlers/connections referencing the same name share it.
- Cache **hit check goes in `HandleHTTP` before `h.dial`** (and before keep-alive reuse), not inside `httpRoundTrip` — otherwise the upstream dial is already paid and nothing is saved.
- **SNI caching only works with MITM TLS termination.** Non-MITM `HandleTLS` pipes ciphertext ([sniffer_tls.go:116](../../internal/util/sniffing/sniffer_tls.go#L116)) and cannot cache. With `Certificate`+`PrivateKey` set, `terminateTLS` → `HandleHTTP`, so the HTTP cache glue covers SNI-with-MITM automatically — no extra code, just document the requirement.
## Files
### New — the generic component
**`core/cache/cache.go`** (~60 lines) — interface only. First-class component (maintainer-approved core addition). Mirrors `core/recorder/recorder.go` complexity.
```go
package cache
var ErrNotFound = errors.New("cache: key not found")
type Entry struct {
Data []byte
Expiration time.Time // zero = never expires
}
func (e *Entry) Expired() bool
func (e *Entry) TTL() time.Duration // remaining; <=0 expired, -1 never
type SetOptions struct{ TTL time.Duration }
type SetOption func(*SetOptions)
func WithTTL(d time.Duration) SetOption
// Cache is a concurrency-safe []byte KV store. Get returns ErrNotFound ONLY
// when the key is absent; expired entries are returned (with Expired()==true)
// to support serve-stale. Callers serialize/deserialize their own types.
type Cache interface {
Get(ctx context.Context, key string) (*Entry, error)
Set(ctx context.Context, key string, data []byte, opts ...SetOption) error
Delete(ctx context.Context, key string) error
}
```
Three methods only. `Get`/`Set` are load-bearing for HTTP; `Delete` is kept as a fundamental KV primitive (needed for a future PURGE API, unambiguous on every backend). `RefreshTTL` deliberately omitted — see scope decisions above.
**`core/handler/option.go`** (+~6 lines) — add `Cache cache.Cache` field + `func CacheOption(c cache.Cache) Option`, exactly like `RecordersOption` (option.go:40,115).
**`x/cache/memory.go`** (~180 lines) — memory backend implementing `cache.Cache`. Self-contained (no external lib), supersedes the trivial [x/internal/util/cache](../../internal/util/cache/cache.go) and the intent of patrickmn/go-cache. Features: `sync.RWMutex`, per-entry TTL (default from config, `WithTTL` override), `maxSize` + `maxBytes` bounds, eviction `oldest|lru`, optional background cleanup goroutine (stopped via `io.Closer` — registry's `Unregister` calls Close, see [registry.go Unregister](../../registry/registry.go)). No `init()` self-registration (recorder-style; instances are registered by the loader under their config name).
**`x/config/parsing/cache/parse.go`** (~40 lines) — `ParseCache(cfg *config.CacheConfig) cache.Cache`, dispatches on backend field. Only `Memory` now; `Redis`/`File`/`Plugin` are follow-ups (do NOT ship un-parseable empty backend structs). Mirrors [ParseRecorder](../../config/parsing/recorder/parse.go).
**`x/registry/cache.go`** (~40 lines) — `cacheRegistry` embedding `registry[cache.Cache]` + hot-reload `cacheWrapper` delegating each method to the current instance. Copy [x/registry/recorder.go](../../registry/recorder.go) exactly.
**`x/internal/util/httpcache/httpcache.go`** (~150 lines) — HTTP-specific glue over `cache.Cache`, shared by both sniffer packages. Holds all HTTP policy so the generic cache stays dumb bytes:
```go
type Policy struct {
DefaultTTL time.Duration
StatusTTL map[int]time.Duration // per-status (200/302→60m, 404→1m)
ServeStale bool
Methods map[string]bool // default GET, HEAD
MaxBodyBytes int
}
type Cache struct {
Store cache.Cache
Policy Policy
sf singleflight.Group // shared thundering-herd guard
}
func Key(method, host, uri string) string // "GET example.com/api"
func (c *Cache) Lookup(req) (resp *http.Response, stale bool, ok bool)
func (c *Cache) Cacheable(req, resp) bool // method + 2xx/3xx + configured statuses
func (c *Cache) TTLFor(status int) time.Duration
func (c *Cache) TeeWriter(w io.Writer) *teeWriter // captures up to MaxBodyBytes, discards if exceeded
func (c *Cache) Store(req, data []byte, ttl)
```
Serialization uses stdlib `resp.Write` / `http.ReadResponse` (pass the request method so HEAD parses body-less).
### Modified — config & wiring (copy recorder pattern verbatim)
- **`x/config/config.go`** — add `CacheConfig{Name, Memory *MemoryCache}` + `MemoryCache{TTL, MaxSize, MaxBytes, CleanupInterval, Eviction}`; `Config.Caches []*CacheConfig`; `ServiceConfig.Cache string` (name ref, like `Limiter string` at config.go:514).
- **`x/registry/registry.go`** — add `cacheReg` singleton (registry.go:49 style) + `CacheRegistry()` accessor (registry.go:187).
- **`x/config/loader/loader.go`** — add a `registerGroup` block iterating `cfg.Caches` → `ParseCache` → `CacheRegistry()` (copy recorder block loader.go:239-247).
- **`x/config/parsing/service/parse.go`** — look up `registry.CacheRegistry().Get(cfg.Cache)`, read HTTP cache policy from handler metadata, build `httpcache.Cache`, inject via `handler.CacheOption(...)` (alongside `RecordersOption`, parse.go:393). New metadata keys in [parsing/parse.go](../../config/parsing/parse.go): `cache.ttl`, `cache.status.<code>` (via `mdutil.GetStringMapString(md,"cache.status")`), `cache.serveStale`, `cache.methods`, `cache.maxBodyBytes`.
### Modified — HTTP consumers (the feature)
Both sniffer packages get the same edit. `SnifferBuilder` gains a `Cache *httpcache.Cache` field, `Build()` copies it onto `Sniffer`:
- **`x/internal/util/forwarder/`** (`sniffer.go`, `sniffer_http.go`) — serves forward/local + forward/remote.
- **`x/internal/util/sniffing/`** (`sniffer.go`, `sniffer_http.go`) — serves sni + http/connect (+ redirect).
In each `HandleHTTP`: **before `h.dial`** (and before keep-alive reuse at the re-dial branch), if `Cache != nil` and method is cacheable, `Lookup`; on fresh hit write to client (`resp.Write(rw)`), record, and continue the keep-alive loop with correct `shouldClose` (do **not** close the client conn). In `httpRoundTrip`: on miss, run dial+roundtrip inside `sf.Do(key,...)`; if `Cacheable`, wrap `rw` with `TeeWriter` so `resp.Write(tee)` captures bytes, then `Store` after a successful write. On upstream error with a stale entry and `ServeStale`, write the stale response instead of erroring.
The four dispatch sites that call `Build()` are unchanged in shape — they already build a fresh sniffer per connection; the cache reference rides along on the builder:
[forward/local/sniffing.go:79](../../handler/forward/local/sniffing.go#L79), [forward/remote/sniffing.go:79](../../handler/forward/remote/sniffing.go#L79), [sni/sniffing.go:104](../../handler/sni/sniffing.go#L104), [http/connect.go:126](../../handler/http/connect.go#L126). Each handler's `SnifferBuilder` is populated from `h.options.Cache` in `Init`.
## What is NOT in this PR (deferred, interface already accommodates)
- Redis / File / Plugin backends (Redis struct + impl land together in a follow-up).
- DNS / router / limiter / TLS-certpool migration to the new interface (Phase 2+).
- PURGE API, Vary, RFC 7234 conditional requests, disk persistence.
## Verification
```bash
cd x && go build ./... && go vet ./...
cd ../gost && go build ./cmd/gost/...
cd ../core && go build ./... && go vet ./... # interface + handler option
```
Unit tests:
- `x/cache/memory_test.go` — set/get/expire, `WithTTL` override, oldest & lru eviction, maxBytes bound, cleanup goroutine stop on Close, `Get` returns expired entry (serve-stale contract). Run with `CGO_ENABLED=1 go test -race`.
- `x/internal/util/httpcache/httpcache_test.go` — key format, per-status TTL, `Cacheable` (method/status matrix), teeWriter cutoff at `MaxBodyBytes`, response round-trip via `resp.Write`/`http.ReadResponse` incl. HEAD.
End-to-end (memory backend, HTTP proxy):
```yaml
caches:
- name: c0
memory: { maxSize: 1000, maxBytes: 268435456, eviction: lru }
services:
- name: web
addr: ":8080"
handler:
type: http
metadata: { cache: c0, cache.ttl: 60m, cache.status.404: 1m, cache.serveStale: true }
forwarder:
nodes: [{ name: up, addr: "backend:80" }]
```
Two identical GETs: first logs `<-> upstream`, second serves from cache (no upstream dial in trace). Kill the backend after warming the cache → with `serveStale: true`, the stale entry is still served.
```bash
cd gost && go run ./cmd/gost/... -C cache.yml -D # -D trace shows hit/miss/dial
```
For SNI: same works only when the service also has MITM certs configured (otherwise TLS is piped and uncacheable — documented behavior).
+7 -6
View File
@@ -9,12 +9,13 @@ require (
github.com/gin-contrib/cors v1.7.2
github.com/gin-gonic/gin v1.12.0
github.com/go-gost/core v0.6.0
github.com/go-gost/go-shadowsocks2 v0.1.3
github.com/go-gost/go-shadowsocks2 v0.1.4
github.com/go-gost/gosocks4 v0.1.0
github.com/go-gost/gosocks5 v0.5.0
github.com/go-gost/plugin v0.5.0
github.com/go-gost/quic-dissector v0.1.0
github.com/go-gost/relay v0.7.0
github.com/go-gost/tls-dissector v0.3.0
github.com/go-gost/tls-dissector v0.3.1
github.com/go-redis/redis/v8 v8.11.5
github.com/gobwas/glob v0.2.3
github.com/golang/snappy v0.0.4
@@ -26,11 +27,11 @@ require (
github.com/miekg/dns v1.1.61
github.com/mitchellh/go-homedir v1.1.0
github.com/patrickmn/go-cache v2.1.0+incompatible
github.com/pion/dtls/v3 v3.1.1
github.com/pion/dtls/v3 v3.1.4
github.com/pires/go-proxyproto v0.8.1
github.com/prometheus/client_golang v1.19.1
github.com/quic-go/quic-go v0.59.1
github.com/quic-go/webtransport-go v0.10.0
github.com/quic-go/quic-go v0.60.0
github.com/quic-go/webtransport-go v0.11.1
github.com/refraction-networking/utls v1.8.2
github.com/rs/xid v1.3.0
github.com/shadowsocks/go-shadowsocks2 v0.1.5
@@ -110,7 +111,7 @@ require (
github.com/leodido/go-urn v1.4.0 // indirect
github.com/magiconair/properties v1.8.7 // indirect
github.com/mattn/go-isatty v0.0.20 // indirect
github.com/mitchellh/mapstructure v1.5.0 // indirect
github.com/mitchellh/mapstructure v1.5.0
github.com/modern-go/concurrent v0.0.0-20180306012644-bacd9c7ef1dd // indirect
github.com/modern-go/reflect2 v1.0.2 // indirect
github.com/onsi/gomega v1.34.2 // indirect
+14 -10
View File
@@ -53,18 +53,20 @@ github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8=
github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc=
github.com/go-gost/core v0.6.0 h1:Y7jG5bI0QxdzDlMOyEtAYxCz+0GxAHKmaei667pRPWc=
github.com/go-gost/core v0.6.0/go.mod h1:WGI43jOka7FAsSAwi/fSMaqxdR+E339ycb4NBGlFr6A=
github.com/go-gost/go-shadowsocks2 v0.1.3 h1:6CUZLp+mTWXnKP2aK8/Z9ZP+ERMX9gSbywmPu4kGX/A=
github.com/go-gost/go-shadowsocks2 v0.1.3/go.mod h1:866zFNNI3He6Wef1M/IvAjTal74WhcfKfBgRpTlkKys=
github.com/go-gost/go-shadowsocks2 v0.1.4 h1:n2Po4TDKdLp1PsvhSiWFB/6S4e/YKZfsKJkA0PUa168=
github.com/go-gost/go-shadowsocks2 v0.1.4/go.mod h1:866zFNNI3He6Wef1M/IvAjTal74WhcfKfBgRpTlkKys=
github.com/go-gost/gosocks4 v0.1.0 h1:eAzev6qw4fzkFQKC9uCHLVNnnPdHyqCggbnfNN80Pmk=
github.com/go-gost/gosocks4 v0.1.0/go.mod h1:hzVjwijJuZR1pp3GqpTj+AKcSGrx68RlWTrQMFMYBP0=
github.com/go-gost/gosocks5 v0.5.0 h1:YE37l1MJwde8diIQdynStqogMotG5enoTdborhA5yic=
github.com/go-gost/gosocks5 v0.5.0/go.mod h1:1G6I7HP7VFVxveGkoK8mnprnJqSqJjdcASKsdUn4Pp4=
github.com/go-gost/plugin v0.5.0 h1:zK3Ezcv+z6bz6aUhaFZcZiJc42w44ohybzrj5tnWzcM=
github.com/go-gost/plugin v0.5.0/go.mod h1:oN23l+yGDCIP9G3KnDl/I/0zVGOobZUDCB2Z5yYYXts=
github.com/go-gost/quic-dissector v0.1.0 h1:zOaw2XjKxMf6vVC1z4BHDKGHbCs4zrYU1Rxaz5d0TQU=
github.com/go-gost/quic-dissector v0.1.0/go.mod h1:ar+I40Izq9ZT2k/aNnLFGEMvdeSOBLLKxhF/i3FSnQQ=
github.com/go-gost/relay v0.7.0 h1:J8e3Sba6DtBJQotXY5j5EaZNWwtv6Z9CoCFQsnrHNcE=
github.com/go-gost/relay v0.7.0/go.mod h1:Dku0f5sfjOClrZFiDmQUrYYJ4uof7rnkCUBfsl0PSAI=
github.com/go-gost/tls-dissector v0.3.0 h1:mbjmcHFXYRMUXGGTbJ78dllg6utjK3gp9Rn04tda4UA=
github.com/go-gost/tls-dissector v0.3.0/go.mod h1:vGfog053fIm93iXBtvmVzMQqEJo5YwbbNaPNVDjbiOc=
github.com/go-gost/tls-dissector v0.3.1 h1:gvOteWog5pjY/HCpc8l+gngmSi8Q6zl5rRrfK8gwRKA=
github.com/go-gost/tls-dissector v0.3.1/go.mod h1:vGfog053fIm93iXBtvmVzMQqEJo5YwbbNaPNVDjbiOc=
github.com/go-logr/logr v1.4.3 h1:CjnDlHq8ikf6E492q6eKboGOC0T8CDaOvkHCIg8idEI=
github.com/go-logr/logr v1.4.3/go.mod h1:9T104GzyrTigFIr8wt5mBrctHMim0Nb2HLGrmQ40KvY=
github.com/go-logr/stdr v1.2.2 h1:hSWxHoqTgW2S2qGc0LTAI563KZ5YKYRhT3MFKZMbjag=
@@ -171,8 +173,8 @@ github.com/patrickmn/go-cache v2.1.0+incompatible h1:HRMgzkcYKYpi3C8ajMPV8OFXaaR
github.com/patrickmn/go-cache v2.1.0+incompatible/go.mod h1:3Qf8kWWT7OJRJbdiICTKqZju1ZixQ/KpMGzzAfe6+WQ=
github.com/pelletier/go-toml/v2 v2.2.4 h1:mye9XuhQ6gvn5h28+VilKrrPoQVanw5PMw/TB0t5Ec4=
github.com/pelletier/go-toml/v2 v2.2.4/go.mod h1:2gIqNv+qfxSVS7cM2xJQKtLSTLUE9V8t9Stt+h56mCY=
github.com/pion/dtls/v3 v3.1.1 h1:wSLMam9Kf7DL1A74hnqRvEb9OT+aXPAsQ5VS+BdXOJ0=
github.com/pion/dtls/v3 v3.1.1/go.mod h1:7FGvVYpHsUV6+aywaFpG7aE4Vz8nBOx74odPRFue6cI=
github.com/pion/dtls/v3 v3.1.4 h1:QhvtMflMfu9Kf0RcDC5BJBle4caPskByrKQR6uuYqpY=
github.com/pion/dtls/v3 v3.1.4/go.mod h1:cr/qotLISUw/9C1m83ZPNZtj9WnXkYLpfCptPqbkInc=
github.com/pion/logging v0.2.4 h1:tTew+7cmQ+Mc1pTBLKH2puKsOvhm32dROumOZ655zB8=
github.com/pion/logging v0.2.4/go.mod h1:DffhXTKYdNZU+KtJ5pyQDjvOAh/GsNSyv1lbkFbe3so=
github.com/pion/transport/v4 v4.0.1 h1:sdROELU6BZ63Ab7FrOLn13M6YdJLY20wldXW2Cu2k8o=
@@ -195,12 +197,14 @@ github.com/prometheus/common v0.48.0 h1:QO8U2CdOzSn1BBsmXJXduaaW+dY/5QLjfB8svtSz
github.com/prometheus/common v0.48.0/go.mod h1:0/KsvlIEfPQCQ5I2iNSAWKPZziNCvRs5EC6ILDTlAPc=
github.com/prometheus/procfs v0.12.0 h1:jluTpSng7V9hY0O2R9DzzJHYb2xULk9VTR1V1R/k6Bo=
github.com/prometheus/procfs v0.12.0/go.mod h1:pcuDEFsWDnvcgNzo4EEweacyhjeA9Zk3cnaOZAZEfOo=
github.com/quic-go/go-ossfuzz-seeds v0.1.0 h1:APacT+iIaNF6fd8AGEiN3bT/Jtkd2jz4v4TzM7MFjy0=
github.com/quic-go/go-ossfuzz-seeds v0.1.0/go.mod h1:3IOHRbJIc+L6YKMwfDtJAM9Vj9k0YY4muhuyUYk5tbk=
github.com/quic-go/qpack v0.6.0 h1:g7W+BMYynC1LbYLSqRt8PBg5Tgwxn214ZZR34VIOjz8=
github.com/quic-go/qpack v0.6.0/go.mod h1:lUpLKChi8njB4ty2bFLX2x4gzDqXwUpaO1DP9qMDZII=
github.com/quic-go/quic-go v0.59.1 h1:0Gmua0HW1Tv7ANR7hUYwRyD0MG5OJfgvYSZasGZzBic=
github.com/quic-go/quic-go v0.59.1/go.mod h1:upnsH4Ju1YkqpLXC305eW3yDZ4NfnNbmQRCMWS58IKU=
github.com/quic-go/webtransport-go v0.10.0 h1:LqXXPOXuETY5Xe8ITdGisBzTYmUOy5eSj+9n4hLTjHI=
github.com/quic-go/webtransport-go v0.10.0/go.mod h1:LeGIXr5BQKE3UsynwVBeQrU1TPrbh73MGoC6jd+V7ow=
github.com/quic-go/quic-go v0.60.0 h1:xcQioE8OM66UQLeUMHltK1CCcOu3JbVB4JAQdDQSB+0=
github.com/quic-go/quic-go v0.60.0/go.mod h1:wpKpjmPpftl30sL6pFh7REVpjbcCVy4zt2vDyK1TuJk=
github.com/quic-go/webtransport-go v0.11.1 h1:rrFQMO+7/52ZDJ04fsrjIaWqn6q1z1MYo9iVFq6JtbA=
github.com/quic-go/webtransport-go v0.11.1/go.mod h1:SHgEzUFVyj+9WUSuGB1P6Zd351Pww2leWV3SwlTovkA=
github.com/refraction-networking/utls v1.8.2 h1:j4Q1gJj0xngdeH+Ox/qND11aEfhpgoEvV+S9iJ2IdQo=
github.com/refraction-networking/utls v1.8.2/go.mod h1:jkSOEkLqn+S/jtpEHPOsVv/4V4EVnelwbMQl4vCWXAM=
github.com/riobard/go-bloom v0.0.0-20200614022211-cdc8013cb5b3 h1:f/FNXud6gA3MNr8meMVVGxhp+QBTqY91tM8HjEuMjGg=
+75 -15
View File
@@ -3,16 +3,21 @@ package local
import (
"bytes"
"context"
"io"
"net"
"time"
quicdissector "github.com/go-gost/quic-dissector"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/bypass"
xbypass "github.com/go-gost/x/bypass"
xctx "github.com/go-gost/x/ctx"
ictx "github.com/go-gost/x/internal/ctx"
xnet "github.com/go-gost/x/internal/net"
"github.com/go-gost/x/internal/net/proxyproto"
"github.com/go-gost/x/internal/util/sniffing"
xrecorder "github.com/go-gost/x/recorder"
)
@@ -100,13 +105,41 @@ func (h *forwardHandler) dialTarget(ctx context.Context, conn net.Conn, ro *xrec
// handleRawForwarding performs node selection, dials the target through the
// router, and pipes the raw connection. It is used when sniffing is disabled
// or the protocol was not HTTP/TLS.
func (h *forwardHandler) handleRawForwarding(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, network, proto string) error {
// When proto is "quic" and clientDgram is set, the first upstream datagram is
// intercepted to parse the TLS ServerHello, then threaded back via
// io.MultiReader before the pipe.
func (h *forwardHandler) handleRawForwarding(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, network, proto string, clientDgram []byte) error {
dr, err := h.dialTarget(ctx, conn, ro, log, network, proto)
if err != nil {
return err
}
defer dr.cc.Close()
// QUIC ServerHello sniff from upstream response.
if proto == sniffing.ProtoQUIC && ro.TLS != nil {
if h.md.sniffingTimeout > 0 {
dr.cc.SetReadDeadline(time.Now().Add(h.md.sniffingTimeout))
}
sdgram := make([]byte, h.md.bufferSize)
n, readErr := dr.cc.Read(sdgram)
if h.md.sniffingTimeout > 0 {
dr.cc.SetReadDeadline(time.Time{})
}
if readErr != nil {
dr.log.Debugf("udp server sniff read: %v", readErr)
}
if n > 0 {
sdgram = sdgram[:n]
if sh, shErr := quicdissector.SniffQUICServerHello(clientDgram, sdgram); shErr == nil {
sniffing.PopulateQUICServerHello(sh, ro)
}
dr.cc = xnet.NewReadWriteConn(io.MultiReader(bytes.NewReader(sdgram), dr.cc), dr.cc, dr.cc)
}
}
t := time.Now()
log.Infof("%s <-> %s", conn.RemoteAddr(), dr.target.Addr)
if err := xnet.Pipe(ctx, conn, dr.cc, xnet.WithReadTimeout(h.md.idleTimeout)); err != nil {
@@ -127,8 +160,36 @@ func (h *forwardHandler) handleRawForwarding(ctx context.Context, conn net.Conn,
//
// The method reuses dialTarget for the shared hop-selection, Router.Dial,
// proxyproto, and recorder preamble.
func (h *forwardHandler) handleRawDatagram(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, network, proto string) error {
dr, err := h.dialTarget(ctx, conn, ro, log, network, proto)
func (h *forwardHandler) handleRawDatagram(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, network string) error {
bufp := make([]byte, h.md.bufferSize)
n, err := conn.Read(bufp)
if err != nil {
log.Debugf("conn read: %v", err)
return err
}
clientDgram := bufp[:n]
if h.md.sniffing {
if info, sniffErr := quicdissector.SniffQUIC(clientDgram); sniffErr == nil {
sniffing.PopulateQUICClientHello(clientDgram, info, ro)
if info.ServerName != "" {
ro.Host = net.JoinHostPort(info.ServerName, "443")
log = log.WithFields(map[string]any{
"host": ro.Host,
"proto": ro.Proto,
})
if h.options.Bypass != nil &&
h.options.Bypass.Contains(ctx, "udp", ro.Host, bypass.WithService(h.options.Service)) {
log.Debug("bypass: ", ro.Host)
return xbypass.ErrBypass
}
}
}
}
dr, err := h.dialTarget(ctx, conn, ro, log, network, ro.Proto)
if err != nil {
return err
}
@@ -137,19 +198,12 @@ func (h *forwardHandler) handleRawDatagram(ctx context.Context, conn net.Conn, r
t := time.Now()
log.Infof("%s <-> %s", conn.RemoteAddr(), dr.target.Addr)
bufp := make([]byte, h.md.bufferSize)
n, err := conn.Read(bufp)
if err != nil {
log.Debugf("conn read: %v", err)
return err
}
if _, err := dr.cc.Write(bufp[:n]); err != nil {
if _, err := dr.cc.Write(clientDgram); err != nil {
log.Debugf("outbound write: %v", err)
return err
}
// Read the response into the same buffer to avoid a second allocation.
// Read the response.
if err := dr.cc.SetReadDeadline(time.Now().Add(h.md.readTimeout)); err != nil {
log.Debugf("set read deadline: %v", err)
return err
@@ -159,11 +213,17 @@ func (h *forwardHandler) handleRawDatagram(ctx context.Context, conn net.Conn, r
log.Debugf("outbound read: %v", err)
return err
}
// Clear the deadline so it doesn't affect subsequent use of the
// underlying connection (e.g., keep-alive pools).
dr.cc.SetReadDeadline(time.Time{})
if _, err := conn.Write(bufp[:n]); err != nil {
respDgram := bufp[:n]
if ro.Proto == sniffing.ProtoQUIC {
if sh, shErr := quicdissector.SniffQUICServerHello(clientDgram, respDgram); shErr == nil {
sniffing.PopulateQUICServerHello(sh, ro)
}
}
if _, err := conn.Write(respDgram); err != nil {
log.Debugf("conn write: %v", err)
return err
}
+7 -7
View File
@@ -25,7 +25,7 @@ func TestHandleRawForwarding_NilHop(t *testing.T) {
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "")
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "", nil)
if err == nil {
t.Fatal("expected error when hop is nil")
}
@@ -41,7 +41,7 @@ func TestHandleRawForwarding_NodeUnavailable(t *testing.T) {
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "")
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "", nil)
if err == nil {
t.Fatal("expected error when node is nil")
}
@@ -65,7 +65,7 @@ func TestHandleRawForwarding_DialError(t *testing.T) {
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "")
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "", nil)
if err == nil {
t.Fatal("expected error when dial fails")
}
@@ -93,7 +93,7 @@ func TestHandleRawForwarding_Success(t *testing.T) {
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "")
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "", nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -131,7 +131,7 @@ func TestHandleRawForwarding_AddrWithoutPort(t *testing.T) {
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "")
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "", nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -162,7 +162,7 @@ func TestHandleRawForwarding_UnixSocket(t *testing.T) {
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "")
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "", nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
@@ -190,7 +190,7 @@ func TestHandleRawForwarding_MarkerReset(t *testing.T) {
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "")
err := h.handleRawForwarding(context.Background(), conn, ro, nopLog(), "tcp", "", nil)
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
+70 -19
View File
@@ -62,16 +62,22 @@ package local
import (
"bufio"
"bytes"
"context"
"io"
"net"
"sync"
"time"
quicdissector "github.com/go-gost/quic-dissector"
"github.com/go-gost/core/bypass"
"github.com/go-gost/core/handler"
"github.com/go-gost/core/hop"
md "github.com/go-gost/core/metadata"
"github.com/go-gost/core/observer/stats"
"github.com/go-gost/core/recorder"
xbypass "github.com/go-gost/x/bypass"
xnet "github.com/go-gost/x/internal/net"
"github.com/go-gost/x/internal/util/httpcache"
"github.com/go-gost/x/internal/util/sniffing"
@@ -209,32 +215,77 @@ func (h *forwardHandler) Handle(ctx context.Context, conn net.Conn, opts ...hand
}
if h.md.stateless {
return h.handleRawDatagram(ctx, conn, ro, log, network, "udp")
return h.handleRawDatagram(ctx, conn, ro, log, network)
}
var proto string
if network == "tcp" && h.md.sniffing {
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Now().Add(h.md.sniffingTimeout))
}
var clientDgram []byte
br := bufio.NewReader(conn)
proto, err = sniffing.Sniff(ctx, br)
ro.Proto = proto
if err != nil {
log.Debugf("sniff: %v", err)
}
if h.md.sniffing {
if network == "tcp" {
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Now().Add(h.md.sniffingTimeout))
}
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Time{})
}
br := bufio.NewReader(conn)
proto, err = sniffing.Sniff(ctx, br)
ro.Proto = proto
if err != nil {
log.Debugf("sniff: %v", err)
}
conn = xnet.NewReadWriteConn(br, conn, conn)
handled, sniffErr := h.handleSniffedProtocol(ctx, conn, ro, log, proto)
if handled {
return sniffErr
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Time{})
}
conn = xnet.NewReadWriteConn(br, conn, conn)
handled, sniffErr := h.handleSniffedProtocol(ctx, conn, ro, log, proto)
if handled {
ro.Time = time.Time{}
return sniffErr
}
} else {
// UDP QUIC ClientHello sniff.
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Now().Add(h.md.sniffingTimeout))
}
dgram := make([]byte, h.md.bufferSize)
n, readErr := conn.Read(dgram)
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Time{})
}
if readErr != nil {
log.Debugf("udp sniff read: %v", readErr)
return readErr
}
dgram = dgram[:n]
if info, sniffErr := quicdissector.SniffQUIC(dgram); sniffErr == nil {
clientDgram = dgram
proto = sniffing.ProtoQUIC
sniffing.PopulateQUICClientHello(dgram, info, ro)
if info.ServerName != "" {
ro.Host = net.JoinHostPort(info.ServerName, "443")
log = log.WithFields(map[string]any{
"host": ro.Host,
"proto": ro.Proto,
})
if h.options.Bypass != nil &&
h.options.Bypass.Contains(ctx, "udp", ro.Host, bypass.WithService(h.options.Service)) {
log.Debug("bypass: ", ro.Host)
return xbypass.ErrBypass
}
}
}
conn = xnet.NewReadWriteConn(io.MultiReader(bytes.NewReader(dgram), conn), conn, conn)
}
}
return h.handleRawForwarding(ctx, conn, ro, log, network, proto)
return h.handleRawForwarding(ctx, conn, ro, log, network, proto, clientDgram)
}
+12 -28
View File
@@ -72,35 +72,19 @@ func (h *forwardHandler) sniffingDial(ctx context.Context, network, address stri
}
// handleSniffedProtocol dispatches a sniffed connection to the protocol-specific
// sniffer (HandleHTTP or HandleTLS). It returns (true, err) when the protocol was
// sniffer registered for proto. It returns (true, err) when the protocol was
// handled and (false, nil) when the caller should fall through to raw forwarding.
func (h *forwardHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithHTTPKeepalive(h.md.httpKeepalive),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
return true, sniffer.HandleTLS(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
return forwarder.Dispatch(h.sniffer.Build(), ctx, conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithHop(h.getHop()),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithHTTPKeepalive(h.md.httpKeepalive),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+9
View File
@@ -93,6 +93,15 @@ func (h *forwardHandler) handleRawForwarding(ctx context.Context, conn net.Conn,
// metadata when set, then falling back to hop selection.
func (h *forwardHandler) selectTarget(ctx context.Context, proto string) *chain.Node {
if host := mdutil.GetString(ictx.MetadataFromContext(ctx), "host"); host != "" {
// Route the tunneled hostname through the hop so a node with a matching
// filter.host is selected (the node's addr is the real target). If the
// hop has no matching node (e.g. no filter configured), fall back to
// dialing the hostname directly — the pre-regression behavior.
if curHop := h.getHop(); curHop != nil {
if node := curHop.Select(ctx, hop.ProtocolSelectOption(proto), hop.HostSelectOption(host)); node != nil {
return node
}
}
return &chain.Node{Addr: host}
}
if curHop := h.getHop(); curHop != nil {
+46 -6
View File
@@ -201,7 +201,52 @@ func TestHandleRawForwarding_MarkerReset(t *testing.T) {
}
}
func TestHandleRawForwarding_ContextHost(t *testing.T) {
// TestHandleRawForwarding_ContextHostHop: a tunneled hostname with a hop whose
// node matches the host filter routes to that node's addr (the documented
// filter.host client routing). The node addr is the real target.
func TestHandleRawForwarding_ContextHostHop(t *testing.T) {
server, client := net.Pipe()
defer server.Close()
defer client.Close()
var gotHost string
h := newInitdHandler(withRouter(&mockRouter{
opts: &chain.RouterOptions{},
dialFn: func(ctx context.Context, network, address string) (net.Conn, error) {
return server, nil
},
}))
h.Forward(&mockHop{
selectFn: func(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
var so hop.SelectOptions
for _, o := range opts {
o(&so)
}
gotHost = so.Host
return &chain.Node{Name: "example", Addr: "10.0.0.2:80"}
},
})
ctx := ictx.ContextWithMetadata(context.Background(), xmd.NewMetadata(map[string]any{
"host": "example.local:80",
}))
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(ctx, conn, ro, nopLog(), "tcp", "")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if gotHost != "example.local:80" {
t.Errorf("expected host passed to hop selection, got '%s'", gotHost)
}
if ro.Host != "10.0.0.2:80" {
t.Errorf("expected node addr to be dialed, got '%s'", ro.Host)
}
}
// TestHandleRawForwarding_ContextHostNoHop: when there is no hop, the tunneled
// hostname is dialed directly (fallback preserving the pre-regression behavior).
func TestHandleRawForwarding_ContextHostNoHop(t *testing.T) {
server, client := net.Pipe()
defer server.Close()
defer client.Close()
@@ -212,11 +257,6 @@ func TestHandleRawForwarding_ContextHost(t *testing.T) {
return server, nil
},
}))
h.Forward(&mockHop{
selectFn: func(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
return &chain.Node{Addr: "10.0.0.2:80"}
},
})
ctx := ictx.ContextWithMetadata(context.Background(), xmd.NewMetadata(map[string]any{
"host": "10.0.0.99:8080",
}))
+1
View File
@@ -228,6 +228,7 @@ func (h *forwardHandler) Handle(ctx context.Context, conn net.Conn, opts ...hand
conn = xnet.NewReadWriteConn(br, conn, conn)
handled, sniffErr := h.handleSniffedProtocol(ctx, conn, ro, log, proto)
if handled {
ro.Time = time.Time{}
return sniffErr
}
}
+12 -28
View File
@@ -72,35 +72,19 @@ func (h *forwardHandler) sniffingDial(ctx context.Context, network, address stri
}
// handleSniffedProtocol dispatches a sniffed connection to the protocol-specific
// sniffer (HandleHTTP or HandleTLS). It returns (true, err) when the protocol was
// sniffer registered for proto. It returns (true, err) when the protocol was
// handled and (false, nil) when the caller should fall through to raw forwarding.
func (h *forwardHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithHTTPKeepalive(h.md.httpKeepalive),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
return true, sniffer.HandleTLS(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
return forwarder.Dispatch(h.sniffer.Build(), ctx, conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithHop(h.getHop()),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithHTTPKeepalive(h.md.httpKeepalive),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+9 -22
View File
@@ -80,6 +80,7 @@ func (h *httpHandler) handleConnect(ctx context.Context, conn net.Conn, ro *xrec
if h.md.sniffing {
snifferHandled, err = h.sniffAndHandle(ctx, conn, cc, ro, log)
if snifferHandled {
ro.Time = time.Time{}
return err
}
}
@@ -127,28 +128,14 @@ func (h *httpHandler) sniffAndHandle(ctx context.Context, conn net.Conn, cc net.
sniffer := h.sniffer.Build()
conn = xnet.NewReadWriteConn(br, conn, conn)
switch proto {
case sniffing.ProtoHTTP:
return true, sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoTLS:
return true, sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
return false, nil
return sniffing.Dispatch(sniffer, ctx, "tcp", conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
// dial establishes an upstream connection through the proxy chain router.
+19 -17
View File
@@ -368,9 +368,6 @@ func (h *masqueHandler) handleConnectUDP(ctx context.Context, w http.ResponseWri
return ErrDatagramNotSupport
}
// Get the underlying HTTP/3 stream
stream := streamer.HTTPStream()
// Get target connection - either through router/chain or direct
var targetPC net.PacketConn
@@ -393,15 +390,17 @@ func (h *masqueHandler) handleConnectUDP(ctx context.Context, w http.ResponseWri
ro.SrcAddr = c.LocalAddr().String()
// The connection from router should be a PacketConn (e.g., from masque connector)
if pc, ok := c.(net.PacketConn); ok {
targetPC = pc
log.Debugf("relaying UDP to %s via chain", targetAddr)
} else {
// Wrap as PacketConn if it's a regular Conn
targetPC = &connPacketConn{Conn: c, raddr: raddr}
log.Debugf("relaying UDP to %s via chain (wrapped)", targetAddr)
}
// Always wrap the router-returned conn as a point-to-point PacketConn
// that writes via Conn.Write against the fixed target. A direct UDP
// dial returns a *net.UDPConn that satisfies net.PacketConn but is
// pre-connected, so calling WriteTo on it fails with "use of WriteTo
// with pre-connected connection" and no packet ever reaches the target.
// Routing writes through Conn.Write works for both direct dials and
// stream-based upstream MASQUE connectors. The MASQUE target is fixed
// for the connection lifetime (parsed from the request path), so a
// single remote address is always correct.
targetPC = &connPacketConn{Conn: c, raddr: raddr}
log.Debugf("relaying UDP to %s via chain", targetAddr)
} else {
// Direct connection - create local UDP socket
directConn, err := net.ListenPacket("udp", "")
@@ -424,14 +423,17 @@ func (h *masqueHandler) handleConnectUDP(ctx context.Context, w http.ResponseWri
// Wrap target with metrics
targetPC = metrics.WrapPacketConn(h.options.Service, targetPC)
// Send success response with capsule-protocol header
// Send the success response WITH the mandatory Capsule-Protocol header
// (RFC 9298), then hijack the stream. HTTPStream() flushes the response
// internally and commits the headers, so Capsule-Protocol MUST be set
// before it; otherwise the client never sees Capsule-Protocol: ?1 and
// rejects the tunnel. Dialing the target first (above) means we only
// respond 200 once the relay path is ready, matching handleConnectTCP.
w.Header().Set("Capsule-Protocol", "?1")
w.WriteHeader(http.StatusOK)
// Flush the response headers
if flusher, ok := w.(http.Flusher); ok {
flusher.Flush()
}
// Hijack the underlying HTTP/3 stream for datagram exchange.
stream := streamer.HTTPStream()
// Create datagram connection wrapping the HTTP/3 stream (client side)
datagramConn := masque_util.NewDatagramConn(stream, laddr, raddr)
+25 -3
View File
@@ -5,6 +5,7 @@ import (
"bytes"
"context"
"crypto/tls"
"io"
"net"
"strings"
"time"
@@ -224,10 +225,16 @@ func (h *redirectHandler) Handle(ctx context.Context, conn net.Conn, opts ...han
ReadTimeout: h.md.readTimeout,
}
conn = xnet.NewReadWriteConn(br, conn, conn)
// Capture bytes consumed by the sniffer so we can replay
// them when sniffingFallback is enabled.
origConn := conn
capture := new(bytes.Buffer)
sniffConn := xnet.NewReadWriteConn(io.TeeReader(br, capture), origConn, origConn)
var sniffErr error
switch proto {
case sniffing.ProtoHTTP:
return sniffer.HandleHTTP(ctx, "tcp", conn,
sniffErr = sniffer.HandleHTTP(ctx, "tcp", sniffConn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
@@ -235,8 +242,12 @@ func (h *redirectHandler) Handle(ctx context.Context, conn net.Conn, opts ...han
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
if sniffErr == nil {
ro.Time = time.Time{}
return nil
}
case sniffing.ProtoTLS:
return sniffer.HandleTLS(ctx, ro.Network, conn,
sniffErr = sniffer.HandleTLS(ctx, ro.Network, sniffConn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
@@ -244,6 +255,17 @@ func (h *redirectHandler) Handle(ctx context.Context, conn net.Conn, opts ...han
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
if sniffErr == nil {
return nil
}
}
if h.md.sniffingFallback {
log.Debugf("sniffing(%s) failed, falling back: %v", proto, sniffErr)
conn = xnet.NewReadWriteConn(io.MultiReader(capture, br), origConn, origConn)
// fall through to raw forwarding below
} else {
return sniffErr
}
}
+72
View File
@@ -3,9 +3,12 @@ package redirect
import (
"bytes"
"context"
"io"
"net"
"time"
quicdissector "github.com/go-gost/quic-dissector"
"github.com/go-gost/core/bypass"
"github.com/go-gost/core/handler"
md "github.com/go-gost/core/metadata"
@@ -15,6 +18,7 @@ import (
xctx "github.com/go-gost/x/ctx"
ictx "github.com/go-gost/x/internal/ctx"
xnet "github.com/go-gost/x/internal/net"
"github.com/go-gost/x/internal/util/sniffing"
rate_limiter "github.com/go-gost/x/limiter/rate"
xstats "github.com/go-gost/x/observer/stats"
stats_wrapper "github.com/go-gost/x/observer/stats/wrapper"
@@ -128,6 +132,49 @@ func (h *redirectHandler) Handle(ctx context.Context, conn net.Conn, opts ...han
return xbypass.ErrBypass
}
var clientDgram []byte // saved for QUIC ServerHello sniff from upstream
if h.md.sniffing {
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Now().Add(h.md.sniffingTimeout))
}
dgram := make([]byte, 8192)
n, readErr := conn.Read(dgram)
dgram = dgram[:n]
if h.md.sniffingTimeout > 0 {
conn.SetReadDeadline(time.Time{})
}
if readErr != nil {
log.Debugf("udp sniff read: %v", readErr)
return readErr
}
info, sniffErr := quicdissector.SniffQUIC(dgram)
if sniffErr == nil {
clientDgram = dgram
sniffing.PopulateQUICClientHello(dgram, info, ro)
if info.ServerName != "" {
ro.Host = net.JoinHostPort(info.ServerName, "443")
log = log.WithFields(map[string]any{
"host": ro.Host,
"proto": ro.Proto,
})
if h.options.Bypass != nil &&
h.options.Bypass.Contains(ctx, "udp", ro.Host, bypass.WithService(h.options.Service)) {
log.Debug("bypass: ", ro.Host)
return xbypass.ErrBypass
}
}
}
conn = xnet.NewReadWriteConn(io.MultiReader(bytes.NewReader(dgram), conn), conn, conn)
}
var buf bytes.Buffer
cc, err := h.options.Router.Dial(ictx.ContextWithBuffer(ctx, &buf), dstAddr.Network(), dstAddr.String())
ro.Route = buf.String()
@@ -137,6 +184,31 @@ func (h *redirectHandler) Handle(ctx context.Context, conn net.Conn, opts ...han
}
defer cc.Close()
// QUIC ServerHello sniff from upstream response.
if clientDgram != nil {
if h.md.sniffingTimeout > 0 {
cc.SetReadDeadline(time.Now().Add(h.md.sniffingTimeout))
}
sdgram := make([]byte, 8192)
n, readErr := cc.Read(sdgram)
if h.md.sniffingTimeout > 0 {
cc.SetReadDeadline(time.Time{})
}
if readErr != nil {
log.Debugf("udp server sniff read: %v", readErr)
}
if n > 0 {
sdgram = sdgram[:n]
if sh, shErr := quicdissector.SniffQUICServerHello(clientDgram, sdgram); shErr == nil {
sniffing.PopulateQUICServerHello(sh, ro)
}
cc = xnet.NewReadWriteConn(io.MultiReader(bytes.NewReader(sdgram), cc), cc, cc)
}
}
log = log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
ro.SrcAddr = cc.LocalAddr().String()
ro.DstAddr = cc.RemoteAddr().String()
+10 -1
View File
@@ -1,11 +1,20 @@
package redirect
import (
"time"
mdata "github.com/go-gost/core/metadata"
mdutil "github.com/go-gost/x/metadata/util"
)
type metadata struct{}
type metadata struct {
sniffing bool
sniffingTimeout time.Duration
}
func (h *redirectHandler) parseMetadata(md mdata.Metadata) (err error) {
h.md.sniffing = mdutil.GetBool(md, "sniffing")
h.md.sniffingTimeout = mdutil.GetDuration(md, "sniffing.timeout")
return
}
+13 -19
View File
@@ -231,25 +231,19 @@ func (h *relayHandler) handleConnect(ctx context.Context, conn net.Conn, network
}
conn = xnet.NewReadWriteConn(br, conn, conn)
switch proto {
case sniffing.ProtoHTTP:
return sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoTLS:
return sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
handled, err := sniffing.Dispatch(sniffer, ctx, "tcp", conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
if handled {
// Reset the recording start time: the sniffer handles the
// connection and manages ro.Time from here on.
ro.Time = time.Time{}
return err
}
}
+1
View File
@@ -189,6 +189,7 @@ func (h *sniHandler) Handle(ctx context.Context, conn net.Conn, opts ...handler.
conn = xnet.NewReadWriteConn(br, conn, conn)
handled, sniffErr := h.handleSniffedProtocol(ctx, conn, ro, log, proto)
if handled {
ro.Time = time.Time{}
return sniffErr
}
+14 -31
View File
@@ -90,41 +90,24 @@ func (h *sniHandler) sniffingDialTLS(ctx context.Context, network, address strin
}
// handleSniffedProtocol dispatches a sniffed connection to the
// protocol-specific sniffer ([sniffing.Sniffer.HandleHTTP] or
// [sniffing.Sniffer.HandleTLS]).
// protocol-specific sniffer registered for proto.
//
// It returns (true, err) when the protocol was handled and (false, nil)
// when the protocol is unrecognised and the caller should handle the
// connection directly (which for the SNI handler means silently dropping it).
func (h *sniHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
dialTLS := func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error) {
return h.sniffingDialTLS(ctx, network, address, ro, cfg)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
return true, sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
dialTLS := func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error) {
return h.sniffingDialTLS(ctx, network, address, ro, cfg)
}
return sniffing.Dispatch(h.sniffer.Build(), ctx, "tcp", conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+2
View File
@@ -233,6 +233,7 @@ func (h *ssHandler) Handle(ctx context.Context, conn net.Conn, opts ...handler.H
conn = xnet.NewReadWriteConn(br, conn, conn)
switch proto {
case sniffing.ProtoHTTP:
ro.Time = time.Time{}
return sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
@@ -242,6 +243,7 @@ func (h *ssHandler) Handle(ctx context.Context, conn net.Conn, opts ...handler.H
sniffing.WithLog(log),
)
case sniffing.ProtoTLS:
ro.Time = time.Time{}
return sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
+2
View File
@@ -177,6 +177,7 @@ func (h *unixHandler) Handle(ctx context.Context, conn net.Conn, opts ...handler
conn = xnet.NewReadWriteConn(br, conn, conn)
switch proto {
case sniffing.ProtoHTTP:
ro.Time = time.Time{}
return sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
@@ -186,6 +187,7 @@ func (h *unixHandler) Handle(ctx context.Context, conn net.Conn, opts ...handler
sniffing.WithLog(log),
)
case sniffing.ProtoTLS:
ro.Time = time.Time{}
return sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
+277
View File
@@ -0,0 +1,277 @@
package hop
import (
"context"
"time"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/routing"
"github.com/go-gost/core/selector"
"github.com/go-gost/x/internal/probe"
xlogger "github.com/go-gost/x/logger"
xs "github.com/go-gost/x/selector"
)
// HopEntry wraps a hop.Hop with an optional matcher and an independent
// failure marker for hop-level probe tracking. The entry's marker is
// separate from any node-level markers inside the hop so that probe
// Reset() cannot undo a node-level failCodes Mark().
type HopEntry struct {
hop hop.Hop
matcher routing.Matcher
probe *chain.ProbeConfig
marker selector.Marker
}
// NewHopEntry creates a HopEntry. matcher may be nil (unconditional
// catch-all). probe may be nil (no health check for this entry).
func NewHopEntry(h hop.Hop, m routing.Matcher, probe *chain.ProbeConfig) *HopEntry {
return &HopEntry{
hop: h,
matcher: m,
probe: probe,
marker: selector.NewFailMarker(),
}
}
func (e *HopEntry) Select(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
return e.hop.Select(ctx, opts...)
}
// Marker implements selector.Markable.
func (e *HopEntry) Marker() selector.Marker {
return e.marker
}
// HopGroupOption configures a hopGroup.
type HopGroupOption func(*hopGroupOptions)
type hopGroupOptions struct {
entries []*HopEntry
selector selector.Selector[hop.Hop]
logger logger.Logger
}
// WithEntriesOption sets the hop entries for the group.
func WithEntriesOption(entries ...*HopEntry) HopGroupOption {
return func(o *hopGroupOptions) {
o.entries = entries
}
}
// WithGroupSelectorOption sets the selector for the group.
func WithGroupSelectorOption(s selector.Selector[hop.Hop]) HopGroupOption {
return func(o *hopGroupOptions) {
o.selector = s
}
}
// WithGroupLoggerOption sets the logger for the group.
func WithGroupLoggerOption(log logger.Logger) HopGroupOption {
return func(o *hopGroupOptions) {
o.logger = log
}
}
type hopGroup struct {
entries []*HopEntry
selector selector.Selector[hop.Hop]
logger logger.Logger
cancelFunc context.CancelFunc
}
// NewHopGroup creates a new hopGroup and starts probe goroutines for
// entries that have probe configurations.
func NewHopGroup(opts ...HopGroupOption) *hopGroup {
var options hopGroupOptions
for _, opt := range opts {
if opt != nil {
opt(&options)
}
}
ctx, cancel := context.WithCancel(context.Background())
g := &hopGroup{
entries: options.entries,
selector: options.selector,
logger: options.logger,
cancelFunc: cancel,
}
if g.logger == nil {
g.logger = xlogger.Nop()
}
if g.selector == nil {
g.selector = xs.NewSelector(
xs.RoundRobinStrategy[hop.Hop](),
xs.FailFilter[hop.Hop](xs.DefaultMaxFails, xs.DefaultFailTimeout),
xs.BackupFilter[hop.Hop](),
)
}
for _, e := range g.entries {
if e.probe != nil {
go g.runEntryProbe(ctx, e)
}
}
return g
}
// Select implements hop.Hop. It filters entries by matcher, applies the
// group selector pipeline (FailFilter + strategy) to pick one hop entry,
// and delegates to the chosen entry's underlying hop.
func (g *hopGroup) Select(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
if g == nil || len(g.entries) == 0 {
return nil
}
var options hop.SelectOptions
for _, opt := range opts {
opt(&options)
}
// Stage 1: matcher filter — build eligible pool.
req := routing.Request{
Network: options.Network,
Host: options.Host,
Protocol: options.Protocol,
Method: options.Method,
Path: options.Path,
Query: options.Query,
Header: options.Header,
Body: options.Body,
ClientIP: options.ClientIP,
}
var eligible []hop.Hop
for _, e := range g.entries {
if e.matcher != nil {
if !e.matcher.Match(&req) {
continue
}
g.logger.Debugf("hop entry matched request %s %s", req.Protocol, req.Host)
}
eligible = append(eligible, e)
}
if len(eligible) == 0 {
return nil
}
if len(eligible) == 1 {
return eligible[0].Select(ctx, opts...)
}
// Stage 2: group selector — FailFilter + strategy.
if g.selector != nil {
if se := g.selector.Select(ctx, eligible...); se != nil {
return se.Select(ctx, opts...)
}
}
return eligible[0].Select(ctx, opts...)
}
// Nodes implements hop.NodeList, aggregating all nodes from every entry.
func (g *hopGroup) Nodes() []*chain.Node {
if g == nil {
return nil
}
var nodes []*chain.Node
for _, e := range g.entries {
if nl, ok := e.hop.(hop.NodeList); ok {
nodes = append(nodes, nl.Nodes()...)
}
}
return nodes
}
// Close stops all probe goroutines. It does NOT close the underlying hops:
// they come from the HopRegistry and are shared — the registry closes them
// on unregister.
func (g *hopGroup) Close() error {
g.cancelFunc()
return nil
}
// runEntryProbe periodically probes a node sampled from the hop and marks
// the hop entry (not the node) on failure. This keeps the entry's marker
// independent of node-level markers — probe Reset() cannot undo a
// failCodes Mark().
func (g *hopGroup) runEntryProbe(ctx context.Context, e *HopEntry) {
cfg := e.probe
interval := cfg.Interval
if interval <= 0 {
interval = 30 * time.Second
}
g.probeEntry(e, cfg) // first probe immediately
ticker := time.NewTicker(interval)
defer ticker.Stop()
for {
select {
case <-ticker.C:
g.probeEntry(e, cfg)
case <-ctx.Done():
return
}
}
}
func (g *hopGroup) probeEntry(e *HopEntry, cfg *chain.ProbeConfig) {
// cmd probe runs a shell command — no node selection needed.
if cfg.Type == chain.ProbeTypeCmd {
probe.RunCmdProbe(cfg, e.marker, g.logger)
return
}
// Select a node from the hop — round-robin sampling over time.
node := e.hop.Select(context.Background())
if node == nil {
e.marker.Mark()
g.logger.Debug("hop entry probe: no node available")
return
}
// Probe through the node's transport (same pattern as
// x/chain/node.go probeNode).
timeout := cfg.Timeout
if timeout <= 0 {
timeout = 10 * time.Second
}
probeCtx, cancel := context.WithTimeout(context.Background(), timeout)
defer cancel()
tr := node.Options().Transport
conn, err := tr.Dial(probeCtx, node.Addr)
if err != nil {
e.marker.Mark()
g.logger.Debugf("hop entry probe dial %s failed: %v", node.Addr, err)
return
}
if hc, err2 := tr.Handshake(probeCtx, conn); err2 == nil {
conn = hc
} else {
conn.Close()
e.marker.Mark()
g.logger.Debugf("hop entry probe handshake %s failed: %v", node.Addr, err2)
return
}
if cfg.Type == chain.ProbeTypeHTTP {
err = probe.NewHTTPProber(cfg).Probe(conn)
conn.Close()
if err != nil {
e.marker.Mark()
g.logger.Debugf("hop entry probe http %s failed: %v", node.Addr, err)
return
}
} else {
conn.Close()
}
e.marker.Reset()
}
+115 -96
View File
@@ -9,9 +9,7 @@ import (
"encoding/json"
"errors"
"io"
"net"
"sort"
"strings"
"sync"
"time"
@@ -22,6 +20,7 @@ import (
"github.com/go-gost/core/routing"
"github.com/go-gost/core/selector"
"github.com/go-gost/x/config"
xs "github.com/go-gost/x/selector"
mdutil "github.com/go-gost/x/metadata/util"
node_parser "github.com/go-gost/x/config/parsing/node"
"github.com/go-gost/x/internal/loader"
@@ -33,6 +32,12 @@ type options struct {
nodes []*chain.Node
bypass bypass.Bypass
selector selector.Selector[*chain.Node]
// failMaxFails / failTimeout mirror the FailFilter config used by the
// selector. Node selection consults them via selector.IsFailed to skip
// nodes currently marked failed (FailFilter itself no-ops on a single
// node, so the marker is checked here instead).
failMaxFails int
failTimeout time.Duration
fileLoader loader.Loader
redisLoader loader.Loader
httpLoader loader.Loader
@@ -71,6 +76,16 @@ func SelectorOption(s selector.Selector[*chain.Node]) Option {
}
}
// FailFilterSettingsOption sets the maxFails/failTimeout used by the hop's
// FailFilter. The pre-selection dead-node filter consults these to decide
// whether a node is currently marked failed.
func FailFilterSettingsOption(maxFails int, timeout time.Duration) Option {
return func(o *options) {
o.failMaxFails = maxFails
o.failTimeout = timeout
}
}
// ReloadPeriodOption sets the interval for periodic reloading of node lists.
func ReloadPeriodOption(period time.Duration) Option {
return func(opts *options) {
@@ -154,13 +169,12 @@ func (p *chainHop) Nodes() []*chain.Node {
// Selection pipeline:
//
// 1. Hop-level bypass — entire hop skipped if bypass matches (addr/host)
// 2. Per-node filter (pool) — each candidate must pass at least one gate:
// a. routing.Matcher — boolean expression (Host/Protocol/Method/Path/
// Query/Header/Body). Match → non-zero Priority.
// b. isEligible fallback — when no Matcher set, checks node Options.Filter:
// checkHost(Host), checkProtocol(Protocol),
// checkPath(Path prefix).
// Nodes that fail all gates or hit a node-level bypass are excluded.
// 2. Per-node matcher (pool) — each candidate must pass the gate:
// routing.Matcher — boolean expression (Host/Protocol/Method/
// Path/Query/Header/Body). Match → non-zero
// Priority. Nodes without a matcher are
// unconditional candidates.
// Nodes that fail the gate or hit a node-level bypass are excluded.
// 3. Priority short-circuit — if the top node has strictly higher Priority
// (>0) than the rest and no backup flag is present,
// it wins directly (selector skipped).
@@ -192,40 +206,26 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
maxPriCount int
maxPriNode *chain.Node
hasBackup bool
sawFailed bool
)
for _, node := range p.Nodes() {
if node == nil {
continue
}
// node level bypass
if node.Options().Bypass != nil &&
node.Options().Bypass.Contains(ctx, options.Network, options.Addr, bypass.WithHostOption(options.Host)) {
// A node currently marked failed (http.failCodes match, probe failure)
// is never selected, regardless of its priority or matcher. Checked
// before the eligibility gates so failed nodes skip matcher evaluation
// (body matching reads a large request prefix) and never reach the
// priority short-circuit.
if xs.IsFailed(node, p.options.failMaxFails, p.options.failTimeout) {
sawFailed = true
continue
}
if !p.nodeMatches(ctx, node, &options) {
continue
}
if matcher := node.Options().Matcher; matcher != nil {
// Gate 2a: routing.Matcher — boolean expression on HTTP-level fields.
// A match assigns a non-zero Priority (from rule specificity).
req := routing.Request{
ClientIP: options.ClientIP,
Network: options.Network,
Host: options.Host,
Protocol: options.Protocol,
Method: options.Method,
Path: options.Path,
Query: options.Query,
Header: options.Header,
Body: options.Body,
}
if !matcher.Match(&req) {
continue
}
log.Debugf("node %s match request %s %s, priority %d", node.Name, req.Protocol, req.Host, node.Options().Priority)
} else {
// Gate 2b: fallback eligibility — simple host/protocol/path filters.
if !p.isEligible(node, &options) {
continue
}
log.Debugf("node %s match request %s %s, priority %d", node.Name, options.Protocol, options.Host, node.Options().Priority)
}
pri := node.Options().Priority
@@ -244,6 +244,22 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
nodes = append(nodes, node)
}
if len(nodes) == 0 {
if sawFailed {
// All candidates are marked failed. Best-effort last resort: return
// the first candidate that matches this request, so a transient
// failure (e.g. a single 429) doesn't take the hop fully down — a
// successful dial in the caller resets the marker, so the node
// recovers as soon as it works again. Requests matching no node
// still yield nil (no route).
for _, node := range p.Nodes() {
if node != nil &&
xs.IsFailed(node, p.options.failMaxFails, p.options.failTimeout) &&
p.nodeMatches(ctx, node, &options) {
log.Debugf("last resort: node %s selected (all candidates failed)", node.Name)
return node
}
}
}
return nil
}
if len(nodes) == 1 {
@@ -252,9 +268,10 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
// Stage 3: priority short-circuit.
// maxPriCount == 1 means the top-priority node is strictly higher than
// all others (no tie). We already tracked maxPriority, hasBackup, and
// maxPriCount during Stage 2 — no separate scan needed.
if maxPriority > 0 && !hasBackup && maxPriCount == 1 {
// all others (no tie). Only fire when the top node itself is a primary —
// if it's a backup we must fall through to Stage 4 so the selector can
// prefer any lower-priority primary.
if maxPriority > 0 && maxPriCount == 1 && !isBackupNode(maxPriNode) {
p.logger.Debugf("priority shortcut: node %s selected", maxPriNode.Name)
return maxPriNode
}
@@ -273,8 +290,18 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
// Mixed priorities. Sort descending, then feed each priority tier to
// the selector, falling through when a tier yields nothing.
if s == nil {
return maxPriNode
// Scan once: does the candidate pool contain at least one primary?
// When it does, skip backup-only tiers entirely — a lower-priority
// primary always beats a higher-priority backup.
var hasAnyPrimary bool
if hasBackup {
for _, n := range nodes {
if !mdutil.GetBool(n.Options().Metadata, "backup") {
hasAnyPrimary = true
break
}
}
}
sort.Slice(nodes, func(i, j int) bool {
@@ -283,8 +310,17 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
start := 0
for i := 1; i <= len(nodes); i++ {
if i == len(nodes) || nodes[i].Options().Priority != nodes[start].Options().Priority {
if v := s.Select(ctx, nodes[start:i]...); v != nil {
return v
// Skip backup-only tier when primaries exist in the pool.
if hasAnyPrimary && isAllBackup(nodes[start:i]) {
start = i
continue
}
if s != nil {
if v := s.Select(ctx, nodes[start:i]...); v != nil {
return v
}
} else {
return nodes[start]
}
start = i
}
@@ -292,68 +328,51 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
return nil
}
func (p *chainHop) isEligible(node *chain.Node, opts *hop.SelectOptions) bool {
// isBackupNode reports whether a node is marked as backup in its metadata.
func isBackupNode(node *chain.Node) bool {
return mdutil.GetBool(node.Options().Metadata, "backup")
}
// isAllBackup reports whether all nodes in the slice are backup nodes.
func isAllBackup(nodes []*chain.Node) bool {
for _, n := range nodes {
if !mdutil.GetBool(n.Options().Metadata, "backup") {
return false
}
}
return len(nodes) > 0
}
// nodeMatches reports whether node passes the per-node bypass and matcher
// gates for the given selection options. Shared by the candidate-pool scan
// and the all-failed last-resort fallback. A node with no matcher is an
// unconditional candidate (legacy filter config is normalized to a matcher
// at parse time).
func (p *chainHop) nodeMatches(ctx context.Context, node *chain.Node, opts *hop.SelectOptions) bool {
if node == nil {
return false
}
if node.Options().Filter == nil {
return true
}
if !p.checkHost(opts.Host, node) || !p.checkProtocol(opts.Protocol, node) || !p.checkPath(opts.Path, node) {
if node.Options().Bypass != nil &&
node.Options().Bypass.Contains(ctx, opts.Network, opts.Addr, bypass.WithHostOption(opts.Host)) {
return false
}
if matcher := node.Options().Matcher; matcher != nil {
req := routing.Request{
ClientIP: opts.ClientIP,
Network: opts.Network,
Host: opts.Host,
Protocol: opts.Protocol,
Method: opts.Method,
Path: opts.Path,
Query: opts.Query,
Header: opts.Header,
Body: opts.Body,
}
return matcher.Match(&req)
}
return true
}
func (p *chainHop) checkHost(host string, node *chain.Node) bool {
var vhost string
if filter := node.Options().Filter; filter != nil {
vhost = filter.Host
}
if vhost == "" { // backup node
return true
}
if host == "" {
return false
}
if v, _, _ := net.SplitHostPort(host); v != "" {
host = v
}
if vhost == host || vhost[0] == '.' && strings.HasSuffix(host, vhost[1:]) {
return true
}
return false
}
func (p *chainHop) checkProtocol(protocol string, node *chain.Node) bool {
var prot string
if filter := node.Options().Filter; filter != nil {
prot = filter.Protocol
}
if prot == "" {
return true
}
return prot == protocol
}
func (p *chainHop) checkPath(path string, node *chain.Node) bool {
var pathFilter string
if filter := node.Options().Filter; filter != nil {
pathFilter = filter.Path
}
if pathFilter == "" {
return true
}
return strings.HasPrefix(path, pathFilter)
}
func (p *chainHop) periodReload(ctx context.Context) error {
if err := p.reload(ctx); err != nil {
p.logger.Warnf("reload: %v", err)
+201 -275
View File
@@ -17,10 +17,23 @@ import (
"github.com/go-gost/core/logger"
"github.com/go-gost/core/metadata"
"github.com/go-gost/core/routing"
"github.com/go-gost/x/registry"
xmd "github.com/go-gost/x/metadata"
xlogger "github.com/go-gost/x/logger"
xselector "github.com/go-gost/x/selector"
xrouting "github.com/go-gost/x/routing"
"github.com/go-gost/x/registry"
)
// mustMatcher parses a matcher DSL rule, failing the test on error.
func mustMatcher(t *testing.T, rule string) routing.Matcher {
t.Helper()
m, err := xrouting.NewMatcher(rule)
if err != nil {
t.Fatalf("NewMatcher(%q): %v", rule, err)
}
return m
}
// --- Mock types ---
type testBypass struct {
@@ -28,8 +41,8 @@ type testBypass struct {
contains bool
}
func (b *testBypass) IsWhitelist() bool { return b.whitelist }
func (b *testBypass) Init(md metadata.Metadata) error { return nil }
func (b *testBypass) IsWhitelist() bool { return b.whitelist }
func (b *testBypass) Init(md metadata.Metadata) error { return nil }
func (b *testBypass) Contains(ctx context.Context, network, addr string, opts ...bypass.Option) bool {
return b.contains
}
@@ -113,13 +126,13 @@ func (d *stubDialer) Dial(ctx context.Context, addr string, opts ...dialer.DialO
type stubConn struct{}
func (c *stubConn) Read(b []byte) (n int, err error) { return 0, io.EOF }
func (c *stubConn) Write(b []byte) (n int, err error) { return len(b), nil }
func (c *stubConn) Close() error { return nil }
func (c *stubConn) LocalAddr() net.Addr { return &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0} }
func (c *stubConn) RemoteAddr() net.Addr { return &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0} }
func (c *stubConn) SetDeadline(t time.Time) error { return nil }
func (c *stubConn) SetReadDeadline(t time.Time) error { return nil }
func (c *stubConn) SetWriteDeadline(t time.Time) error { return nil }
func (c *stubConn) Write(b []byte) (n int, err error) { return len(b), nil }
func (c *stubConn) Close() error { return nil }
func (c *stubConn) LocalAddr() net.Addr { return &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0} }
func (c *stubConn) RemoteAddr() net.Addr { return &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0} }
func (c *stubConn) SetDeadline(t time.Time) error { return nil }
func (c *stubConn) SetReadDeadline(t time.Time) error { return nil }
func (c *stubConn) SetWriteDeadline(t time.Time) error { return nil }
func init() {
logger.SetDefault(xlogger.Nop())
@@ -547,6 +560,110 @@ func TestSelect_EqualPriorityDoesNotShortcut(t *testing.T) {
}
}
// TestSelect_FailCodesHighPriorityNode reproduces the reverse-proxy failCodes
// bug: a node with a stricter (longer) matcher wins by auto-computed priority,
// so the priority shortcut selected it directly and bypassed the FailFilter —
// a node marked failed by http.failCodes could never be excluded, so failover
// never happened. After the fix, marking the high-priority node makes Select
// fall through to the healthy lower-priority node.
func TestSelect_FailCodesHighPriorityNode(t *testing.T) {
// highPriority: stricter matcher -> longer rule -> higher auto priority.
highPriority := chain.NewNode("high", "127.0.0.1:8080",
chain.MatcherNodeOption(&testMatcher{match: true}),
chain.PriorityNodeOption(100), // simulates len(longer-rule)
)
// healthy: less specific matcher -> lower priority.
healthy := chain.NewNode("healthy", "127.0.0.1:9090",
chain.MatcherNodeOption(&testMatcher{match: true}),
chain.PriorityNodeOption(10),
)
sel := xselector.NewSelector[*chain.Node](
xselector.FIFOStrategy[*chain.Node](),
xselector.FailFilter[*chain.Node](1, 60*time.Second),
)
h := newTestHop(NodeOption(highPriority, healthy), SelectorOption(sel),
FailFilterSettingsOption(1, 60*time.Second))
defer h.Close()
// Before marking: high-priority node wins via the shortcut.
node := h.Select(context.Background())
if node == nil {
t.Fatal("expected node, got nil")
}
if node.Name != "high" {
t.Fatalf("expected 'high' (higher priority) before marking, got %q", node.Name)
}
// Simulate http.failCodes marking the high-priority node failed (429/5xx).
highPriority.Marker().Mark()
// After marking: Select must fall through to the healthy node.
node = h.Select(context.Background())
if node == nil {
t.Fatal("expected a node after marking, got nil")
}
if node.Name != "healthy" {
t.Fatalf("expected 'healthy' (marked node excluded by FailFilter), got %q", node.Name)
}
}
// TestSelect_SingleNodeFailedLastResort: with all candidates failed, Select
// falls back to the first matching node (best effort) instead of returning
// nil — a transient failure (e.g. one 429) must not take a lone-node hop
// fully down. A successful dial in the caller resets the marker.
func TestSelect_SingleNodeFailedLastResort(t *testing.T) {
only := chain.NewNode("only", "127.0.0.1:8080")
h := newTestHop(NodeOption(only))
defer h.Close()
only.Marker().Mark()
node := h.Select(context.Background())
if node == nil || node.Name != "only" {
t.Fatalf("expected last-resort node 'only', got %v", node)
}
}
// TestSelect_AllFailedLastResort: when every candidate is marked failed, the
// first matching node is still returned as a best-effort fallback.
func TestSelect_AllFailedLastResort(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
n1.Marker().Mark()
n2.Marker().Mark()
node := h.Select(context.Background())
if node == nil {
t.Fatal("expected a last-resort node, got nil")
}
if node.Name != "n1" && node.Name != "n2" {
t.Fatalf("expected n1 or n2, got %q", node.Name)
}
}
// TestSelect_AllFailedNoMatchReturnsNil: the last resort must still respect
// the matcher gates — a request matching no node yields nil even when nodes
// are marked failed (no route, not a fallback target).
func TestSelect_AllFailedNoMatchReturnsNil(t *testing.T) {
only := chain.NewNode("only", "127.0.0.1:8080",
chain.MatcherNodeOption(&testMatcher{match: false}),
)
h := newTestHop(NodeOption(only))
defer h.Close()
only.Marker().Mark()
node := h.Select(context.Background())
if node != nil {
t.Fatalf("expected nil (no node matches), got %q", node.Name)
}
}
func TestSelect_NilNodesSkipped(t *testing.T) {
h := newTestHop()
defer h.Close()
@@ -562,259 +679,12 @@ func TestSelect_NilNodesSkipped(t *testing.T) {
}
// =============================================================================
// isEligible tests
// Select with matcher integration tests
// =============================================================================
func TestIsEligible_NilNode(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
if ch.isEligible(nil, &hop.SelectOptions{}) {
t.Error("expected false for nil node")
}
}
func TestIsEligible_NoFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.isEligible(n, &hop.SelectOptions{}) {
t.Error("expected true when no filter")
}
}
func TestIsEligible_HostMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if !ch.isEligible(n, &hop.SelectOptions{Host: "example.com"}) {
t.Error("expected true for matching host")
}
}
func TestIsEligible_HostMismatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if ch.isEligible(n, &hop.SelectOptions{Host: "other.com"}) {
t.Error("expected false for mismatching host")
}
}
func TestIsEligible_ProtocolMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
if !ch.isEligible(n, &hop.SelectOptions{Protocol: "http"}) {
t.Error("expected true for matching protocol")
}
}
func TestIsEligible_ProtocolMismatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
if ch.isEligible(n, &hop.SelectOptions{Protocol: "socks5"}) {
t.Error("expected false for mismatching protocol")
}
}
func TestIsEligible_PathMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if !ch.isEligible(n, &hop.SelectOptions{Path: "/api/v1/users"}) {
t.Error("expected true for matching path prefix")
}
}
func TestIsEligible_PathMismatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if ch.isEligible(n, &hop.SelectOptions{Path: "/other/route"}) {
t.Error("expected false for mismatching path")
}
}
func TestIsEligible_AllFiltersMustPass(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{
Host: "example.com",
Protocol: "http",
}
if ch.isEligible(n, &hop.SelectOptions{Host: "example.com", Protocol: "socks5"}) {
t.Error("expected false: host matches but protocol doesn't")
}
}
// =============================================================================
// checkHost tests
// =============================================================================
func TestCheckHost_EmptyFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.checkHost("anything.com", n) {
t.Error("expected true when no host filter")
}
}
func TestCheckHost_ExactMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if !ch.checkHost("example.com", n) {
t.Error("expected true for exact host match")
}
}
func TestCheckHost_WildcardMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: ".example.com"}
if !ch.checkHost("sub.example.com", n) {
t.Error("expected true for wildcard host match")
}
}
func TestCheckHost_WildcardNoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: ".example.com"}
if ch.checkHost("other.net", n) {
t.Error("expected false for non-matching wildcard")
}
}
func TestCheckHost_EmptyInputHost(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if ch.checkHost("", n) {
t.Error("expected false when input host is empty and filter is set")
}
}
func TestCheckHost_StripPort(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if !ch.checkHost("example.com:443", n) {
t.Error("expected true after stripping port")
}
}
func TestCheckHost_IPWithPort(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "192.168.1.1"}
if !ch.checkHost("192.168.1.1:3128", n) {
t.Error("expected true for IP with port")
}
}
func TestCheckHost_IPv6WithPort(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "::1"}
if !ch.checkHost("[::1]:8080", n) {
t.Error("expected true for bracketed IPv6 with port")
}
}
func TestCheckHost_NoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if ch.checkHost("other.com", n) {
t.Error("expected false for non-matching host")
}
}
// =============================================================================
// checkProtocol tests
// =============================================================================
func TestCheckProtocol_EmptyFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.checkProtocol("http", n) {
t.Error("expected true when no protocol filter")
}
}
func TestCheckProtocol_Match(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
if !ch.checkProtocol("http", n) {
t.Error("expected true for matching protocol")
}
}
func TestCheckProtocol_NoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "socks5"}
if ch.checkProtocol("http", n) {
t.Error("expected false for non-matching protocol")
}
}
// =============================================================================
// checkPath tests
// =============================================================================
func TestCheckPath_EmptyFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.checkPath("/any/path", n) {
t.Error("expected true when no path filter")
}
}
func TestCheckPath_PrefixMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if !ch.checkPath("/api/v1/users", n) {
t.Error("expected true for prefix match")
}
}
func TestCheckPath_ExactMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if !ch.checkPath("/api", n) {
t.Error("expected true for exact match (prefix of self)")
}
}
func TestCheckPath_NoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if ch.checkPath("/other", n) {
t.Error("expected false for non-prefix match")
}
}
func TestCheckPath_ShorterInput(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if ch.checkPath("/ap", n) {
t.Error("expected false when input is shorter than filter prefix")
}
}
// =============================================================================
// Select with filter integration tests
// =============================================================================
func TestSelect_FilterHost_Match(t *testing.T) {
func TestSelect_MatcherHost_Match(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`)")
h := newTestHop(NodeOption(n1))
defer h.Close()
@@ -824,9 +694,9 @@ func TestSelect_FilterHost_Match(t *testing.T) {
}
}
func TestSelect_FilterHost_NoMatch(t *testing.T) {
func TestSelect_MatcherHost_NoMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
@@ -840,9 +710,9 @@ func TestSelect_FilterHost_NoMatch(t *testing.T) {
}
}
func TestSelect_FilterProtocol_Match(t *testing.T) {
func TestSelect_MatcherProtocol_Match(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
n1.Options().Matcher = mustMatcher(t, "Proto(`http`)")
h := newTestHop(NodeOption(n1))
defer h.Close()
@@ -852,9 +722,9 @@ func TestSelect_FilterProtocol_Match(t *testing.T) {
}
}
func TestSelect_FilterProtocol_NoMatch(t *testing.T) {
func TestSelect_MatcherProtocol_NoMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
n1.Options().Matcher = mustMatcher(t, "Proto(`http`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
@@ -868,6 +738,39 @@ func TestSelect_FilterProtocol_NoMatch(t *testing.T) {
}
}
func TestSelect_MatcherWildcardHost_MatchesApexAndSubdomain(t *testing.T) {
// filter `.example.com` (legacy wildcard) matches apex AND subdomains;
// the converter emits `Host(example.com) || Host(.example.com)`.
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`) || Host(`.example.com`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
for _, host := range []string{"example.com", "sub.example.com", "a.b.example.com"} {
node := h.Select(context.Background(), hop.HostSelectOption(host))
if node == nil || node.Name != "n1" {
t.Errorf("expected n1 for host %q, got %v", host, node)
}
}
}
func TestSelect_MatcherWildcardHost_NoSubdomainMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`) || Host(`.example.com`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
node := h.Select(context.Background(), hop.HostSelectOption("other.net"))
if node == nil {
t.Fatal("expected node, got nil")
}
if node.Name != "n2" {
t.Errorf("expected 'n2' (n1 host mismatch), got %q", node.Name)
}
}
// =============================================================================
// Select with select options
// =============================================================================
@@ -1314,14 +1217,13 @@ func TestSelect_OnlyNegativePriorities(t *testing.T) {
}
// =============================================================================
// Select with backup node (empty host filter means backup)
// Select with catch-all node (no matcher = unconditional candidate)
// =============================================================================
func TestSelect_BackupNode_HostFilterEmpty(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: ""}
func TestSelect_CatchAllNode_TakesOverOnNoMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080") // no matcher: catch-all
n2 := chain.NewNode("n2", "127.0.0.1:9090")
n2.Options().Filter = &chain.NodeFilterSettings{Host: "specific.com"}
n2.Options().Matcher = mustMatcher(t, "Host(`specific.com`)")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
@@ -1330,15 +1232,14 @@ func TestSelect_BackupNode_HostFilterEmpty(t *testing.T) {
t.Fatal("expected node, got nil")
}
if node.Name != "n1" {
t.Errorf("expected 'n1' (backup), got %q", node.Name)
t.Errorf("expected 'n1' (catch-all), got %q", node.Name)
}
}
func TestSelect_SpecificHostNodeWins(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: ""}
n1 := chain.NewNode("n1", "127.0.0.1:8080") // no matcher: catch-all
n2 := chain.NewNode("n2", "127.0.0.1:9090")
n2.Options().Filter = &chain.NodeFilterSettings{Host: "specific.com"}
n2.Options().Matcher = mustMatcher(t, "Host(`specific.com`)")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
@@ -1351,6 +1252,31 @@ func TestSelect_SpecificHostNodeWins(t *testing.T) {
}
}
func TestSelect_BackupPriority_LowerPriorityPrimaryWins(t *testing.T) {
// A lower-priority primary must beat a higher-priority backup.
// Node A: priority=100, backup=true (high-priority backup)
// Node B: priority=10, backup=false (low-priority primary)
// Expected: B wins — primary always beats backup regardless of priority.
n1 := chain.NewNode("backup-high",
"127.0.0.1:8080",
chain.PriorityNodeOption(100),
chain.MetadataNodeOption(xmd.NewMetadata(map[string]any{"backup": true})),
)
n2 := chain.NewNode("primary-low",
"127.0.0.1:9090",
chain.PriorityNodeOption(10),
)
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
node := h.Select(context.Background())
if node == nil {
t.Fatal("expected node, got nil")
}
if node.Name != "primary-low" {
t.Errorf("expected 'primary-low' (lower-priority primary beats higher-priority backup), got %q", node.Name)
}
}
// =============================================================================
// Interface satisfaction
// =============================================================================
+32
View File
@@ -15,6 +15,38 @@ func ApplyKeepalive(conn net.Conn, cfg net.KeepAliveConfig) {
}
}
// ApplyNoDelay disables Nagle's algorithm on conn if it is a *net.TCPConn.
//
// gost tunnels carry small, latency-sensitive datagrams (e.g. WireGuard UDP
// over relay+ws/wss). Without this, Nagle coalesces each frame until a peer
// ACK returns, adding up to one RTT per datagram — the root cause of the
// latency regression reported in go-gost/gost#858. Match wstunnel, which sets
// TCP_NODELAY on its tunnel socket by default.
func ApplyNoDelay(conn net.Conn) {
if tc, ok := conn.(*net.TCPConn); ok {
tc.SetNoDelay(true)
}
}
// WrapNoDelayListener wraps ln so that TCP_NODELAY is applied to every accepted
// *net.TCPConn before it enters the middleware wrapper chain.
func WrapNoDelayListener(ln net.Listener) net.Listener {
return &noDelayListener{Listener: ln}
}
type noDelayListener struct {
net.Listener
}
func (l *noDelayListener) Accept() (net.Conn, error) {
conn, err := l.Listener.Accept()
if err != nil {
return nil, err
}
ApplyNoDelay(conn)
return conn, nil
}
type keepaliveListener struct {
net.Listener
cfg net.KeepAliveConfig
+21
View File
@@ -6,6 +6,10 @@ import (
"os/exec"
"runtime"
"time"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/selector"
)
// CmdProber runs a shell command to determine node health.
@@ -32,6 +36,23 @@ func (p *CmdProber) Probe() error {
return nil
}
// RunCmdProbe runs a cmd probe and updates the entry marker on success/failure.
// It is shared by chainGroup and hopGroup probe loops to avoid duplicated code.
func RunCmdProbe(cfg *chain.ProbeConfig, marker selector.Marker, log logger.Logger) {
timeout := cfg.Timeout
if timeout <= 0 {
timeout = 10 * time.Second
}
if err := (&CmdProber{Command: cfg.Command, Timeout: timeout}).Probe(); err != nil {
marker.Mark()
if log != nil {
log.Debugf("cmd probe failed: %v", err)
}
} else {
marker.Reset()
}
}
// shellCmd returns the platform shell and its flag for passing a command string.
func shellCmd() (shell, flag string) {
if runtime.GOOS == "windows" {
+50
View File
@@ -0,0 +1,50 @@
package forwarder
import (
"context"
"net"
"sync"
"github.com/go-gost/x/internal/util/sniffing"
)
// HandleFunc handles a connection whose protocol has been detected as proto,
// mirroring sniffing.HandleFunc but without the network parameter. The
// receiver s carries the per-connection Sniffer state.
type HandleFunc func(s *Sniffer, ctx context.Context, conn net.Conn, opts ...sniffing.HandleOption) error
var protoHandlers sync.Map // map[string]HandleFunc
// Register associates a protocol with its handler. Duplicate registration
// panics, mirroring the house registry pattern.
func Register(proto string, fn HandleFunc) {
if _, dup := protoHandlers.LoadOrStore(proto, fn); dup {
panic("forwarder: duplicate protocol handler: " + proto)
}
}
// Get returns the handler registered for proto.
func Get(proto string) (HandleFunc, bool) {
v, ok := protoHandlers.Load(proto)
if !ok {
return nil, false
}
return v.(HandleFunc), true
}
// Dispatch routes a detected protocol to its registered handler. It returns
// (false, nil) when no handler is registered for proto, signalling the caller
// to fall through to raw forwarding.
func Dispatch(s *Sniffer, ctx context.Context, conn net.Conn, proto string, opts ...sniffing.HandleOption) (handled bool, err error) {
fn, ok := Get(proto)
if !ok {
return false, nil
}
return true, fn(s, ctx, conn, opts...)
}
func init() {
Register(sniffing.ProtoHTTP, (*Sniffer).HandleHTTP)
Register(sniffing.ProtoTLS, (*Sniffer).HandleTLS)
Register(sniffing.ProtoRedis, (*Sniffer).HandleRedis)
}
+86
View File
@@ -0,0 +1,86 @@
package forwarder
import (
"context"
"net"
"testing"
"github.com/go-gost/x/internal/util/sniffing"
)
// TestRegisterGet verifies a handler can be registered and looked up by proto.
func TestRegisterGet(t *testing.T) {
const proto = "testproto-get"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ net.Conn, _ ...sniffing.HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
fn, ok := Get(proto)
if !ok {
t.Fatal("Get: handler not found")
}
if err := fn(&Sniffer{}, context.Background(), nil); err != nil {
t.Fatalf("fn() error = %v", err)
}
if !called {
t.Fatal("registered handler was not called")
}
if _, ok := Get("no-such-proto"); ok {
t.Fatal("Get: unexpected handler for unknown proto")
}
}
// TestRegisterDuplicate verifies registering the same proto twice panics.
func TestRegisterDuplicate(t *testing.T) {
const proto = "testproto-dup"
Register(proto, func(*Sniffer, context.Context, net.Conn, ...sniffing.HandleOption) error {
return nil
})
defer protoHandlers.Delete(proto)
defer func() {
if recover() == nil {
t.Fatal("expected panic on duplicate registration")
}
}()
Register(proto, func(*Sniffer, context.Context, net.Conn, ...sniffing.HandleOption) error {
return nil
})
}
// TestDispatch_Unknown verifies Dispatch falls through (false, nil) for an
// unregistered proto.
func TestDispatch_Unknown(t *testing.T) {
handled, err := Dispatch(&Sniffer{}, context.Background(), nil, "no-such-proto")
if handled {
t.Fatal("Dispatch: unknown proto handled = true, want false")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
}
// TestDispatch_Known verifies Dispatch routes a registered proto to its handler.
func TestDispatch_Known(t *testing.T) {
const proto = "testproto-known"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ net.Conn, _ ...sniffing.HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
handled, err := Dispatch(&Sniffer{}, context.Background(), nil, proto)
if !handled {
t.Fatal("Dispatch: known proto handled = false, want true")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
if !called {
t.Fatal("Dispatch: registered handler was not called")
}
}
+8 -76
View File
@@ -1,7 +1,6 @@
package forwarder
import (
"context"
"crypto"
"crypto/tls"
"crypto/x509"
@@ -11,13 +10,11 @@ import (
"github.com/go-gost/core/bypass"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/recorder"
"github.com/go-gost/x/config"
tls_util "github.com/go-gost/x/internal/util/tls"
"github.com/go-gost/x/internal/util/httpcache"
xrecorder "github.com/go-gost/x/recorder"
"github.com/go-gost/x/internal/util/sniffing"
tls_util "github.com/go-gost/x/internal/util/tls"
)
const (
@@ -33,77 +30,12 @@ const DefaultReadTimeout = 30 * time.Second
// DefaultCertPool is the default in-memory certificate pool used for TLS MITM.
var DefaultCertPool = tls_util.NewMemoryCertPool()
// HandleOptions holds configuration options for sniffing handlers.
type HandleOptions struct {
service string
dial func(ctx context.Context, network, address string) (net.Conn, error)
httpKeepalive bool
readTimeout time.Duration
node *chain.Node
hop hop.Hop
bypass bypass.Bypass
recorderObject *xrecorder.HandlerRecorderObject
log logger.Logger
}
// HandleOptions aliases sniffing.HandleOptions, the unified option set shared
// by the sniffing and forwarder Sniffer implementations.
type HandleOptions = sniffing.HandleOptions
// HandleOption configures HandleOptions for sniffing handlers.
type HandleOption func(opts *HandleOptions)
// WithService sets the service name for bypass and selection lookups.
func WithService(service string) HandleOption {
return func(opts *HandleOptions) {
opts.service = service
}
}
// WithDial sets the dial function used to establish upstream connections.
func WithDial(dial func(ctx context.Context, network, address string) (net.Conn, error)) HandleOption {
return func(opts *HandleOptions) {
opts.dial = dial
}
}
// WithHTTPKeepalive enables or disables HTTP keep-alive on the upstream connection.
func WithHTTPKeepalive(keepalive bool) HandleOption {
return func(opts *HandleOptions) {
opts.httpKeepalive = keepalive
}
}
// WithNode sets a pre-resolved chain node to connect to, bypassing hop selection.
func WithNode(node *chain.Node) HandleOption {
return func(opts *HandleOptions) {
opts.node = node
}
}
// WithHop sets the hop used for node selection when routing requests.
func WithHop(h hop.Hop) HandleOption {
return func(opts *HandleOptions) {
opts.hop = h
}
}
// WithBypass sets the bypass rules for filtering requests by host.
func WithBypass(bypass bypass.Bypass) HandleOption {
return func(opts *HandleOptions) {
opts.bypass = bypass
}
}
// WithRecorderObject sets the recorder object for capturing traffic metadata.
func WithRecorderObject(ro *xrecorder.HandlerRecorderObject) HandleOption {
return func(opts *HandleOptions) {
opts.recorderObject = ro
}
}
// WithLog sets the logger for the handler.
func WithLog(log logger.Logger) HandleOption {
return func(opts *HandleOptions) {
opts.log = log
}
}
type HandleOption = sniffing.HandleOption
// Sniffer handles HTTP and TLS traffic sniffing, recording, and MITM TLS
// termination for protocol-aware forwarding. It can intercept HTTP requests,
@@ -171,8 +103,8 @@ func normalizeHost(host, defaultPort string) string {
// effectiveReadTimeout returns the read timeout from options, falling back to
// the Sniffer's ReadTimeout, then to DefaultReadTimeout.
func (h *Sniffer) effectiveReadTimeout(ho *HandleOptions) time.Duration {
if ho.readTimeout > 0 {
return ho.readTimeout
if ho.ReadTimeout > 0 {
return ho.ReadTimeout
}
if h.ReadTimeout > 0 {
return h.ReadTimeout
+2 -2
View File
@@ -32,8 +32,8 @@ func (h *Sniffer) serveH2(ctx context.Context, conn net.Conn, ho *HandleOptions)
return errors.New("h2: invalid client preface")
}
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
ro.Time = time.Time{}
+40 -31
View File
@@ -51,7 +51,7 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
for _, opt := range opts {
opt(&ho)
}
ho.readTimeout = h.effectiveReadTimeout(&ho)
ho.ReadTimeout = h.effectiveReadTimeout(&ho)
pStats := xstats.Stats{}
conn = stats_wrapper.WrapConn(conn, &pStats)
@@ -62,13 +62,21 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
return err
}
log := ho.log
log := ho.Log
if log.IsLevelEnabled(logger.TraceLevel) {
dump, _ := httputil.DumpRequest(req, false)
log.Trace(string(dump))
}
ro := ho.recorderObject
ro := ho.RecorderObject
// Copy ro so that all internal recording (cache-hit serveCachedResponse
// and cache-miss httpRoundTrip) happens on a local clone, preventing
// double-record when the caller's defer also records ro.
ro2 := &xrecorder.HandlerRecorderObject{}
*ro2 = *ro
ro = ro2
ho.RecorderObject = ro2
if clientIP := xhttp.GetClientIP(req); clientIP != nil {
clientAddr := &net.TCPAddr{IP: clientIP}
@@ -132,8 +140,8 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
}
upstreamHost = normalizeHost(ro.HTTP.Host, "80")
ho.log = log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log = ho.log
ho.Log = log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log = ho.Log
log.Debugf("connected to node %s(%s)", node.Name, node.Addr)
ro.SrcAddr = cc.LocalAddr().String()
@@ -171,7 +179,7 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
ro.Host = reqHost
log = log.WithFields(map[string]any{"host": reqHost})
ho.log = log
ho.Log = log
}
}
}
@@ -182,6 +190,7 @@ func (h *Sniffer) serveCachedResponse(ctx context.Context, rw io.Writer, req *ht
defer resp.Body.Close()
ro.Time = time.Now()
ro.HTTP.StatusCode = resp.StatusCode
ro.HTTP.Cached = true
ro.HTTP.Response.Header = resp.Header.Clone()
ro.HTTP.Response.ContentLength = resp.ContentLength
log.Debugf("cache hit: %s", httpcache.Key(req.Method, req.Host, req.RequestURI))
@@ -218,19 +227,19 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
StatusCode: http.StatusServiceUnavailable,
}
if ho.bypass != nil &&
ho.bypass.Contains(ctx, "tcp", host,
bypass.WithService(ho.service),
if ho.Bypass != nil &&
ho.Bypass.Contains(ctx, "tcp", host,
bypass.WithService(ho.Service),
bypass.WithPathOption(req.RequestURI)) {
ho.log.Debugf("bypass: %s %s", host, req.RequestURI)
ho.Log.Debugf("bypass: %s %s", host, req.RequestURI)
res.StatusCode = http.StatusForbidden
return nil, res, xbypass.ErrBypass
}
node = &chain.Node{}
if ho.hop != nil {
if ho.Hop != nil {
var clientIP net.IP
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.recorderObject.ClientAddr); clientAddr != nil {
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.RecorderObject.ClientAddr); clientAddr != nil {
clientIP = clientAddr.IP
}
@@ -239,7 +248,7 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
// without consuming the stream that is still forwarded below.
var bodyPrefix []byte
var maxBodySize int
if nl, ok := ho.hop.(hop.NodeList); ok {
if nl, ok := ho.Hop.(hop.NodeList); ok {
for _, n := range nl.Nodes() {
if n == nil {
continue
@@ -262,7 +271,7 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
}
}
node = ho.hop.Select(ctx,
node = ho.Hop.Select(ctx,
hop.ClientIPSelectOption(clientIP),
hop.ProtocolSelectOption(sniffing.ProtoHTTP),
hop.HostSelectOption(host),
@@ -274,11 +283,11 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
)
}
if node == nil {
ho.log.Warnf("node for %s not found", host)
ho.Log.Warnf("node for %s not found", host)
res.StatusCode = http.StatusBadGateway
return nil, res, errors.New("node not available")
}
ho.recorderObject.Node = node.Name
ho.RecorderObject.Node = node.Name
if node.Addr == "" {
node = &chain.Node{
Name: node.Name,
@@ -290,21 +299,21 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
// dial selects a node, establishes a connection, and sends the request upstream.
func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho *HandleOptions) (node *chain.Node, cc net.Conn, err error) {
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
if node = ho.node; node != nil {
if node = ho.Node; node != nil {
cc, err = dial(ctx, "tcp", node.Addr)
return
}
ro := ho.recorderObject
ro := ho.RecorderObject
host := normalizeHost(req.Host, "80")
if host != "" {
ro.Host = host
ho.log = ho.log.WithFields(map[string]any{
ho.Log = ho.Log.WithFields(map[string]any{
"host": host,
})
}
@@ -313,7 +322,7 @@ func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho
if resolveErr != nil {
ro.HTTP.StatusCode = res.StatusCode
if werr := res.Write(conn); werr != nil {
ho.log.Warnf("write error response: %v", werr)
ho.Log.Warnf("write error response: %v", werr)
}
return nil, nil, resolveErr
}
@@ -327,20 +336,20 @@ func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho
}
ro.Host = node.Addr
ho.log = ho.log.WithFields(map[string]any{
ho.Log = ho.Log.WithFields(map[string]any{
"node": node.Name,
"dst": node.Addr,
})
ho.log.Debugf("find node for host %s -> %s(%s)", host, node.Name, node.Addr)
ho.Log.Debugf("find node for host %s -> %s(%s)", host, node.Name, node.Addr)
cc, err = dial(ctx, "tcp", node.Addr)
if err != nil {
if marker := node.Marker(); marker != nil {
marker.Mark()
}
ho.log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
ho.Log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
if werr := res.Write(conn); werr != nil {
ho.log.Warnf("write error response: %v", werr)
ho.Log.Warnf("write error response: %v", werr)
}
return
}
@@ -359,9 +368,9 @@ func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho
func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser, node *chain.Node, req *http.Request, pStats stats.Stats, ho *HandleOptions, staleResp *http.Response) (shouldClose bool, err error) {
shouldClose = true
log := ho.log
log := ho.Log
ro := &xrecorder.HandlerRecorderObject{}
*ro = *ho.recorderObject
*ro = *ho.RecorderObject
ro.Time = time.Now()
log.Infof("%s <-> %s", ro.RemoteAddr, req.Host)
@@ -430,7 +439,7 @@ func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser,
if httpSettings != nil {
if auther := httpSettings.Auther; auther != nil {
username, password, _ := req.BasicAuth()
id, ok := auther.Authenticate(ctx, username, password, auth.WithService(ho.service))
id, ok := auther.Authenticate(ctx, username, password, auth.WithService(ho.Service))
if !ok {
res.StatusCode = http.StatusUnauthorized
ro.HTTP.StatusCode = res.StatusCode
@@ -525,7 +534,7 @@ func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser,
br := bufio.NewReader(cc)
var resp *http.Response
for {
xio.SetReadDeadline(cc, time.Now().Add(ho.readTimeout))
xio.SetReadDeadline(cc, time.Now().Add(ho.ReadTimeout))
resp, err = http.ReadResponse(br, req)
if err != nil {
log.Errorf("read response: %v", err)
@@ -591,7 +600,7 @@ func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser,
resp.ProtoMinor = req.ProtoMinor
}
if !ho.httpKeepalive {
if !ho.HTTPKeepalive {
resp.Header.Set("Connection", "close")
}
@@ -670,7 +679,7 @@ func (h *Sniffer) serveStale(rw io.Writer, staleResp *http.Response, ro *xrecord
if h.Cache == nil || staleResp == nil || !h.Cache.ServeStale() {
return false
}
if !ho.httpKeepalive {
if !ho.HTTPKeepalive {
staleResp.Header.Set("Connection", "close")
}
ro.HTTP.StatusCode = staleResp.StatusCode
+141
View File
@@ -0,0 +1,141 @@
package forwarder
import (
"bytes"
"context"
"fmt"
"io"
"net"
"time"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
xnet "github.com/go-gost/x/internal/net"
"github.com/go-gost/x/internal/util/sniffing"
)
// HandleRedis proxies a sniffed Redis connection through a hop-selected node.
// It parses the first RESP message for metadata recording, selects an upstream
// node via resolveRedisNode (which passes ProtoRedis to the hop's protocol
// selector), replays the captured bytes, and pipes traffic bidirectionally.
func (h *Sniffer) HandleRedis(ctx context.Context, conn net.Conn, opts ...HandleOption) error {
var ho HandleOptions
for _, opt := range opts {
opt(&ho)
}
buf := new(bytes.Buffer)
if err := sniffing.ParseRedisMetadata(io.TeeReader(conn, buf), ho.RecorderObject); err != nil {
return err
}
ro := ho.RecorderObject
node, cc, err := h.dialRedis(ctx, &ho)
if err != nil {
return err
}
defer cc.Close()
ho.Node = node
log := ho.Log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log.Debugf("connected to node %s(%s)", node.Name, node.Addr)
ro.SrcAddr = cc.LocalAddr().String()
ro.DstAddr = cc.RemoteAddr().String()
if _, err := buf.WriteTo(cc); err != nil {
return err
}
log.Infof("%s <-> %s", ro.RemoteAddr, ro.Host)
xnet.Pipe(ctx, conn, cc)
log.WithFields(map[string]any{
"duration": time.Since(ro.Time),
}).Infof("%s >-< %s", ro.RemoteAddr, ro.Host)
return nil
}
// resolveRedisNode selects an upstream node for a Redis connection via hop
// selection with ProtocolSelectOption. Unlike resolveTLSNode, there is no
// host to route by — routing is driven by ProtocolSelectOption(ProtoRedis)
// and ClientIPSelectOption.
func resolveRedisNode(ctx context.Context, ho *HandleOptions) (node *chain.Node, err error) {
node = &chain.Node{}
if ho.Hop != nil {
var clientIP net.IP
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.RecorderObject.ClientAddr); clientAddr != nil {
clientIP = clientAddr.IP
}
node = ho.Hop.Select(ctx,
hop.ClientIPSelectOption(clientIP),
hop.ProtocolSelectOption(sniffing.ProtoRedis),
)
}
if node == nil {
return nil, fmt.Errorf("node not available")
}
ho.RecorderObject.Node = node.Name
return node, nil
}
// dialRedis selects a node via resolveRedisNode and establishes a TCP
// connection to the node's address.
func (h *Sniffer) dialRedis(ctx context.Context, ho *HandleOptions) (node *chain.Node, cc net.Conn, err error) {
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
if node = ho.Node; node != nil {
cc, err = dial(ctx, "tcp", node.Addr)
return
}
node, err = resolveRedisNode(ctx, ho)
if err != nil {
return
}
ro := ho.RecorderObject
addr := node.Addr
network := "tcp"
if opts := node.Options(); opts != nil {
switch opts.Network {
case "unix":
network = opts.Network
default:
if _, _, splitErr := net.SplitHostPort(addr); splitErr != nil {
addr += ":6379"
}
}
} else {
if _, _, splitErr := net.SplitHostPort(addr); splitErr != nil {
addr += ":6379"
}
}
ro.Host = addr
ho.Log = ho.Log.WithFields(map[string]any{
"node": node.Name,
"dst": fmt.Sprintf("%s/%s", addr, network),
})
ho.Log.Debugf("find node for redis -> %s(%s)", node.Name, addr)
cc, err = dial(ctx, network, addr)
if err != nil {
if marker := node.Marker(); marker != nil {
marker.Mark()
}
ho.Log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
return
}
if marker := node.Marker(); marker != nil {
marker.Reset()
}
cc = tlsWrapConn(cc, node.Options().TLS)
return
}
+18 -6
View File
@@ -497,10 +497,22 @@ func (b *rewriteBody) apply(body []byte, opts ...rewriter.RewriteOption) ([]byte
body = rewritten
}
} else if rw.Pattern != nil {
if strings.HasPrefix(rw.Type, "json:") {
if strings.HasPrefix(rw.Type, "json:") || strings.HasPrefix(rw.Type, "json-:") {
del := strings.HasPrefix(rw.Type, "json-:")
path := rw.Type[5:]
if del {
path = rw.Type[6:]
}
if rw.Pattern.MatchString(gjson.GetBytes(body, path).String()) {
replaced, err := sjson.SetBytes(body, path, string(rw.Replacement))
var (
replaced []byte
err error
)
if del {
replaced, err = sjson.DeleteBytes(body, path)
} else {
replaced, err = sjson.SetBytes(body, path, string(rw.Replacement))
}
if err == nil {
body = replaced
}
@@ -514,9 +526,9 @@ func (b *rewriteBody) apply(body []byte, opts ...rewriter.RewriteOption) ([]byte
}
// shouldApply reports whether a rewrite rule's configured type matches the
// actual request Content-Type. A type prefix "json:" matches only
// "application/json" content. An empty type defaults to "text/html", or "*"
// when a Rewriter is set.
// actual request Content-Type. JSON rules ("json:" or "json-:" Type prefixes)
// match only "application/json" content. An empty type defaults to
// "text/html", or "*" when a Rewriter is set.
func shouldApply(configuredType, actualContentType string, hasRewriter bool) bool {
t := configuredType
if t == "" {
@@ -526,7 +538,7 @@ func shouldApply(configuredType, actualContentType string, hasRewriter bool) boo
t = "text/html"
}
}
if strings.HasPrefix(t, "json:") {
if strings.HasPrefix(t, "json:") || strings.HasPrefix(t, "json-:") {
return actualContentType == "application/json"
}
return t == "*" || strings.Contains(t, actualContentType)
+182 -69
View File
@@ -6,6 +6,7 @@ import (
"compress/gzip"
"compress/zlib"
"context"
"encoding/json"
"io"
"net"
"net/http"
@@ -21,9 +22,11 @@ import (
"github.com/go-gost/core/hop"
"github.com/go-gost/core/recorder"
"github.com/go-gost/core/rewriter"
"github.com/klauspost/compress/zstd"
xlogger "github.com/go-gost/x/logger"
xrecorder "github.com/go-gost/x/recorder"
"github.com/klauspost/compress/zstd"
"github.com/go-gost/x/internal/util/sniffing"
)
// =============================================================================
@@ -36,6 +39,16 @@ func (n *noopRecorder) Record(_ context.Context, _ []byte, _ ...recorder.RecordO
return nil
}
// captureRecorder records every payload written through recorder.Recorder.
type captureRecorder struct {
data [][]byte
}
func (r *captureRecorder) Record(_ context.Context, b []byte, _ ...recorder.RecordOption) error {
r.data = append(r.data, append([]byte(nil), b...))
return nil
}
// mockBypass is a configurable bypass.Bypass for tests.
type mockBypass struct {
contains bool
@@ -230,7 +243,6 @@ func TestRewriteReqBody_Gzip(t *testing.T) {
})
}
func TestClampBodySize(t *testing.T) {
tests := []struct {
name string
@@ -297,7 +309,7 @@ func TestEffectiveReadTimeout(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
h := &Sniffer{ReadTimeout: tt.snifferTO}
ho := &HandleOptions{readTimeout: tt.hoTO}
ho := &HandleOptions{ReadTimeout: tt.hoTO}
got := h.effectiveReadTimeout(ho)
if got != tt.want {
t.Errorf("effectiveReadTimeout() = %v, want %v", got, tt.want)
@@ -541,9 +553,10 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
}))
defer upstream.Close()
rec := &captureRecorder{}
h := &Sniffer{
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
Recorder: rec,
}
ro := &xrecorder.HandlerRecorderObject{}
@@ -555,12 +568,12 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithService("test-svc"),
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithService("test-svc"),
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return net.Dial("tcp", upstream.Listener.Addr().String())
}),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -581,17 +594,35 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
t.Errorf("status = %d, want %d", resp.StatusCode, http.StatusOK)
}
clientConn.Close()
if err := <-errCh; err != nil {
t.Logf("HandleHTTP returned: %v", err)
// Drain the body so HandleHTTP's response write completes before the
// client side closes (net.Pipe writes block until read).
body, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatalf("read body: %v", err)
}
if string(body) != "OK from upstream" {
t.Errorf("body = %q, want %q", body, "OK from upstream")
}
// Verify recorder object was populated.
if ro.SrcAddr == "" {
t.Error("SrcAddr should be populated")
clientConn.Close()
if err := <-errCh; err != nil {
t.Errorf("HandleHTTP returned: %v", err)
}
if ro.DstAddr == "" {
t.Error("DstAddr should be populated")
// HandleHTTP records on an internal clone of ro and publishes it via
// h.Recorder, so assert on the captured payload.
if len(rec.data) != 1 {
t.Fatalf("expected 1 recorder event, got %d", len(rec.data))
}
var recorded xrecorder.HandlerRecorderObject
if err := json.Unmarshal(rec.data[0], &recorded); err != nil {
t.Fatalf("unmarshal recorder event: %v", err)
}
if recorded.SrcAddr == "" {
t.Error("recorded srcAddr should be populated")
}
if recorded.DstAddr == "" {
t.Error("recorded dstAddr should be populated")
}
}
@@ -612,9 +643,9 @@ func TestHandleHTTP_HTTP2Detection(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithService("test-svc"),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithService("test-svc"),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -637,16 +668,16 @@ func TestHandleHTTP_HTTP2Detection(t *testing.T) {
func TestWithService(t *testing.T) {
opts := &HandleOptions{}
WithService("mysvc")(opts)
if opts.service != "mysvc" {
t.Errorf("service = %q, want %q", opts.service, "mysvc")
sniffing.WithService("mysvc")(opts)
if opts.Service != "mysvc" {
t.Errorf("service = %q, want %q", opts.Service, "mysvc")
}
}
func TestWithHTTPKeepalive(t *testing.T) {
opts := &HandleOptions{}
WithHTTPKeepalive(true)(opts)
if !opts.httpKeepalive {
sniffing.WithHTTPKeepalive(true)(opts)
if !opts.HTTPKeepalive {
t.Errorf("httpKeepalive = false, want true")
}
}
@@ -654,8 +685,8 @@ func TestWithHTTPKeepalive(t *testing.T) {
func TestWithNode(t *testing.T) {
opts := &HandleOptions{}
node := &chain.Node{Name: "test-node", Addr: "127.0.0.1:8080"}
WithNode(node)(opts)
if opts.node != node {
sniffing.WithNode(node)(opts)
if opts.Node != node {
t.Errorf("node not set")
}
}
@@ -663,8 +694,8 @@ func TestWithNode(t *testing.T) {
func TestWithHop(t *testing.T) {
opts := &HandleOptions{}
mh := &mockHop{}
WithHop(mh)(opts)
if opts.hop != mh {
sniffing.WithHop(mh)(opts)
if opts.Hop != mh {
t.Errorf("hop not set")
}
}
@@ -672,8 +703,8 @@ func TestWithHop(t *testing.T) {
func TestWithBypass(t *testing.T) {
opts := &HandleOptions{}
bp := &mockBypass{contains: true}
WithBypass(bp)(opts)
if opts.bypass != bp {
sniffing.WithBypass(bp)(opts)
if opts.Bypass != bp {
t.Errorf("bypass not set")
}
}
@@ -681,8 +712,8 @@ func TestWithBypass(t *testing.T) {
func TestWithRecorderObject(t *testing.T) {
opts := &HandleOptions{}
ro := &xrecorder.HandlerRecorderObject{}
WithRecorderObject(ro)(opts)
if opts.recorderObject != ro {
sniffing.WithRecorderObject(ro)(opts)
if opts.RecorderObject != ro {
t.Errorf("recorderObject not set")
}
}
@@ -690,8 +721,8 @@ func TestWithRecorderObject(t *testing.T) {
func TestWithLog(t *testing.T) {
opts := &HandleOptions{}
log := xlogger.Nop()
WithLog(log)(opts)
if opts.log != log {
sniffing.WithLog(log)(opts)
if opts.Log != log {
t.Errorf("log not set")
}
}
@@ -732,11 +763,11 @@ func TestHandleHTTP_DialError(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return nil, io.ErrUnexpectedEOF
}),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -782,11 +813,11 @@ func TestHandleHTTP_HTTP10Request(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return net.Dial("tcp", upstream.Listener.Addr().String())
}),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -926,10 +957,10 @@ var _ hop.Hop = (*configurableHop)(nil)
func TestResolveHTTPNode_Bypass(t *testing.T) {
ho := &HandleOptions{
service: "test-svc",
bypass: &mockBypass{contains: true},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Service: "test-svc",
Bypass: &mockBypass{contains: true},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -950,8 +981,8 @@ func TestResolveHTTPNode_Bypass(t *testing.T) {
func TestResolveHTTPNode_NoHop(t *testing.T) {
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -972,9 +1003,9 @@ func TestResolveHTTPNode_NoHop(t *testing.T) {
func TestResolveHTTPNode_HopReturnsNil(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: nil},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: nil},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -999,9 +1030,9 @@ func TestResolveHTTPNode_HopReturnsNil(t *testing.T) {
func TestResolveHTTPNode_HopReturnsValidNode(t *testing.T) {
expectedNode := &chain.Node{Name: "backend", Addr: "10.0.0.1:8080"}
ho := &HandleOptions{
hop: &configurableHop{node: expectedNode},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: expectedNode},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -1019,9 +1050,9 @@ func TestResolveHTTPNode_HopReturnsValidNode(t *testing.T) {
func TestResolveHTTPNode_HopReturnsNodeWithoutAddr(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: &chain.Node{Name: "backend"}},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: &chain.Node{Name: "backend"}},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -1049,8 +1080,8 @@ func TestResolveHTTPNode_HopReturnsNodeWithoutAddr(t *testing.T) {
func TestResolveTLSNode_NoHop(t *testing.T) {
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1067,9 +1098,9 @@ func TestResolveTLSNode_NoHop(t *testing.T) {
func TestResolveTLSNode_HopReturnsNil(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: nil},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: nil},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1087,9 +1118,9 @@ func TestResolveTLSNode_HopReturnsNil(t *testing.T) {
func TestResolveTLSNode_HopReturnsValidNode(t *testing.T) {
expectedNode := &chain.Node{Name: "backend", Addr: "10.0.0.1:443"}
ho := &HandleOptions{
hop: &configurableHop{node: expectedNode},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: expectedNode},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1103,9 +1134,9 @@ func TestResolveTLSNode_HopReturnsValidNode(t *testing.T) {
func TestResolveTLSNode_HopReturnsNodeWithoutAddr(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: &chain.Node{Name: "backend"}},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: &chain.Node{Name: "backend"}},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1720,8 +1751,8 @@ func TestServeH2_InvalidPreface(t *testing.T) {
Recorder: &noopRecorder{},
}
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
clientConn, serverConn := net.Pipe()
@@ -1744,7 +1775,6 @@ func TestServeH2_InvalidPreface(t *testing.T) {
// SSE (Server-Sent Events) Tests
// =============================================================================
func TestRewriteRespBody_SSE_ContentType(t *testing.T) {
// rewriteRespBody now rewrites SSE per-event via the universal wrapper.
resp := &http.Response{
@@ -1960,6 +1990,89 @@ func TestNewRewriteBody(t *testing.T) {
})
}
func TestRewriteJSONDelete(t *testing.T) {
makeRewrite := func(rewriteType, match, replace string) chain.HTTPBodyRewriteSettings {
return chain.HTTPBodyRewriteSettings{
Pattern: regexp.MustCompile(match),
Type: rewriteType,
Replacement: []byte(replace),
}
}
ctx := context.Background()
t.Run("delete matched field", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json-:user.id", "123", ""),
}, "application/json", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{},"keep":1}` {
t.Errorf("body = %q, want %q", string(got), `{"user":{},"keep":1}`)
}
})
t.Run("no match leaves body unchanged", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json-:user.id", "^9$", ""),
}, "application/json", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{"id":123},"keep":1}` {
t.Errorf("body = %q, want %q (unchanged)", string(got), `{"user":{"id":123},"keep":1}`)
}
})
t.Run("non-json content type filtered out", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json-:user.id", "123", ""),
}, "text/plain", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{"id":123},"keep":1}` {
t.Errorf("body = %q, want %q (unchanged)", string(got), `{"user":{"id":123},"keep":1}`)
}
})
t.Run("json: prefix still replaces", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json:user.id", "123", "999"),
}, "application/json", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{"id":"999"},"keep":1}` {
t.Errorf("body = %q, want %q", string(got), `{"user":{"id":"999"},"keep":1}`)
}
})
}
func TestRewriteRespBody_CompressedEncodings(t *testing.T) {
tests := []struct {
name string
+28 -28
View File
@@ -32,7 +32,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
for _, opt := range opts {
opt(&ho)
}
ho.readTimeout = h.effectiveReadTimeout(&ho)
ho.ReadTimeout = h.effectiveReadTimeout(&ho)
buf := new(bytes.Buffer)
clientHello, err := dissector.ParseClientHello(io.TeeReader(conn, buf))
@@ -40,7 +40,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
return err
}
ro := ho.recorderObject
ro := ho.RecorderObject
ro.TLS = &xrecorder.TLSRecorderObject{
ServerName: clientHello.ServerName,
ClientHello: hex.EncodeToString(buf.Bytes()),
@@ -51,14 +51,14 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
host := normalizeHost(clientHello.ServerName, "443")
if host == "" {
if ho.log != nil {
ho.log.Debugf("no sni in clienthello from %s", conn.RemoteAddr())
if ho.Log != nil {
ho.Log.Debugf("no sni in clienthello from %s", conn.RemoteAddr())
}
return errors.New("tls: sni is empty, closing connection")
} else {
ro.Host = host
}
ro.Host = host
if ho.bypass != nil && ho.bypass.Contains(ctx, "tcp", host, bypass.WithService(ho.service)) {
if ho.Bypass != nil && ho.Bypass.Contains(ctx, "tcp", host, bypass.WithService(ho.Service)) {
return xbypass.ErrBypass
}
@@ -67,9 +67,9 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
return err
}
defer cc.Close()
ho.node = node
ho.Node = node
log := ho.log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log := ho.Log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log.Debugf("connected to node %s(%s)", node.Name, node.Addr)
ro.SrcAddr = cc.LocalAddr().String()
@@ -89,7 +89,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
return err
}
xio.SetReadDeadline(cc, time.Now().Add(ho.readTimeout))
xio.SetReadDeadline(cc, time.Now().Add(ho.ReadTimeout))
serverHello, serverHelloErr := dissector.ParseServerHello(io.TeeReader(cc, buf))
xio.SetReadDeadline(cc, time.Time{})
@@ -128,12 +128,12 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
// resolveTLSNode selects a node for a TLS connection by applying hop selection.
func resolveTLSNode(ctx context.Context, host string, ho *HandleOptions) (node *chain.Node, err error) {
node = &chain.Node{}
if ho.hop != nil {
if ho.Hop != nil {
var clientIP net.IP
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.recorderObject.ClientAddr); clientAddr != nil {
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.RecorderObject.ClientAddr); clientAddr != nil {
clientIP = clientAddr.IP
}
node = ho.hop.Select(ctx,
node = ho.Hop.Select(ctx,
hop.ClientIPSelectOption(clientIP),
hop.HostSelectOption(host),
hop.ProtocolSelectOption(sniffing.ProtoTLS),
@@ -142,7 +142,7 @@ func resolveTLSNode(ctx context.Context, host string, ho *HandleOptions) (node *
if node == nil {
return nil, errors.New("node not available")
}
ho.recorderObject.Node = node.Name
ho.RecorderObject.Node = node.Name
if node.Addr == "" {
node = &chain.Node{
Name: node.Name,
@@ -154,12 +154,12 @@ func resolveTLSNode(ctx context.Context, host string, ho *HandleOptions) (node *
// dialTLS selects a node and establishes a TLS connection.
func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (node *chain.Node, cc net.Conn, err error) {
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
if node = ho.node; node != nil {
if node = ho.Node; node != nil {
cc, err = dial(ctx, "tcp", node.Addr)
return
}
@@ -169,7 +169,7 @@ func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (
return
}
ro := ho.recorderObject
ro := ho.RecorderObject
addr := node.Addr
network := "tcp"
if opts := node.Options(); opts != nil {
@@ -188,19 +188,19 @@ func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (
}
ro.Host = addr
ho.log = ho.log.WithFields(map[string]any{
ho.Log = ho.Log.WithFields(map[string]any{
"host": host,
"node": node.Name,
"dst": fmt.Sprintf("%s/%s", addr, network),
})
ho.log.Debugf("find node for host %s -> %s(%s)", host, node.Name, addr)
ho.Log.Debugf("find node for host %s -> %s(%s)", host, node.Name, addr)
cc, err = dial(ctx, network, addr)
if err != nil {
if marker := node.Marker(); marker != nil {
marker.Mark()
}
ho.log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
ho.Log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
return
}
@@ -217,8 +217,8 @@ func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (
// dynamically generated certificate. The decrypted traffic is then handled
// as HTTP.
func (h *Sniffer) terminateTLS(ctx context.Context, conn, cc net.Conn, clientHello *dissector.ClientHelloInfo, ho *HandleOptions) error {
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
nextProtos := clientHello.SupportedProtos
if h.NegotiatedProtocol != "" {
@@ -317,14 +317,14 @@ func (h *Sniffer) terminateTLS(ctx context.Context, conn, cc net.Conn, clientHel
}
opts := []HandleOption{
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return clientConn, nil
}),
WithHTTPKeepalive(true),
WithNode(ho.node),
WithBypass(ho.bypass),
WithRecorderObject(ro),
WithLog(log),
sniffing.WithHTTPKeepalive(true),
sniffing.WithNode(ho.Node),
sniffing.WithBypass(ho.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
}
return h.HandleHTTP(ctx, serverConn, opts...)
}
+1 -1
View File
@@ -32,7 +32,7 @@ func (c *Client) Dial(ctx context.Context, addr string) (net.Conn, error) {
addr = net.JoinHostPort(c.Host, strconv.Itoa(raddr.Port))
}
connCtx, cancel := context.WithCancel(ctx)
connCtx, cancel := context.WithCancel(context.WithoutCancel(ctx))
cid := xid.New().String()
cn := &clientConn{
client: c.Client,
+46
View File
@@ -0,0 +1,46 @@
package sniffing
import (
"encoding/hex"
dissector "github.com/go-gost/tls-dissector"
tls_util "github.com/go-gost/x/internal/util/tls"
xrecorder "github.com/go-gost/x/recorder"
)
// PopulateQUICClientHello sets ro.Proto to ProtoQUIC and populates ro.TLS
// from the parsed QUIC ClientHello. dgram is the raw client datagram (stored
// as hex in ro.TLS.ClientHello). Values are the client-offered ones and may
// be overwritten by PopulateQUICServerHello with negotiated values later.
func PopulateQUICClientHello(dgram []byte, info *dissector.ClientHelloInfo, ro *xrecorder.HandlerRecorderObject) {
ro.Proto = ProtoQUIC
ro.TLS = &xrecorder.TLSRecorderObject{
ServerName: info.ServerName,
ClientHello: hex.EncodeToString(dgram),
}
if len(info.SupportedProtos) > 0 {
ro.TLS.Proto = info.SupportedProtos[0]
}
if len(info.SupportedVersions) > 0 {
ro.TLS.Version = tls_util.Version(info.SupportedVersions[0]).String()
}
if len(info.CipherSuites) > 0 {
ro.TLS.CipherSuite = tls_util.CipherSuite(info.CipherSuites[0]).String()
}
if len(info.CompressionMethods) > 0 {
ro.TLS.CompressionMethod = info.CompressionMethods[0]
}
}
// PopulateQUICServerHello overwrites ro.TLS fields with negotiated values
// from a parsed QUIC ServerHello. ro.TLS must already be set (typically by
// PopulateQUICClientHello).
func PopulateQUICServerHello(sh *dissector.ServerHelloInfo, ro *xrecorder.HandlerRecorderObject) {
ro.TLS.CipherSuite = tls_util.CipherSuite(sh.CipherSuite).String()
ro.TLS.Version = tls_util.Version(sh.Version).String()
ro.TLS.CompressionMethod = sh.CompressionMethod
if sh.Proto != "" {
ro.TLS.Proto = sh.Proto
}
}
+48
View File
@@ -0,0 +1,48 @@
package sniffing
import (
"context"
"net"
"sync"
)
// HandleFunc handles a connection whose protocol has been detected as proto.
// The receiver s carries the per-connection Sniffer state (MITM certificate,
// HTTP cache, recorder, etc.).
type HandleFunc func(s *Sniffer, ctx context.Context, network string, conn net.Conn, opts ...HandleOption) error
var protoHandlers sync.Map // map[string]HandleFunc
// Register associates a protocol with its handler. Duplicate registration
// panics, mirroring the house registry pattern.
func Register(proto string, fn HandleFunc) {
if _, dup := protoHandlers.LoadOrStore(proto, fn); dup {
panic("sniffing: duplicate protocol handler: " + proto)
}
}
// Get returns the handler registered for proto.
func Get(proto string) (HandleFunc, bool) {
v, ok := protoHandlers.Load(proto)
if !ok {
return nil, false
}
return v.(HandleFunc), true
}
// Dispatch routes a detected protocol to its registered handler. It returns
// (false, nil) when no handler is registered for proto, signalling the caller
// to fall through to raw forwarding.
func Dispatch(s *Sniffer, ctx context.Context, network string, conn net.Conn, proto string, opts ...HandleOption) (handled bool, err error) {
fn, ok := Get(proto)
if !ok {
return false, nil
}
return true, fn(s, ctx, network, conn, opts...)
}
func init() {
Register(ProtoHTTP, (*Sniffer).HandleHTTP)
Register(ProtoTLS, (*Sniffer).HandleTLS)
Register(ProtoRedis, (*Sniffer).HandleRedis)
}
+84
View File
@@ -0,0 +1,84 @@
package sniffing
import (
"context"
"net"
"testing"
)
// TestRegisterGet verifies a handler can be registered and looked up by proto.
func TestRegisterGet(t *testing.T) {
const proto = "testproto-get"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ string, _ net.Conn, _ ...HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
fn, ok := Get(proto)
if !ok {
t.Fatal("Get: handler not found")
}
if err := fn(&Sniffer{}, context.Background(), "tcp", nil); err != nil {
t.Fatalf("fn() error = %v", err)
}
if !called {
t.Fatal("registered handler was not called")
}
if _, ok := Get("no-such-proto"); ok {
t.Fatal("Get: unexpected handler for unknown proto")
}
}
// TestRegisterDuplicate verifies registering the same proto twice panics.
func TestRegisterDuplicate(t *testing.T) {
const proto = "testproto-dup"
Register(proto, func(*Sniffer, context.Context, string, net.Conn, ...HandleOption) error {
return nil
})
defer protoHandlers.Delete(proto)
defer func() {
if recover() == nil {
t.Fatal("expected panic on duplicate registration")
}
}()
Register(proto, func(*Sniffer, context.Context, string, net.Conn, ...HandleOption) error {
return nil
})
}
// TestDispatch_Unknown verifies Dispatch falls through (false, nil) for an
// unregistered proto.
func TestDispatch_Unknown(t *testing.T) {
handled, err := Dispatch(&Sniffer{}, context.Background(), "tcp", nil, "no-such-proto")
if handled {
t.Fatal("Dispatch: unknown proto handled = true, want false")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
}
// TestDispatch_Known verifies Dispatch routes a registered proto to its handler.
func TestDispatch_Known(t *testing.T) {
const proto = "testproto-known"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ string, _ net.Conn, _ ...HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
handled, err := Dispatch(&Sniffer{}, context.Background(), "tcp", nil, proto)
if !handled {
t.Fatal("Dispatch: known proto handled = false, want true")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
if !called {
t.Fatal("Dispatch: registered handler was not called")
}
}
+17 -3
View File
@@ -12,12 +12,26 @@ import (
)
const (
ProtoHTTP = "http"
ProtoTLS = "tls"
ProtoSSH = "ssh"
ProtoHTTP = "http"
ProtoTLS = "tls"
ProtoSSH = "ssh"
ProtoQUIC = "quic"
ProtoRedis = "redis"
)
func Sniff(ctx context.Context, r *bufio.Reader) (proto string, err error) {
// Tier 1: Peek 1 byte to detect protocols identifiable by first byte.
// Peek(1) does not block (unless the connection has zero bytes).
b, err := r.Peek(1)
if err != nil {
return
}
switch b[0] {
case '*', '+', '-', ':', '$':
return ProtoRedis, nil
}
// Tier 2: Peek 5 bytes for TLS/HTTP/SSH (existing behavior).
hdr, err := r.Peek(dissector.RecordHeaderLen)
if err != nil {
return
+44 -13
View File
@@ -10,6 +10,8 @@ import (
"time"
"github.com/go-gost/core/bypass"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/recorder"
"github.com/go-gost/x/internal/util/httpcache"
@@ -33,58 +35,87 @@ const (
var DefaultCertPool = tls_util.NewMemoryCertPool()
// HandleOptions holds configuration options for sniffing handlers.
// Fields are exported because forwarder's protocol handlers set them
// across the package boundary.
type HandleOptions struct {
service string
dial func(ctx context.Context, network, address string) (net.Conn, error)
dialTLS func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error)
Service string
Dial func(ctx context.Context, network, address string) (net.Conn, error)
DialTLS func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error)
bypass bypass.Bypass
recorderObject *xrecorder.HandlerRecorderObject
log logger.Logger
HTTPKeepalive bool
ReadTimeout time.Duration
Node *chain.Node
Hop hop.Hop
Bypass bypass.Bypass
RecorderObject *xrecorder.HandlerRecorderObject
Log logger.Logger
}
// HandleOption configures HandleOptions for sniffing handlers.
type HandleOption func(opts *HandleOptions)
// WithService sets the service name for bypass lookups.
// WithService sets the service name for bypass and selection lookups.
func WithService(service string) HandleOption {
return func(opts *HandleOptions) {
opts.service = service
opts.Service = service
}
}
// WithDial sets the dial function used to establish upstream connections.
func WithDial(dial func(ctx context.Context, network, address string) (net.Conn, error)) HandleOption {
return func(opts *HandleOptions) {
opts.dial = dial
opts.Dial = dial
}
}
// WithDialTLS sets the dial function used for TLS-wrapped upstream connections.
func WithDialTLS(dialTLS func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error)) HandleOption {
return func(opts *HandleOptions) {
opts.dialTLS = dialTLS
opts.DialTLS = dialTLS
}
}
// WithHTTPKeepalive enables or disables HTTP keep-alive on the upstream connection.
func WithHTTPKeepalive(keepalive bool) HandleOption {
return func(opts *HandleOptions) {
opts.HTTPKeepalive = keepalive
}
}
// WithNode sets a pre-resolved chain node to connect to, bypassing hop selection.
func WithNode(node *chain.Node) HandleOption {
return func(opts *HandleOptions) {
opts.Node = node
}
}
// WithHop sets the hop used for node selection when routing requests.
func WithHop(h hop.Hop) HandleOption {
return func(opts *HandleOptions) {
opts.Hop = h
}
}
// WithBypass sets the bypass rules for filtering requests by host.
func WithBypass(bypass bypass.Bypass) HandleOption {
return func(opts *HandleOptions) {
opts.bypass = bypass
opts.Bypass = bypass
}
}
// WithRecorderObject sets the recorder object for capturing traffic metadata.
func WithRecorderObject(ro *xrecorder.HandlerRecorderObject) HandleOption {
return func(opts *HandleOptions) {
opts.recorderObject = ro
opts.RecorderObject = ro
}
}
// WithLog sets the logger for the handler.
func WithLog(log logger.Logger) HandleOption {
return func(opts *HandleOptions) {
opts.log = log
opts.Log = log
}
}
+3 -3
View File
@@ -33,14 +33,14 @@ func (h *Sniffer) serveH2(ctx context.Context, network string, conn net.Conn, ho
return errors.New("h2: invalid client preface")
}
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
ro.Time = time.Time{}
tr := &http2.Transport{
DialTLSContext: func(ctx context.Context, nw, addr string, cfg *tls.Config) (net.Conn, error) {
if dial := ho.dialTLS; dial != nil {
if dial := ho.DialTLS; dial != nil {
return dial(ctx, network, addr, cfg)
}
+12 -4
View File
@@ -44,13 +44,20 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, network string, conn net.Conn,
return err
}
log := ho.log
log := ho.Log
if log.IsLevelEnabled(logger.TraceLevel) {
dump, _ := httputil.DumpRequest(req, false)
log.Trace(string(dump))
}
ro := ho.recorderObject
ro := ho.RecorderObject
// Copy ro so that all internal recording (cache-hit serveCachedResponse
// and cache-miss httpRoundTrip) happens on a local clone, preventing
// double-record when the caller's defer also records ro.
ro2 := &xrecorder.HandlerRecorderObject{}
*ro2 = *ro
ro = ro2
if clientIP := xhttp.GetClientIP(req); clientIP != nil {
ro.ClientIP = clientIP.String()
@@ -67,12 +74,12 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, network string, conn net.Conn,
ro.Host = host
log = log.WithFields(map[string]any{"host": host})
if ho.bypass != nil && ho.bypass.Contains(ctx, network, host, bypass.WithService(ho.service)) {
if ho.Bypass != nil && ho.Bypass.Contains(ctx, network, host, bypass.WithService(ho.Service)) {
return xbypass.ErrBypass
}
}
dialFn := ho.dial
dialFn := ho.Dial
if dialFn == nil {
dialFn = (&net.Dialer{}).DialContext
}
@@ -188,6 +195,7 @@ func (h *Sniffer) serveCachedResponse(ctx context.Context, rw io.Writer, req *ht
defer resp.Body.Close()
ro.Time = time.Now()
ro.HTTP.StatusCode = resp.StatusCode
ro.HTTP.Cached = true
ro.HTTP.Response.Header = resp.Header.Clone()
ro.HTTP.Response.ContentLength = resp.ContentLength
log.Debugf("cache hit: %s", httpcache.Key(req.Method, req.Host, req.RequestURI))
+188
View File
@@ -0,0 +1,188 @@
package sniffing
import (
"bufio"
"bytes"
"context"
"fmt"
"io"
"net"
"strconv"
"strings"
"time"
xnet "github.com/go-gost/x/internal/net"
xrecorder "github.com/go-gost/x/recorder"
)
// ParseRedisMetadata reads a single RESP message from r and populates
// ro.Redis with the extracted command and key.
//
// r must provide exactly one RESP message — the caller is expected to
// wrap the conn with io.TeeReader(conn, buf) so that the bytes are both
// parsed and captured in buf for replay to the upstream.
//
// On success, ro.Redis is non-nil. On error (malformed RESP, truncated
// message), ro.Redis is nil and the caller should fall back to raw
// forwarding.
func ParseRedisMetadata(r io.Reader, ro *xrecorder.HandlerRecorderObject) error {
br := bufio.NewReader(r)
b, err := br.ReadByte()
if err != nil {
return err
}
switch b {
case '*':
return parseRESPArray(br, ro)
case '$':
s, err := readBulkStringPayload(br)
if err != nil {
return err
}
ro.Redis = &xrecorder.RedisRecorderObject{Command: strings.ToUpper(s)}
return nil
case '+', '-':
s, err := readRESPLine(br)
if err != nil {
return err
}
ro.Redis = &xrecorder.RedisRecorderObject{Command: strings.ToUpper(s)}
return nil
case ':':
s, err := readRESPLine(br)
if err != nil {
return err
}
ro.Redis = &xrecorder.RedisRecorderObject{Command: s}
return nil
default:
return fmt.Errorf("unknown RESP type byte: 0x%02x", b)
}
}
// readRESPLine reads until '\n' and strips the trailing "\r\n".
func readRESPLine(br *bufio.Reader) (string, error) {
line, err := br.ReadString('\n')
if err != nil {
return "", err
}
return strings.TrimRight(line, "\r\n"), nil
}
// readBulkStringPayload reads <len>\r\n<bytes>\r\n (the leading '$' must
// already be consumed by the caller). Returns the string value.
func readBulkStringPayload(br *bufio.Reader) (string, error) {
line, err := readRESPLine(br)
if err != nil {
return "", err
}
n, err := strconv.Atoi(line)
if err != nil {
return "", fmt.Errorf("invalid bulk string length: %s", line)
}
if n < 0 {
return "", nil // null bulk string
}
buf := make([]byte, n)
if _, err := io.ReadFull(br, buf); err != nil {
return "", err
}
// consume trailing \r\n
if _, err := br.Discard(2); err != nil {
return "", err
}
return string(buf), nil
}
// parseRESPArray parses a RESP array: <n>\r\n, then n bulk strings.
// The leading '*' must already be consumed by the caller.
func parseRESPArray(br *bufio.Reader, ro *xrecorder.HandlerRecorderObject) error {
line, err := readRESPLine(br)
if err != nil {
return err
}
n, err := strconv.Atoi(line)
if err != nil {
return fmt.Errorf("invalid RESP array length: %s", line)
}
if n < 0 {
return nil // null array
}
for i := 0; i < n; i++ {
b, err := br.ReadByte()
if err != nil {
return err
}
if b != '$' {
return fmt.Errorf("expected bulk string in RESP array, got byte 0x%02x", b)
}
s, err := readBulkStringPayload(br)
if err != nil {
return err
}
if i == 0 {
// ponytail: first element allocated inline, second appended.
// A nil check on the first element is enough — RESP
// commands always have at least one element.
ro.Redis = &xrecorder.RedisRecorderObject{Command: strings.ToUpper(s)}
} else if i == 1 {
ro.Redis.Key = s
}
}
if ro.Redis == nil {
ro.Redis = &xrecorder.RedisRecorderObject{}
}
return nil
}
// HandleRedis proxies a sniffed Redis connection. It parses the first RESP
// message for metadata recording, then replays the captured bytes to the
// upstream and pipes traffic bidirectionally.
//
// The network parameter is passed through to the injected dial function.
// Bypass checks are not performed here — Redis has no host concept for
// bypass to match against; protocol-based routing is handled by the
// hop selector in the forwarder path.
func (h *Sniffer) HandleRedis(ctx context.Context, network string, conn net.Conn, opts ...HandleOption) error {
var ho HandleOptions
for _, opt := range opts {
opt(&ho)
}
buf := new(bytes.Buffer)
if err := ParseRedisMetadata(io.TeeReader(conn, buf), ho.RecorderObject); err != nil {
return err
}
log := ho.Log
ro := ho.RecorderObject
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
cc, err := dial(ctx, network, ro.Host)
if err != nil {
return err
}
defer cc.Close()
log = log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
ro.SrcAddr = cc.LocalAddr().String()
ro.DstAddr = cc.RemoteAddr().String()
if _, err := buf.WriteTo(cc); err != nil {
return err
}
log.Infof("%s <-> %s", ro.RemoteAddr, ro.Host)
xnet.Pipe(ctx, conn, cc)
log.WithFields(map[string]any{
"duration": time.Since(ro.Time),
}).Infof("%s >-< %s", ro.RemoteAddr, ro.Host)
return nil
}
+55 -24
View File
@@ -5,6 +5,7 @@ import (
"bytes"
"context"
"crypto/tls"
"encoding/json"
"errors"
"io"
"net"
@@ -30,6 +31,16 @@ func (n *noopRecorder) Record(_ context.Context, _ []byte, _ ...recorder.RecordO
return nil
}
// captureRecorder records every payload written through recorder.Recorder.
type captureRecorder struct {
data [][]byte
}
func (r *captureRecorder) Record(_ context.Context, b []byte, _ ...recorder.RecordOption) error {
r.data = append(r.data, append([]byte(nil), b...))
return nil
}
type mockBypass struct {
contains bool
whitelist bool
@@ -177,8 +188,8 @@ func TestWrapErr(t *testing.T) {
func TestWithService(t *testing.T) {
opts := &HandleOptions{}
WithService("mysvc")(opts)
if opts.service != "mysvc" {
t.Errorf("service = %q, want %q", opts.service, "mysvc")
if opts.Service != "mysvc" {
t.Errorf("service = %q, want %q", opts.Service, "mysvc")
}
}
@@ -188,7 +199,7 @@ func TestWithDial(t *testing.T) {
return nil, nil
}
WithDial(dial)(opts)
if opts.dial == nil {
if opts.Dial == nil {
t.Error("dial should be set")
}
}
@@ -199,7 +210,7 @@ func TestWithDialTLS(t *testing.T) {
return nil, nil
}
WithDialTLS(dialTLS)(opts)
if opts.dialTLS == nil {
if opts.DialTLS == nil {
t.Error("dialTLS should be set")
}
}
@@ -208,7 +219,7 @@ func TestWithBypass(t *testing.T) {
opts := &HandleOptions{}
bp := &mockBypass{contains: true}
WithBypass(bp)(opts)
if opts.bypass != bp {
if opts.Bypass != bp {
t.Errorf("bypass not set")
}
}
@@ -217,7 +228,7 @@ func TestWithRecorderObject(t *testing.T) {
opts := &HandleOptions{}
ro := &xrecorder.HandlerRecorderObject{}
WithRecorderObject(ro)(opts)
if opts.recorderObject != ro {
if opts.RecorderObject != ro {
t.Errorf("recorderObject not set")
}
}
@@ -226,7 +237,7 @@ func TestWithLog(t *testing.T) {
opts := &HandleOptions{}
log := xlogger.Nop()
WithLog(log)(opts)
if opts.log != log {
if opts.Log != log {
t.Errorf("log not set")
}
}
@@ -242,9 +253,10 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
}))
defer upstream.Close()
rec := &captureRecorder{}
h := &Sniffer{
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
Recorder: rec,
}
ro := &xrecorder.HandlerRecorderObject{}
@@ -278,16 +290,35 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
t.Errorf("status = %d, want %d", resp.StatusCode, http.StatusOK)
}
clientConn.Close()
if err := <-errCh; err != nil {
t.Logf("HandleHTTP returned: %v", err)
// Drain the body so HandleHTTP's response write completes before the
// client side closes (net.Pipe writes block until read).
body, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatalf("read body: %v", err)
}
if string(body) != "OK from upstream" {
t.Errorf("body = %q, want %q", body, "OK from upstream")
}
if ro.SrcAddr == "" {
t.Error("SrcAddr should be populated")
clientConn.Close()
if err := <-errCh; err != nil {
t.Errorf("HandleHTTP returned: %v", err)
}
if ro.DstAddr == "" {
t.Error("DstAddr should be populated")
// HandleHTTP records on an internal clone of ro and publishes it via
// h.Recorder, so assert on the captured payload.
if len(rec.data) != 1 {
t.Fatalf("expected 1 recorder event, got %d", len(rec.data))
}
var recorded xrecorder.HandlerRecorderObject
if err := json.Unmarshal(rec.data[0], &recorded); err != nil {
t.Fatalf("unmarshal recorder event: %v", err)
}
if recorded.SrcAddr == "" {
t.Error("recorded srcAddr should be populated")
}
if recorded.DstAddr == "" {
t.Error("recorded dstAddr should be populated")
}
}
@@ -416,8 +447,8 @@ func TestCopyWebsocketFrame_Basic(t *testing.T) {
}
var frame bytes.Buffer
frame.WriteByte(0x81) // FIN + text opcode
frame.WriteByte(0x82) // MASK + len=2
frame.WriteByte(0x81) // FIN + text opcode
frame.WriteByte(0x82) // MASK + len=2
frame.Write(mask)
frame.Write(maskedPayload)
@@ -664,8 +695,8 @@ func TestServeH2_InvalidPreface(t *testing.T) {
Recorder: &noopRecorder{},
}
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
clientConn, serverConn := net.Pipe()
@@ -692,8 +723,8 @@ func TestServeH2_ShortRead(t *testing.T) {
Recorder: &noopRecorder{},
}
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
clientConn, serverConn := net.Pipe()
@@ -1058,9 +1089,9 @@ func TestHandleTLS_NonTLSData(t *testing.T) {
func TestHandleUpgradeResponse_WebsocketEnabled(t *testing.T) {
h := &Sniffer{
Websocket: true,
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
Websocket: true,
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
}
ro := &xrecorder.HandlerRecorderObject{}
+10 -11
View File
@@ -6,7 +6,6 @@ import (
"crypto/tls"
"crypto/x509"
"encoding/hex"
"errors"
"io"
"net"
"time"
@@ -37,9 +36,9 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
return err
}
log := ho.log
log := ho.Log
ro := ho.recorderObject
ro := ho.RecorderObject
ro.TLS = &xrecorder.TLSRecorderObject{
ServerName: clientHello.ServerName,
ClientHello: hex.EncodeToString(buf.Bytes()),
@@ -53,15 +52,15 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
if log != nil {
log.Debugf("no sni in clienthello from %s", conn.RemoteAddr())
}
return errors.New("tls: sni is empty, closing connection")
} else {
ro.Host = host
}
ro.Host = host
if ho.bypass != nil && ho.bypass.Contains(ctx, network, host, bypass.WithService(ho.service)) {
if ho.Bypass != nil && ho.Bypass.Contains(ctx, network, host, bypass.WithService(ho.Service)) {
return xbypass.ErrBypass
}
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
@@ -80,7 +79,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
if host == "" {
host = ro.Host
}
if h.MitmBypass == nil || !h.MitmBypass.Contains(ctx, network, host, bypass.WithService(ho.service)) {
if h.MitmBypass == nil || !h.MitmBypass.Contains(ctx, network, host, bypass.WithService(ho.Service)) {
return h.terminateTLS(ctx, network, xnet.NewReadWriteConn(io.MultiReader(buf, conn), conn, conn), cc, clientHello, &ho)
}
}
@@ -129,8 +128,8 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
// dynamically generated certificate. The decrypted traffic is then handled
// as HTTP.
func (h *Sniffer) terminateTLS(ctx context.Context, network string, conn, cc net.Conn, clientHello *dissector.ClientHelloInfo, ho *HandleOptions) error {
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
nextProtos := clientHello.SupportedProtos
if h.NegotiatedProtocol != "" {
@@ -235,7 +234,7 @@ func (h *Sniffer) terminateTLS(ctx context.Context, network string, conn, cc net
WithDialTLS(func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error) {
return clientConn, nil
}),
WithBypass(ho.bypass),
WithBypass(ho.Bypass),
WithRecorderObject(ro),
WithLog(log),
}
+3
View File
@@ -69,6 +69,9 @@ func ParseAuthorizedKeysFile(name string) (map[string]bool, error) {
return nil, err
}
authorizedKeysMap[string(pubKey.Marshal())] = true
if len(rest) == 0 {
break
}
authorizedKeysBytes = rest
}
+19 -6
View File
@@ -2,6 +2,7 @@ package ws
import (
"context"
"io"
"net"
"sync"
"time"
@@ -21,7 +22,7 @@ type WebsocketConn interface {
type websocketConn struct {
*websocket.Conn
rb []byte
r io.Reader
ctx context.Context
mux sync.Mutex
}
@@ -44,12 +45,24 @@ func ContextConn(ctx context.Context, conn *websocket.Conn) WebsocketConn {
}
func (c *websocketConn) Read(b []byte) (n int, err error) {
if len(c.rb) == 0 {
_, c.rb, err = c.Conn.ReadMessage()
for {
if c.r == nil {
_, c.r, err = c.Conn.NextReader()
if err != nil {
return 0, err
}
}
n, err = c.r.Read(b)
if err == io.EOF {
c.r = nil
if n > 0 {
return n, nil
}
continue
}
return n, err
}
n = copy(b, c.rb)
c.rb = c.rb[n:]
return
}
func (c *websocketConn) Write(b []byte) (n int, err error) {
+6
View File
@@ -31,6 +31,12 @@ func (l *redirectListener) listenUDP(addr string) (*net.UDPConn, error) {
lc := net.ListenConfig{
Control: func(network, address string, c syscall.RawConn) error {
return c.Control(func(fd uintptr) {
// Must match dialUDP's socket so per-packet tproxy sockets can
// bind a destination port that overlaps this listener without
// "address already in use" (gost#413).
if err := unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEADDR, 1); err != nil {
l.logger.Errorf("SetsockoptInt(SOL_SOCKET, SO_REUSEADDR, 1): %v", err)
}
if err := unix.SetsockoptInt(int(fd), unix.SOL_IP, unix.IP_TRANSPARENT, 1); err != nil {
l.logger.Errorf("SetsockoptInt(SOL_IP, IP_TRANSPARENT, 1): %v", err)
}
+1
View File
@@ -71,6 +71,7 @@ func (l *tlsListener) Init(md md.Metadata) (err error) {
l.logger.Debugf("tcp keepalive enabled: idle=%v interval=%v count=%d",
l.md.keepaliveIdle, l.md.keepaliveInterval, l.md.keepaliveCount)
}
ln = xnet.WrapNoDelayListener(ln)
ln = proxyproto.WrapListener(l.options.ProxyProtocol, ln, 10*time.Second)
ln = metrics.WrapListener(l.options.Service, ln)
ln = stats.WrapListener(ln, l.options.Stats)
+1
View File
@@ -122,6 +122,7 @@ func (l *wsListener) Init(md md.Metadata) (err error) {
l.log.Debugf("tcp keepalive enabled: idle=%v interval=%v count=%d",
l.md.keepaliveIdle, l.md.keepaliveInterval, l.md.keepaliveCount)
}
ln = xnet.WrapNoDelayListener(ln)
ln = proxyproto.WrapListener(l.options.ProxyProtocol, ln, 10*time.Second)
ln = metrics.WrapListener(l.options.Service, ln)
ln = stats.WrapListener(ln, l.options.Stats)
+37 -7
View File
@@ -62,8 +62,17 @@ func LevelOption(level logger.LogLevel) Option {
// slogLogger implements logger.Logger backed by log/slog.
type slogLogger struct {
// base is the pristine slog.Logger before any WithFields attrs are
// applied. It is used to rebuild logger after each WithFields call.
base *slog.Logger
// logger is the effective slog.Logger used for emitting entries.
logger *slog.Logger
level *slog.LevelVar
// args holds the accumulated WithFields key-value pairs in insertion
// order, deduplicated by key (last write wins). slog.Logger.With does not
// deduplicate, so chaining WithFields with a repeated key would otherwise
// emit duplicate JSON keys.
args []any
level *slog.LevelVar
}
// NewLogger creates a new logger.Logger backed by log/slog.
@@ -90,6 +99,7 @@ func NewLogger(opts ...Option) logger.Logger {
}
return &slogLogger{
base: sl,
logger: sl,
level: levelVar,
}
@@ -186,18 +196,38 @@ func revertLevel(lvl slog.Level) logger.LogLevel {
}
// WithFields returns a new Logger with the given fields attached to every
// subsequent log entry.
// subsequent log entry. A field whose key was already set by an earlier
// WithFields call is overwritten (last write wins) rather than duplicated.
func (l *slogLogger) WithFields(fields map[string]any) logger.Logger {
attrs := make([]any, 0, len(fields)*2)
for k, v := range fields {
attrs = append(attrs, k, v)
}
args := mergeArgs(l.args, fields)
return &slogLogger{
logger: l.logger.With(attrs...),
base: l.base,
logger: l.base.With(args...),
args: args,
level: l.level,
}
}
// mergeArgs merges fields into args (a flat [key, value, ...] slice),
// overwriting any existing key in place to preserve insertion order.
func mergeArgs(args []any, fields map[string]any) []any {
index := make(map[string]int, len(args)/2+len(fields))
out := make([]any, 0, len(args)+len(fields)*2)
for i := 0; i < len(args); i += 2 {
index[args[i].(string)] = len(out)
out = append(out, args[i], args[i+1])
}
for k, v := range fields {
if i, ok := index[k]; ok {
out[i+1] = v
} else {
index[k] = len(out)
out = append(out, k, v)
}
}
return out
}
// Trace logs at Trace level.
func (l *slogLogger) Trace(args ...any) {
l.log(levelTrace, args...)
+20
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"log/slog"
"regexp"
"strings"
"testing"
"time"
@@ -184,6 +185,25 @@ func TestWithFields(t *testing.T) {
}
}
func TestWithFieldsDeduplicatesKeys(t *testing.T) {
var buf bytes.Buffer
l := NewLogger(
OutputOption(&buf),
LevelOption(logger.InfoLevel),
FormatOption(logger.JSONFormat),
)
l = l.WithFields(map[string]any{"kind": "service", "service": "s"}).WithFields(map[string]any{"kind": "handler"})
l.Info("hello")
out := buf.String()
// "kind" must appear exactly once, with the last value winning.
if strings.Count(out, `"kind"`) != 1 {
t.Errorf("expected one %q key, got %q", "kind", out)
}
if !strings.Contains(out, `"kind":"handler"`) {
t.Errorf("expected kind=handler, got %q", out)
}
}
func TestIsLevelEnabled(t *testing.T) {
l := NewLogger(LevelOption(logger.InfoLevel))
if !l.IsLevelEnabled(logger.WarnLevel) {
+8
View File
@@ -55,6 +55,7 @@ type HTTPRecorderObject struct {
Proto string `json:"proto"`
Scheme string `json:"scheme"`
URI string `json:"uri"`
Cached bool `json:"cached"`
StatusCode int `json:"statusCode"`
Request HTTPRequestRecorderObject `json:"request"`
Response HTTPResponseRecorderObject `json:"response"`
@@ -113,6 +114,12 @@ type DNSRecorderObject struct {
Cached bool `json:"cached"`
}
// RedisRecorderObject holds the recorded data of a sniffed Redis command.
type RedisRecorderObject struct {
Command string `json:"command"`
Key string `json:"key"`
}
// HandlerRecorderObject bundles traffic metadata recorded at the handler
// level — addresses, protocols, transferred bytes, and optional sub-records
// for HTTP, WebSocket, TLS, and DNS traffic.
@@ -134,6 +141,7 @@ type HandlerRecorderObject struct {
Websocket *WebsocketRecorderObject `json:"websocket,omitempty"`
TLS *TLSRecorderObject `json:"tls,omitempty"`
DNS *DNSRecorderObject `json:"dns,omitempty"`
Redis *RedisRecorderObject `json:"redis,omitempty"`
Route string `json:"route,omitempty"`
InputBytes uint64 `json:"inputBytes"`
OutputBytes uint64 `json:"outputBytes"`
+39 -28
View File
@@ -30,39 +30,50 @@ func (f *failFilter[T]) Filter(ctx context.Context, vs ...T) []T {
}
var l []T
for _, v := range vs {
maxFails := f.maxFails
failTimeout := f.failTimeout
if mi, _ := any(v).(metadata.Metadatable); mi != nil {
if md := mi.Metadata(); md != nil {
if md.IsExists(labelMaxFails) {
maxFails = mdutil.GetInt(md, labelMaxFails)
}
if md.IsExists(labelFailTimeout) {
failTimeout = mdutil.GetDuration(md, labelFailTimeout)
}
}
if !IsFailed(v, f.maxFails, f.failTimeout) {
l = append(l, v)
}
if maxFails <= 0 {
maxFails = 1
}
if failTimeout <= 0 {
failTimeout = DefaultFailTimeout
}
if mi, _ := any(v).(selector.Markable); mi != nil {
if marker := mi.Marker(); marker != nil {
if marker.Count() < int64(maxFails) ||
time.Since(marker.Time()) >= failTimeout {
l = append(l, v)
}
continue
}
}
l = append(l, v)
}
return l
}
// IsFailed reports whether v is currently marked failed: its failure count
// has reached maxFails and the last failure is still within failTimeout.
// Per-item metadata keys "maxFails" and "failTimeout" override the given
// defaults. Items without a failure marker are never considered failed.
func IsFailed[T any](v T, maxFails int, failTimeout time.Duration) bool {
maxFails, failTimeout = effectiveFailSettings(v, maxFails, failTimeout)
mi, _ := any(v).(selector.Markable)
if mi == nil || mi.Marker() == nil {
return false
}
marker := mi.Marker()
return !(marker.Count() < int64(maxFails) || time.Since(marker.Time()) >= failTimeout)
}
// effectiveFailSettings resolves the maxFails/failTimeout for v, honoring
// per-item metadata overrides and defaulting to 1 / DefaultFailTimeout.
func effectiveFailSettings[T any](v T, maxFails int, failTimeout time.Duration) (int, time.Duration) {
if mi, _ := any(v).(metadata.Metadatable); mi != nil {
if md := mi.Metadata(); md != nil {
if md.IsExists(labelMaxFails) {
maxFails = mdutil.GetInt(md, labelMaxFails)
}
if md.IsExists(labelFailTimeout) {
failTimeout = mdutil.GetDuration(md, labelFailTimeout)
}
}
}
if maxFails <= 0 {
maxFails = 1
}
if failTimeout <= 0 {
failTimeout = DefaultFailTimeout
}
return maxFails, failTimeout
}
type backupFilter[T any] struct{}
// BackupFilter filters the backup objects.
+30
View File
@@ -264,3 +264,33 @@ func TestFilterMetadataLabels(t *testing.T) {
t.Fatalf("failTimeout: expected 30s, got %v", ft)
}
}
// --- IsFailed shared helper tests ---
func TestIsFailed(t *testing.T) {
item := newTestMarkable(nil)
if IsFailed(item, 1, 10*time.Second) {
t.Fatal("unmarked item should not be failed")
}
item.Marker().Mark()
if !IsFailed(item, 1, 10*time.Second) {
t.Fatal("marked item with maxFails=1 should be failed")
}
if IsFailed(item, 2, 10*time.Second) {
t.Fatal("single mark below maxFails=2 should not be failed")
}
// Per-item metadata override beats the given defaults.
item2 := newTestMarkable(map[string]any{"maxFails": 1, "failTimeout": "60s"})
item2.Marker().Mark()
if !IsFailed(item2, 5, 10*time.Second) {
t.Fatal("metadata maxFails override should take precedence")
}
// Non-markable items are never failed.
if IsFailed(42, 1, time.Second) {
t.Fatal("non-markable item should not be failed")
}
}
+42 -7
View File
@@ -26,6 +26,8 @@ import (
"github.com/go-gost/core/service"
xctx "github.com/go-gost/x/ctx"
xlogger "github.com/go-gost/x/logger"
xbypass "github.com/go-gost/x/bypass"
rate_limiter "github.com/go-gost/x/limiter/rate"
xmetrics "github.com/go-gost/x/metrics"
xstats "github.com/go-gost/x/observer/stats"
"github.com/google/shlex"
@@ -44,6 +46,7 @@ type options struct {
observerPeriod time.Duration
logger logger.Logger
labels map[string]string
closers []io.Closer
}
// Option is a functional option for configuring a service.
@@ -122,6 +125,13 @@ func LabelsOption(labels map[string]string) Option {
}
}
// ClosersOption registers io.Closers that are called on service shutdown.
func ClosersOption(closers ...io.Closer) Option {
return func(opts *options) {
opts.closers = append(opts.closers, closers...)
}
}
// LoggerOption sets the logger for the service. If not provided, a no-op
// logger is used.
func LoggerOption(logger logger.Logger) Option {
@@ -327,13 +337,17 @@ func (s *defaultService) Serve() error {
}
if err := s.handler.Handle(ctx, conn); err != nil {
log.Error(err)
if v := xmetrics.GetCounter(xmetrics.MetricServiceHandlerErrorsCounter,
metrics.Labels{"service": s.name, "client": clientIP}); v != nil {
v.Inc()
}
if sts := s.status.stats; sts != nil {
sts.Add(stats.KindTotalErrs, 1)
if isServerError(err) {
log.Error(err)
if v := xmetrics.GetCounter(xmetrics.MetricServiceHandlerErrorsCounter,
metrics.Labels{"service": s.name, "client": clientIP}); v != nil {
v.Inc()
}
if sts := s.status.stats; sts != nil {
sts.Add(stats.KindTotalErrs, 1)
}
} else {
log.Debug(err)
}
}
}()
@@ -364,6 +378,11 @@ func (s *defaultService) Close() error {
}
}
}
for _, c := range s.options.closers {
if err := c.Close(); err != nil {
errs = append(errs, err)
}
}
if err := s.listener.Close(); err != nil {
errs = append(errs, err)
}
@@ -471,6 +490,22 @@ func (s *defaultService) observeStats(ctx context.Context) {
}
}
// isServerError reports whether err represents a server-side or infrastructure
// failure that should be counted as a handler error. Client-side disconnects
// (io.EOF, net.ErrClosed, context cancellation) and policy decisions (rate
// limiting, bypass routing) are not server errors.
func isServerError(err error) bool {
if errors.Is(err, io.EOF) ||
errors.Is(err, net.ErrClosed) ||
errors.Is(err, context.Canceled) ||
errors.Is(err, context.DeadlineExceeded) ||
errors.Is(err, xbypass.ErrBypass) ||
errors.Is(err, rate_limiter.ErrRateLimit) {
return false
}
return true
}
// ServiceEvent is an observer event representing a service state change.
type ServiceEvent struct {
Kind string
+22 -1
View File
@@ -585,13 +585,34 @@ func TestCloseJoinsErrors(t *testing.T) {
type closeErrHandler struct {
err error
}
func (h *closeErrHandler) Init(metadata.Metadata) error { return nil }
func (h *closeErrHandler) Handle(context.Context, net.Conn, ...handler.HandleOption) error {
return nil
}
func (h *closeErrHandler) Close() error { return h.err }
type mockCloser struct {
closed atomic.Bool
}
func (c *mockCloser) Close() error {
c.closed.Store(true)
return nil
}
func TestCloseClosesClosers(t *testing.T) {
ln := newMockListener()
c1, c2 := &mockCloser{}, &mockCloser{}
svc := NewService("test", ln, newMockHandler(nil), ClosersOption(c1, c2)).(*defaultService)
if err := svc.Close(); err != nil {
t.Errorf("Close() = %v", err)
}
if !c1.closed.Load() || !c2.closed.Load() {
t.Error("closers were not closed")
}
}
// --- Status tests ---
func TestStatusReturnsStatus(t *testing.T) {