Compare commits

...
20 Commits
Author SHA1 Message Date
ginuerzh 93bd2ea760 fix(hop): parenthesize host OR and normalize wildcard in filter->matcher rule
filterToMatcherRule produced `Host(x) || Host(.x) && Proto(...) && ...`,
but the matcher DSL (vulcand/predicate) binds && tighter than ||, so the
apex Host(x) branch bypassed the protocol/path conjuncts — a request to
the apex with a non-matching protocol/path matched where the legacy
filter (AND of host/protocol/path) rejected it.

Also normalize `*.example.com` to `.example.com` before the leading-dot
branch: legacy ParseNode did this so both forms matched apex+subdomains;
without it, `*.example.com` reached Host() as a wildcard matching only
subdomains, dropping apex coverage that `.example.com` still had.

- Parenthesize the host OR when protocol/path conjuncts follow.
- Strip leading `*` + ensure leading dot, mirroring legacy normalization.
- Update TestFilterToMatcherRule expectations; add dot/star combined cases.
- Add TestFilterToMatcherRule_Precedence: runtime guard that apex+wrong
  proto/path is rejected and apex/subdomain+valid is accepted, for both
  `.example.com` and `*.example.com` forms.
2026-08-26 21:02:50 +08:00
ginuerzh 9292e0af3e fix(tunnel, forward): route tunneled hostname through hop filter.host
The tunnel bind conns (bindConn/bindUDPConn) now expose their metadata (the
tunneled hostname) via the xctx.Context interface, so the service accept loop
forwards it into the handler context. handleRawForwarding selects a hop node by
that hostname when a hop is configured, so a node with a matching filter.host is
picked; otherwise it falls back to dialing the hostname directly. Fixes
reverse-proxy-tunnel TCP routing (gost#898) where the docs example failed with
"node not available" because the mux path never set the host metadata.
2026-08-26 20:28:37 +08:00
ginuerzh 6ad7269644 refactor(hop): normalize deprecated node filter into matcher at parse time
Fold the deprecated per-node filter (host/protocol/path) into an equivalent
matcher DSL rule during config parsing, so hop.Select keeps a single
filtering path (routing.Matcher). Legacy filter configs stay accepted.

- filterToMatcherRule() maps host->Host(), protocol->Proto(), path->PathPrefix();
  a leading-dot host emits `Host(x) || Host(.x)` to preserve the legacy
  apex+subdomain coverage. Converted rules get priority=-1 to keep the
  always-through-selector behavior of the legacy path.
- Remove isEligible/checkHost/checkProtocol/checkPath from x/hop.
  core/ chain.NodeFilterSettings is left untouched (dead but harmless).
- Migrate hop tests to matchers; add TestFilterToMatcherRule.
2026-08-26 20:26:26 +08:00
ginuerzh 4ea4efaba9 fix(forwarder, sniffing): assert BasicProxy recording via captured output
Since 486a7766 HandleHTTP records on an internal clone of the passed
recorder object, TestHandleHTTP_BasicProxy's assertions on the original
ro were always empty, failing every run. Assert SrcAddr/DstAddr on the
captured recorder payload instead, and drain the response body before
closing the net.Pipe so HandleHTTP completes the write and exits cleanly.
2026-08-25 22:17:05 +08:00
ginuerzh faeede7881 feat(forwarder): delete JSON body fields via json-: rewrite match
Add a json-: match prefix to http.rewriteRequestBody/rewriteResponseBody
that deletes the matched JSON field (sjson.DeleteBytes) instead of
replacing it, keeping the existing value-regex conditional semantics.
2026-08-25 21:54:27 +08:00
ginuerzh a38a701dc4 refactor(sniffing): register protocol handlers via init() registries
Replace the 5 duplicated switch statements that dispatch sniffed
connections (forward/local, forward/remote, http CONNECT, sni, relay)
with per-package registries keyed by proto string. Protocol handlers
self-register via init(), following the house registration pattern, so
adding a protocol no longer requires touching every dispatch site.

Unify the two HandleOptions types (forwarder lacked dialTLS; sniffing
lacked hop/node/httpKeepalive) into one exported struct in sniffing;
forwarder aliases it. Unknown protocols fall through to raw forwarding
exactly as before; SSH remains detection-only. redirect/tcp's special
fallback-replay dispatch is intentionally left untouched.

Adds Dispatch helpers and 8 registry tests (register/get/duplicate/
unknown/known) in both packages. Net -151 lines.
2026-08-25 21:35:45 +08:00
ginuerzh 0132cefaec fix(api): save config back to the loaded file path (gost/x#123)
POST /config without an explicit path= always wrote gost.yaml/gost.json to
the process CWD, silently leaving the config loaded via -C untouched. Track
the last file read by ReadFile and default the save target to it, deriving
the output format from the file extension so a JSON config is saved back as
JSON. Falls back to the CWD default when no config file was loaded.
2026-08-21 21:43:17 +08:00
ginuerzh 25cd60d42d fix(connector/ss): own the target addr instead of aliasing the pooled buffer (gost#896)
WrapConn stores the target and only writes it on first I/O, but the
connector encoded it into a bufpool buffer that was returned to the pool
when Connect returned. A later Connect could reuse the backing array and
overwrite the bytes, misrouting the first connection. Allocate the target
per connection instead of sharing a pooled buffer.

Regression test drives two Connect calls and asserts the first conn's
wire target survives a second Connect reusing the pool.
2026-08-21 21:42:43 +08:00
ginuerzh f8ca4908f3 fix(hop): skip failed nodes before priority short-circuit, last-resort fallback
Failed nodes (http.failCodes marker) were still selected when the
priority short-circuit fired or when FailFilter saw a single-node tier,
so failover never happened for a high-priority matcher node returning
429. Check IsFailed first in the candidate-pool scan, and when every
candidate is failed return the first node matching the request gates as
a best-effort last resort instead of nil.

Refactors the fail-code check into shared selector.IsFailed (cee264e)
wired through FailFilterSettingsOption (64ea518).
2026-08-18 23:42:08 +08:00
ginuerzh cee264e184 refactor(selector): share IsFailed dead-node check with FailFilter
Extract the per-item failed-state decision (failure count >= maxFails,
last failure within failTimeout, metadata overrides) from FailFilter
into an exported IsFailed helper, and reuse it in the hop's
pre-selection dead-node filter instead of a local duplicate.
2026-08-18 23:21:34 +08:00
ginuerzh 64ea5184f8 fix(hop): drop marked-failed nodes before priority selection
http.failCodes (and probe) marking had no effect when the failing node
had a strictly higher matcher priority: the priority short-circuit
returned the top node directly, and FailFilter no-ops on a single-node
tier (len(vs) <= 1 guard), so the marked node was always re-selected
and failover never happened.

Pre-filter nodes marked failed (same Count >= maxFails && within
failTimeout criteria as FailFilter) out of the candidate pool before
priority selection. Single-node hops keep their lone node (safety net),
and an all-failed pool returns nil as before.

Wired maxFails/failTimeout from the hop selector config into the hop
via FailFilterSettingsOption.
2026-08-18 23:15:59 +08:00
ginuerzh fc5fc052e4 chore(deps): bump go-shadowsocks2 to v0.1.4
Pulls in the classic-AEAD UDP [addr][payload] response-framing fix that
restores shadowsocks UDP interop with third-party servers/clients
(go-shadowsocks2, shadowsocks-libev). Fixes go-gost/gost#894.
2026-08-18 22:17:20 +08:00
ginuerzh e6aeba8701 fix(listener/redirect/udp): set SO_REUSEADDR on tproxy listener socket (gost#413)
Match the dialUDP socket so per-packet tproxy sockets can bind a
destination port overlapping this listener without "address already in use".
2026-08-18 21:55:12 +08:00
ginuerzh cc02cff00c fix(logger): deduplicate keys across WithFields chains (gost#895)
slog.Logger.With does not deduplicate keys, so chained WithFields calls
(e.g. service -> listener/handler, each setting "kind") emitted duplicate
JSON keys. slogLogger now rebuilds from a pristine base logger with
accumulated attrs deduplicated by key, last write wins.
2026-08-18 21:49:11 +08:00
ginuerzh 18df26e9bc Merge branch 'master' of github.com:go-gost/x 2026-08-17 23:05:14 +08:00
Vincent Yang a9be7e6dce fix(masque): repair server CONNECT-UDP relay (#122)
The MASQUE handler's CONNECT-UDP path was broken in two ways, so no
standards-compliant client (e.g. Surge) could establish UDP tunnels:

1. Missing Capsule-Protocol response header. HTTPStream() flushes the
   response internally and commits the headers, but it was called before
   Header().Set("Capsule-Protocol", "?1"), so the mandatory RFC 9298
   header was silently dropped and clients rejected the tunnel.

2. WriteTo on a pre-connected socket. A direct UDP dial via the router
   returns a *net.UDPConn that satisfies net.PacketConn but is
   pre-connected; the relay called WriteTo on it, which fails with "use
   of WriteTo with pre-connected connection", so no packet reached the
   target.

Set the Capsule-Protocol header before hijacking the stream (dialing the
target first, matching handleConnectTCP), and always wrap the
router-returned conn so writes go through Conn.Write. Verified with a
standards-compliant HTTP/3 MASQUE client: CONNECT-UDP now returns
Capsule-Protocol: ?1 and datagrams relay end to end; CONNECT-TCP
unaffected.
2026-08-17 23:05:00 +08:00
ginuerzhandgithub 0e0eab0c52 fix(cmd): restore unix socket port-forward semantics
Commit b3b5986 crammed the URL path into the listen/connect address and
disabled forward mode for path-based protocols, breaking the
`unix://a.sock/b.sock` (listen on a, forward to b) port-forward form and
producing an invalid `a.sockb.sock` address.

Distinguish the two URL forms by host presence:
- three-slash (unix:///abs.sock, empty host) → path is the absolute socket
  address, no forward target;
- two-slash (unix://a.sock/b.sock) → host is the socket and the path is a
  forward target (service) or ignored (node).

Add regression tests for both forms.

Co-Authored-By: Claude (claude-opus-4-8[1m])
2026-08-17 22:51:00 +08:00
Dean Chen 98e9651361 cmd: keep absolute unix socket paths for path-based schemes (#121)
Path-based address selection used the dialer/listener type after registry
fallback, so unix remapped to tcp skipped EscapedPath and dropped
addrs like /var/run/docker.sock from three-slash URLs.

Decide path-based protocols from the original scheme components.

Signed-off-by: Dean Chen <862469039@qq.com>
2026-08-17 22:29:17 +08:00
PahaKush e4e36ca095 x: stream websocket reads instead of buffering each frame (#120)
websocketConn.Read called Conn.ReadMessage, which is NextReader plus
io.ReadAll. io.ReadAll starts at 512 bytes and grows by doubling, so every
frame allocated a fresh slice and a chain of intermediate ones: a stream of
64KB frames costs about 2.5 bytes of garbage per byte carried, even though
the caller already supplied a buffer to copy into.

On a relay+wss tunnel moving 750MB this was 2526KB of garbage per MB of
payload and 757 GC cycles in 15 seconds against a 1.5MB live heap. An
allocation profile attributed 99.92% of all process garbage to io.ReadAll
under ReadMessage; a CPU profile spent 15% in memclrNoHeapPointers and ~21%
in the collector, against 8% in AES-GCM.

Use NextReader and copy straight into the caller's buffer. The reader is
kept across calls and dropped at end of message, so a message larger than
the caller's buffer is still delivered across successive reads.

This also fixes a zero-length message surfacing as a (0, nil) read, which
io.Copy counts as no progress and rejects after 100 in a row.

Benchmark on 64KB frames:

    before  39267 ns/op  1668.97 MB/s  138444 B/op  19 allocs/op
    after   19920 ns/op  3289.98 MB/s      80 B/op   3 allocs/op
2026-08-17 22:28:48 +08:00
Niek van der Maas 746811eea5 fix: support SOCKS5 UDP associations over MASQUE (#119)
* fix: support SOCKS5 UDP over MASQUE

* fix: complete MASQUE SOCKS5 proxy support

* fix: isolate MASQUE UDP destinations
2026-08-17 22:28:20 +08:00
52 changed files with 2431 additions and 980 deletions
+29 -11
View File
@@ -5,6 +5,8 @@ import (
"fmt"
"net/http"
"os"
"path/filepath"
"strings"
"github.com/gin-gonic/gin"
"github.com/go-gost/core/observer/stats"
@@ -127,10 +129,11 @@ func getConfig(ctx *gin.Context) {
// swagger:parameters saveConfigRequest
type saveConfigRequest struct {
// output format, one of yaml|json, default is yaml.
// output format, one of yaml|json, default is derived from the file extension.
// in: query
Format string `form:"format" json:"format"`
// file path, default is gost.yaml|gost.json in current working directory.
// file path, default is the loaded config file, or gost.yaml|gost.json in
// the current working directory when no config file was loaded.
// in: query
Path string `form:"path" json:"path"`
}
@@ -155,16 +158,31 @@ func saveConfig(ctx *gin.Context) {
var req saveConfigRequest
ctx.ShouldBindQuery(&req)
file := "gost.yaml"
switch req.Format {
case "json":
file = "gost.json"
default:
req.Format = "yaml"
file := req.Path
if file == "" {
// Default to the config file that was loaded (via -C), falling back to
// the current working directory when the config came from elsewhere.
file = config.ConfigFile()
}
if req.Path != "" {
file = req.Path
if file == "" {
file = "gost.yaml"
if req.Format == "json" {
file = "gost.json"
}
}
switch strings.ToLower(req.Format) {
case "json":
req.Format = "json"
case "yaml":
req.Format = "yaml"
default:
// Derive the format from the file extension so a loaded JSON config
// is saved back as JSON (ReadFile forces format from extension).
if strings.ToLower(filepath.Ext(file)) == ".json" {
req.Format = "json"
} else {
req.Format = "yaml"
}
}
f, err := os.Create(file)
+38 -21
View File
@@ -397,15 +397,24 @@ func buildServiceConfig(url *url.URL) ([]*config.ServiceConfig, error) {
listener = schemes[1]
}
// For path-based protocols (Unix socket listeners and serial), use path as address
// Decide path-based from the scheme before registry remaps listener to tcp.
isPathBasedProtocol := listener == "unix" || listener == "runix" || listener == "serial"
for _, s := range schemes {
if s == "unix" || s == "runix" || s == "serial" {
isPathBasedProtocol = true
break
}
}
// A path-based protocol with an empty host (three-slash URL like
// unix:///var/run/sock) uses the path as the absolute listen address and
// has no forward target. With a non-empty host (two-slash URL like
// unix://sock/other.sock), the host is the listen address and any path is a
// forward target (port-forward to another socket), handled below.
isPathAddr := isPathBasedProtocol && url.Host == ""
var addrs []string
if isPathBasedProtocol {
path := url.EscapedPath()
if url.Host != "" {
addrs = append(addrs, url.Host+strings.TrimPrefix(path, "/"))
} else if path != "" {
if isPathAddr {
if path := url.EscapedPath(); path != "" {
addrs = append(addrs, path)
} else {
addrs = append(addrs, url.Host)
@@ -436,8 +445,10 @@ func buildServiceConfig(url *url.URL) ([]*config.ServiceConfig, error) {
var nodes []*config.ForwardNodeConfig
// forward mode
// For path-based protocols, path is address not forward target
if remotes := strings.Trim(url.EscapedPath(), "/"); remotes != "" && !isPathBasedProtocol {
// A path-based protocol with an empty host (e.g. unix:///abs.sock) treats
// the path as the listen address, not a forward target. With a host
// (e.g. unix://a.sock/b.sock) the path is a forward target.
if remotes := strings.Trim(url.EscapedPath(), "/"); remotes != "" && !isPathAddr {
i := 0
for _, addr := range strings.Split(remotes, ",") {
addrs := xnet.AddrPortRange(addr).Addrs()
@@ -640,6 +651,18 @@ func buildNodeConfig(url *url.URL, m map[string]any) (*config.NodeConfig, error)
dialer = schemes[1]
}
// Path-based address logic must use the requested scheme, not the
// registry fallback (unix/serial often remap to http/tcp when only
// the CLI registry is partially populated, or in unit tests).
isPathBasedProtocol := dialer == "unix" || dialer == "serial" ||
connector == "unix" || connector == "serial"
for _, s := range schemes {
if s == "unix" || s == "serial" || s == "runix" {
isPathBasedProtocol = true
break
}
}
md := mdx.NewMetadata(m)
if c := registry.ConnectorRegistry().Get(connector); c == nil {
@@ -725,20 +748,14 @@ func buildNodeConfig(url *url.URL, m map[string]any) (*config.NodeConfig, error)
nodeMd[k] = v
}
// For path-based protocols (Unix socket dialers and serial), use path as address
// For path-based protocols (Unix socket dialers and serial), use path as address.
// Three-slash URLs (empty host) give the absolute socket path; two-slash URLs
// use the host as the socket. The path on a host form is not part of the node
// address (it is a forward target for services, ignored for nodes).
var nodeAddr string
isPathBasedProtocol := dialer == "unix" || dialer == "serial"
if isPathBasedProtocol {
path := url.EscapedPath()
if url.Host != "" {
// Two slashes: host + path = relative path
nodeAddr = url.Host + strings.TrimPrefix(path, "/")
} else if path != "" {
// Three slashes: path is absolute (keep leading slash)
nodeAddr = path
} else {
nodeAddr = url.Host
}
if isPathBasedProtocol && url.Host == "" {
// Three slashes: path is absolute (keep leading slash)
nodeAddr = url.EscapedPath()
} else {
nodeAddr = url.Host
}
+62 -6
View File
@@ -1765,10 +1765,10 @@ func TestBuildNodeConfig_TLSWithCert(t *testing.T) {
}
}
func TestBuildNodeConfig_UnixFallsBackToTCP(t *testing.T) {
func TestBuildNodeConfig_UnixAbsolutePath(t *testing.T) {
// http+unix scheme: connector=http, dialer=unix.
// Since unix is not in the test registry, dialer falls back to tcp,
// and the path-based address logic (dialer=="unix") is skipped.
// Dialer falls back to tcp when unix is not in the test registry, but
// the absolute socket path from a three-slash URL must still be kept.
rawURL := "http+unix:///var/run/socket"
u, _ := url.Parse(rawURL)
@@ -1780,12 +1780,68 @@ func TestBuildNodeConfig_UnixFallsBackToTCP(t *testing.T) {
if node.Dialer.Type != "tcp" {
t.Errorf("Dialer.Type = %q, want tcp (unix not in test registry)", node.Dialer.Type)
}
// Addr is url.Host ("") since no host component in three-slash unix URL
if node.Addr != "" {
t.Errorf("Addr = %q, want empty", node.Addr)
if node.Addr != "/var/run/socket" {
t.Errorf("Addr = %q, want /var/run/socket", node.Addr)
}
}
func TestBuildNodeConfig_UnixRelativeHost(t *testing.T) {
u, _ := url.Parse("unix://docker.sock")
node, err := buildNodeConfig(u, map[string]any{})
if err != nil {
t.Fatalf("buildNodeConfig: %v", err)
}
if node.Addr != "docker.sock" {
t.Errorf("Addr = %q, want docker.sock", node.Addr)
}
}
func TestBuildServiceConfig_UnixAbsolutePath(t *testing.T) {
// three-slash: path is the absolute socket listen address, no forward target
svcs, err := buildServiceConfig(mustURL("unix:///var/run/gost.sock"))
if err != nil {
t.Fatalf("buildServiceConfig: %v", err)
}
if len(svcs) != 1 {
t.Fatalf("len(svcs) = %d, want 1", len(svcs))
}
if svcs[0].Addr != "/var/run/gost.sock" {
t.Errorf("Addr = %q, want /var/run/gost.sock", svcs[0].Addr)
}
if svcs[0].Forwarder != nil {
t.Errorf("Forwarder = %+v, want nil (no forward target)", svcs[0].Forwarder)
}
}
func TestBuildServiceConfig_UnixPortForward(t *testing.T) {
// two-slash: host is the listen socket, path is a forward target socket
svcs, err := buildServiceConfig(mustURL("unix://gost.sock/gost2.sock"))
if err != nil {
t.Fatalf("buildServiceConfig: %v", err)
}
if len(svcs) != 1 {
t.Fatalf("len(svcs) = %d, want 1", len(svcs))
}
if svcs[0].Addr != "gost.sock" {
t.Errorf("Addr = %q, want gost.sock", svcs[0].Addr)
}
if svcs[0].Forwarder == nil || len(svcs[0].Forwarder.Nodes) == 0 {
t.Fatalf("Forwarder = %+v, want a forward target node", svcs[0].Forwarder)
}
if got := svcs[0].Forwarder.Nodes[0].Addr; got != "gost2.sock" {
t.Errorf("forward Addr = %q, want gost2.sock", got)
}
}
func mustURL(raw string) *url.URL {
u, err := url.Parse(raw)
if err != nil {
panic(err)
}
return u
}
func TestBuildServiceConfig_RelayTLSForward(t *testing.T) {
// relay+tls scheme: handler=relay, listener=tls.
// relay not in test registry → handler falls back to auto, listener falls back to tcp.
+15
View File
@@ -7,6 +7,7 @@ import (
"reflect"
"strings"
"sync"
"sync/atomic"
"time"
"github.com/mitchellh/mapstructure"
@@ -28,8 +29,21 @@ func init() {
var (
global = &Config{}
globalMux sync.RWMutex
// configFile remembers the last file loaded via ReadFile so the API can
// save back to the same location (issue go-gost/x#123).
configFile atomic.Value // string
)
// ConfigFile returns the path of the last config file loaded via ReadFile,
// or an empty string when the config came from stdin, a URL, or inline JSON.
func ConfigFile() string {
if s, ok := configFile.Load().(string); ok {
return s
}
return ""
}
func Global() *Config {
globalMux.RLock()
defer globalMux.RUnlock()
@@ -780,6 +794,7 @@ func (c *Config) ReadFile(file string) error {
if err := v.ReadInConfig(); err != nil {
return err
}
configFile.Store(file)
return v.Unmarshal(c, viper.DecodeHook(gostDecodeHook()))
}
+12
View File
@@ -15,6 +15,7 @@ import (
selector_parser "github.com/go-gost/x/config/parsing/selector"
xhop "github.com/go-gost/x/hop"
hop_plugin "github.com/go-gost/x/hop/plugin"
xs "github.com/go-gost/x/selector"
"github.com/go-gost/x/internal/loader"
"github.com/go-gost/x/internal/plugin"
"github.com/go-gost/x/metadata"
@@ -158,10 +159,21 @@ func ParseHop(cfg *config.HopConfig, log logger.Logger) (hop.Hop, error) {
sel = selector_parser.DefaultNodeSelector()
}
maxFails, timeout := xs.DefaultMaxFails, xs.DefaultFailTimeout
if cfg.Selector != nil {
if cfg.Selector.MaxFails > 0 {
maxFails = cfg.Selector.MaxFails
}
if cfg.Selector.FailTimeout > 0 {
timeout = cfg.Selector.FailTimeout
}
}
opts := []xhop.Option{
xhop.NameOption(cfg.Name),
xhop.NodeOption(nodes...),
xhop.SelectorOption(sel),
xhop.FailFilterSettingsOption(maxFails, timeout),
xhop.BypassOption(xbypass.BypassGroup(bypass_parser.List(cfg.Bypass, cfg.Bypasses...)...)),
xhop.ReloadPeriodOption(cfg.Reload),
xhop.LoggerOption(log.WithFields(map[string]any{
+100 -47
View File
@@ -36,6 +36,50 @@ const DefaultMatcherBodySize = 1 << 20 // 1MB
// when a node opts in to body matching.
const MaxMatcherBodySize = 10 << 20 // 10MB
// filterToMatcherRule converts a deprecated NodeFilterConfig (host/protocol/
// path) into an equivalent matcher DSL rule. Empty fields are omitted; an
// empty result means the node is unconditional. Path maps to PathPrefix
// (the legacy filter is a prefix match, not exact).
func filterToMatcherRule(filter *config.NodeFilterConfig) string {
if filter == nil {
return ""
}
var parts []string
// Normalize the legacy host forms first: `*.example.com` and
// `.example.com` both meant "apex AND subdomains" in the legacy filter.
// Strip the wildcard so the leading-dot branch below handles both.
host := filter.Host
if strings.HasPrefix(host, "*") {
host = host[1:]
if !strings.HasPrefix(host, ".") {
host = "." + host
}
}
if host != "" {
// A leading-dot host matches the apex AND its subdomains in the
// legacy filter (HasSuffix on the dot-stripped form), but matcher
// Host(.x) only matches subdomains. Emit both to preserve apex
// coverage. Parenthesize the OR when other conjuncts follow, since
// && binds tighter than || in the matcher DSL.
if strings.HasPrefix(host, ".") {
h := "Host(`" + host[1:] + "`) || Host(`" + host + "`)"
if filter.Protocol != "" || filter.Path != "" {
h = "(" + h + ")"
}
parts = append(parts, h)
} else {
parts = append(parts, "Host(`"+host+"`)")
}
}
if prot := filter.Protocol; prot != "" {
parts = append(parts, "Proto(`"+prot+"`)")
}
if path := filter.Path; path != "" {
parts = append(parts, "PathPrefix(`"+path+"`)")
}
return strings.Join(parts, " && ")
}
// ParseNode converts a NodeConfig into a *chain.Node. It resolves the
// connector and dialer from their registries, applies TLS settings, extracts
// metadata-driven options (so_mark, interface, netns, proxy protocol), sets up
@@ -48,8 +92,16 @@ func parseBodyRewrites(vs []config.HTTPBodyRewriteConfig, log logger.Logger) []c
var pattern *regexp.Regexp
var rewriteType string
if js, ok := strings.CutPrefix(v.Match, "json:"); ok {
// json:<path> or json:<path>=<value-regex>
js, ok := strings.CutPrefix(v.Match, "json:")
del := false
if !ok {
// json-:<path> deletes the matched JSON field instead of replacing it.
if js, ok = strings.CutPrefix(v.Match, "json-:"); ok {
del = true
}
}
if ok {
// json:<path>[=<value-regex>] or json-:<path>[=<value-regex>]
path, valRegex, _ := strings.Cut(js, "=")
if valRegex == "" {
valRegex = ".*"
@@ -60,7 +112,11 @@ func parseBodyRewrites(vs []config.HTTPBodyRewriteConfig, log logger.Logger) []c
log.Warnf("invalid JSON value regex %q for path %q: %v", valRegex, path, err)
continue
}
rewriteType = "json:" + path
if del {
rewriteType = "json-:" + path
} else {
rewriteType = "json:" + path
}
} else {
if v.Match != "" {
pattern, _ = regexp.Compile(v.Match)
@@ -209,56 +265,53 @@ func ParseNode(hop string, cfg *config.NodeConfig, log logger.Logger) (*chain.No
chain.NetworkNodeOption(cfg.Network),
}
if filter := cfg.Filter; filter != nil {
// convert *.example.com to .example.com
// convert *example.com to example.com
host := filter.Host
if strings.HasPrefix(host, "*") {
host = host[1:]
if !strings.HasPrefix(host, ".") {
host = "." + host
}
}
var (
rule string
priority int
bodySize int
)
settings := &chain.NodeFilterSettings{
Protocol: filter.Protocol,
Host: host,
Path: filter.Path,
}
opts = append(opts, chain.NodeFilterOption(settings))
// Deprecated filter: normalize into an equivalent matcher DSL rule so the
// runtime keeps a single filtering logic. Explicit -1 priority opts out of
// auto-priority and the priority short-circuit, preserving the legacy
// filter behavior (always through the selector).
if filter := cfg.Filter; filter != nil {
rule = filterToMatcherRule(filter)
priority = -1
bodySize = DefaultMatcherBodySize
}
if cfg.Matcher != nil {
priority := cfg.Matcher.Priority
if rule := strings.TrimSpace(cfg.Matcher.Rule); rule != "" {
if matcher, err := routing.NewMatcher(rule); err == nil {
log.Debugf("new matcher for node %s with rule %s", cfg.Name, cfg.Matcher.Rule)
// Priority 0 means "use default": automatically set to the
// rule length so longer (more specific) rules outrank shorter
// ones. Use a negative priority to opt out of this behavior
// and always go through the selector.
if priority == 0 {
priority = len(cfg.Matcher.Rule)
}
opts = append(opts, chain.MatcherNodeOption(matcher))
} else {
log.Error(err)
priority = -1
}
}
bodySize := cfg.Matcher.BodySize
if bodySize <= 0 {
bodySize = DefaultMatcherBodySize
} else if bodySize > MaxMatcherBodySize {
bodySize = MaxMatcherBodySize
}
opts = append(opts, chain.MatcherBodySizeNodeOption(bodySize))
opts = append(opts, chain.PriorityNodeOption(priority))
priority = cfg.Matcher.Priority
rule = cfg.Matcher.Rule
bodySize = cfg.Matcher.BodySize
}
if rule = strings.TrimSpace(rule); rule != "" {
if matcher, err := routing.NewMatcher(rule); err == nil {
log.Debugf("new matcher for node %s with rule %s", cfg.Name, rule)
// Priority 0 means "use default": automatically set to the
// rule length so longer (more specific) rules outrank shorter
// ones. Use a negative priority to opt out of this behavior
// and always go through the selector.
if priority == 0 {
priority = len(rule)
}
opts = append(opts, chain.MatcherNodeOption(matcher))
} else {
log.Error(err)
priority = -1
}
}
if bodySize <= 0 {
bodySize = DefaultMatcherBodySize
} else if bodySize > MaxMatcherBodySize {
bodySize = MaxMatcherBodySize
}
opts = append(opts, chain.MatcherBodySizeNodeOption(bodySize))
opts = append(opts, chain.PriorityNodeOption(priority))
if cfg.HTTP != nil {
settings := &chain.HTTPNodeSettings{
Host: cfg.HTTP.Host,
+35
View File
@@ -501,3 +501,38 @@ func TestParseNode_DeprecatedFilterFields(t *testing.T) {
t.Fatal("expected non-nil node")
}
}
func TestFilterToMatcherRule(t *testing.T) {
tests := []struct {
name string
filter *config.NodeFilterConfig
want string
}{
{"nil", nil, ""},
{"all empty", &config.NodeFilterConfig{}, ""},
{"host only", &config.NodeFilterConfig{Host: "example.com"}, "Host(`example.com`)"},
{"dot host", &config.NodeFilterConfig{Host: ".example.com"},
"Host(`example.com`) || Host(`.example.com`)"},
{"star host", &config.NodeFilterConfig{Host: "*.example.com"},
"Host(`example.com`) || Host(`.example.com`)"},
{"protocol only", &config.NodeFilterConfig{Protocol: "ssh"}, "Proto(`ssh`)"},
{"path only", &config.NodeFilterConfig{Path: "/api"}, "PathPrefix(`/api`)"},
{"combined", &config.NodeFilterConfig{
Host: ".example.com",
Protocol: "http",
Path: "/api",
}, "(Host(`example.com`) || Host(`.example.com`)) && Proto(`http`) && PathPrefix(`/api`)"},
{"star combined", &config.NodeFilterConfig{
Host: "*.example.com",
Protocol: "http",
Path: "/api",
}, "(Host(`example.com`) || Host(`.example.com`)) && Proto(`http`) && PathPrefix(`/api`)"},
}
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
if got := filterToMatcherRule(tt.filter); got != tt.want {
t.Errorf("filterToMatcherRule(%v) = %q, want %q", tt.filter, got, tt.want)
}
})
}
}
+35
View File
@@ -0,0 +1,35 @@
package node
import (
"testing"
"github.com/go-gost/x/config"
coreRouting "github.com/go-gost/core/routing"
"github.com/go-gost/x/routing"
)
func TestFilterToMatcherRule_Precedence(t *testing.T) {
for _, fc := range []struct{ name, host string }{
{"dot", ".example.com"},
{"star", "*.example.com"},
} {
t.Run(fc.name, func(t *testing.T) {
rule := filterToMatcherRule(&config.NodeFilterConfig{
Host: fc.host, Protocol: "http", Path: "/api",
})
m, err := routing.NewMatcher(rule)
if err != nil {
t.Fatalf("parse %q: %v", rule, err)
}
if m.Match(&coreRouting.Request{Host: "example.com", Protocol: "https", Path: "/other"}) {
t.Errorf("rule %q matched apex+https+/other (should reject)", rule)
}
if !m.Match(&coreRouting.Request{Host: "example.com", Protocol: "http", Path: "/api/v1"}) {
t.Errorf("rule %q rejected apex+http+/api/v1 (should accept)", rule)
}
if !m.Match(&coreRouting.Request{Host: "www.example.com", Protocol: "http", Path: "/api"}) {
t.Errorf("rule %q rejected www+http+/api (should accept)", rule)
}
})
}
}
+328
View File
@@ -0,0 +1,328 @@
package masque
import (
"context"
"errors"
"net"
"os"
"sync"
"time"
"github.com/go-gost/core/logger"
)
const udpAssociationBufferSize = 64 * 1024
var errMissingDestination = errors.New("masque: destination address required")
type udpAssociationDialFunc func(ctx context.Context, addr net.Addr) (net.PacketConn, error)
type udpAssociationResult struct {
data []byte
addr net.Addr
}
type udpAssociationTunnel struct {
conn net.PacketConn
}
type udpAssociationConn struct {
ctx context.Context
cancel context.CancelFunc
localAddr net.Addr
dial udpAssociationDialFunc
closeIdle func() error
log logger.Logger
timeout time.Duration
closed chan struct{}
closeOnce sync.Once
closeErr error
results chan udpAssociationResult
mu sync.Mutex
tunnels map[string]*udpAssociationTunnel
readDeadline time.Time
readDeadlineChanged chan struct{}
}
func newUDPAssociationConn(
ctx context.Context,
localAddr net.Addr,
timeout time.Duration,
dial udpAssociationDialFunc,
closeIdle func() error,
log logger.Logger,
) *udpAssociationConn {
ctx, cancel := context.WithCancel(context.WithoutCancel(ctx))
c := &udpAssociationConn{
ctx: ctx,
cancel: cancel,
localAddr: localAddr,
dial: dial,
closeIdle: closeIdle,
log: log,
timeout: timeout,
closed: make(chan struct{}),
results: make(chan udpAssociationResult, 32),
tunnels: make(map[string]*udpAssociationTunnel),
readDeadlineChanged: make(chan struct{}),
}
return c
}
func (c *udpAssociationConn) ReadFrom(b []byte) (n int, addr net.Addr, err error) {
for {
select {
case <-c.closed:
return 0, nil, net.ErrClosed
default:
}
c.mu.Lock()
deadline := c.readDeadline
deadlineChanged := c.readDeadlineChanged
c.mu.Unlock()
var (
timer *time.Timer
timeout <-chan time.Time
)
if !deadline.IsZero() {
remaining := time.Until(deadline)
if remaining <= 0 {
return 0, nil, os.ErrDeadlineExceeded
}
timer = time.NewTimer(remaining)
timeout = timer.C
}
select {
case result := <-c.results:
if timer != nil {
timer.Stop()
}
return copy(b, result.data), result.addr, nil
case <-deadlineChanged:
if timer != nil {
timer.Stop()
}
continue
case <-timeout:
return 0, nil, os.ErrDeadlineExceeded
case <-c.closed:
if timer != nil {
timer.Stop()
}
return 0, nil, net.ErrClosed
}
}
}
func (c *udpAssociationConn) WriteTo(b []byte, addr net.Addr) (int, error) {
if addr == nil {
return 0, errMissingDestination
}
tunnel, err := c.tunnel(addr)
if err != nil {
if c.isClosed() {
return 0, net.ErrClosed
}
c.logError(addr, err)
return len(b), nil
}
n, err := tunnel.conn.WriteTo(b, addr)
if err == nil {
return n, nil
}
if c.isClosed() {
return 0, net.ErrClosed
}
if c.removeTunnel(addr.String(), tunnel) {
c.logError(addr, err)
}
return len(b), nil
}
func (c *udpAssociationConn) tunnel(addr net.Addr) (*udpAssociationTunnel, error) {
key := addr.String()
c.mu.Lock()
select {
case <-c.closed:
c.mu.Unlock()
return nil, net.ErrClosed
default:
}
if tunnel := c.tunnels[key]; tunnel != nil {
c.mu.Unlock()
return tunnel, nil
}
c.mu.Unlock()
ctx := c.ctx
if c.timeout > 0 {
var cancel context.CancelFunc
ctx, cancel = context.WithTimeout(ctx, c.timeout)
defer cancel()
}
conn, err := c.dial(ctx, addr)
if err != nil {
return nil, err
}
if conn == nil {
return nil, errors.New("masque: nil UDP tunnel")
}
tunnel := &udpAssociationTunnel{conn: conn}
c.mu.Lock()
select {
case <-c.closed:
c.mu.Unlock()
conn.Close()
return nil, net.ErrClosed
default:
}
if current := c.tunnels[key]; current != nil {
c.mu.Unlock()
conn.Close()
return current, nil
}
c.tunnels[key] = tunnel
c.mu.Unlock()
go c.readTunnel(key, tunnel)
return tunnel, nil
}
func (c *udpAssociationConn) readTunnel(key string, tunnel *udpAssociationTunnel) {
buf := make([]byte, udpAssociationBufferSize)
for {
n, addr, err := tunnel.conn.ReadFrom(buf)
if err != nil {
if c.isClosed() {
return
}
if c.removeTunnel(key, tunnel) {
c.logError(key, err)
}
return
}
data := append([]byte(nil), buf[:n]...)
select {
case c.results <- udpAssociationResult{data: data, addr: addr}:
case <-c.closed:
return
}
}
}
func (c *udpAssociationConn) isClosed() bool {
select {
case <-c.closed:
return true
default:
return false
}
}
func (c *udpAssociationConn) logError(addr any, err error) {
if c.log != nil {
c.log.Warnf("masque: UDP tunnel %v: %v", addr, err)
}
}
func (c *udpAssociationConn) removeTunnel(key string, tunnel *udpAssociationTunnel) bool {
c.mu.Lock()
removed := c.tunnels[key] == tunnel
if c.tunnels[key] == tunnel {
delete(c.tunnels, key)
}
c.mu.Unlock()
tunnel.conn.Close()
return removed
}
func (c *udpAssociationConn) Read(b []byte) (int, error) {
n, _, err := c.ReadFrom(b)
return n, err
}
func (c *udpAssociationConn) Write(b []byte) (int, error) {
return 0, errMissingDestination
}
func (c *udpAssociationConn) Close() error {
c.closeOnce.Do(func() {
close(c.closed)
c.cancel()
c.mu.Lock()
close(c.readDeadlineChanged)
tunnels := make([]*udpAssociationTunnel, 0, len(c.tunnels))
for _, tunnel := range c.tunnels {
tunnels = append(tunnels, tunnel)
}
clear(c.tunnels)
c.mu.Unlock()
var errs []error
for _, tunnel := range tunnels {
if err := tunnel.conn.Close(); err != nil {
errs = append(errs, err)
}
}
if c.closeIdle != nil {
if err := c.closeIdle(); err != nil {
errs = append(errs, err)
}
}
c.closeErr = errors.Join(errs...)
})
return c.closeErr
}
func (c *udpAssociationConn) LocalAddr() net.Addr {
return c.localAddr
}
func (c *udpAssociationConn) RemoteAddr() net.Addr {
return &net.UDPAddr{}
}
func (c *udpAssociationConn) SetDeadline(t time.Time) error {
if err := c.SetReadDeadline(t); err != nil {
return err
}
return c.SetWriteDeadline(t)
}
func (c *udpAssociationConn) SetReadDeadline(t time.Time) error {
c.mu.Lock()
defer c.mu.Unlock()
select {
case <-c.closed:
return net.ErrClosed
default:
}
c.readDeadline = t
close(c.readDeadlineChanged)
c.readDeadlineChanged = make(chan struct{})
return nil
}
func (c *udpAssociationConn) SetWriteDeadline(t time.Time) error {
select {
case <-c.closed:
return net.ErrClosed
default:
}
return nil
}
var (
_ net.Conn = (*udpAssociationConn)(nil)
_ net.PacketConn = (*udpAssociationConn)(nil)
)
+321
View File
@@ -0,0 +1,321 @@
package masque
import (
"context"
"errors"
"net"
"sync"
"testing"
"time"
)
func TestUDPAssociationConnRoutesByDestination(t *testing.T) {
type dialCall struct {
addr net.Addr
conn *fakePacketConn
}
var (
mu sync.Mutex
calls []dialCall
)
dial := func(ctx context.Context, addr net.Addr) (net.PacketConn, error) {
conn := newFakePacketConn()
mu.Lock()
calls = append(calls, dialCall{addr: addr, conn: conn})
mu.Unlock()
return conn, nil
}
ctx, cancel := context.WithCancel(context.Background())
conn := newUDPAssociationConn(ctx, &net.UDPAddr{}, 0, dial, nil, nil)
cancel() // The route dial context ends before the UDP association is used.
t.Cleanup(func() { conn.Close() })
addr1 := &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 53}
addr2 := &net.UDPAddr{IP: net.ParseIP("8.8.8.8"), Port: 53}
if _, err := conn.WriteTo([]byte("first"), addr1); err != nil {
t.Fatalf("first WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("second"), addr1); err != nil {
t.Fatalf("second WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("third"), addr2); err != nil {
t.Fatalf("third WriteTo failed: %v", err)
}
mu.Lock()
gotCalls := append([]dialCall(nil), calls...)
mu.Unlock()
if len(gotCalls) != 2 {
t.Fatalf("expected one tunnel per destination, got %d", len(gotCalls))
}
if gotCalls[0].addr.String() != addr1.String() || gotCalls[1].addr.String() != addr2.String() {
t.Fatalf("unexpected tunnel destinations: %s, %s", gotCalls[0].addr, gotCalls[1].addr)
}
if got := gotCalls[0].conn.writtenData(); len(got) != 2 || string(got[0]) != "first" || string(got[1]) != "second" {
t.Fatalf("unexpected writes for first destination: %q", got)
}
if got := gotCalls[1].conn.writtenData(); len(got) != 1 || string(got[0]) != "third" {
t.Fatalf("unexpected writes for second destination: %q", got)
}
replyAddr := &net.UDPAddr{IP: net.ParseIP("127.0.0.1"), Port: 1}
gotCalls[1].conn.queueRead([]byte("reply"), replyAddr)
buf := make([]byte, 32)
n, addr, err := conn.ReadFrom(buf)
if err != nil {
t.Fatalf("ReadFrom failed: %v", err)
}
if string(buf[:n]) != "reply" {
t.Fatalf("unexpected reply: %q", buf[:n])
}
if addr.String() != replyAddr.String() {
t.Fatalf("expected reply from %s, got %s", replyAddr, addr)
}
}
func TestUDPAssociationConnClose(t *testing.T) {
upstream := newFakePacketConn()
idleClosed := 0
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) { return upstream, nil },
func() error {
idleClosed++
return nil
},
nil,
)
if _, err := conn.WriteTo([]byte("request"), &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 53}); err != nil {
t.Fatalf("WriteTo failed: %v", err)
}
if err := conn.Close(); err != nil {
t.Fatalf("Close failed: %v", err)
}
if err := conn.Close(); err != nil {
t.Fatalf("second Close failed: %v", err)
}
if idleClosed != 1 {
t.Fatalf("expected idle stream to be closed once, got %d", idleClosed)
}
if !upstream.isClosed() {
t.Fatal("expected destination tunnel to be closed")
}
if _, err := conn.WriteTo([]byte("request"), &net.UDPAddr{}); !errors.Is(err, net.ErrClosed) {
t.Fatalf("expected net.ErrClosed after Close, got %v", err)
}
}
func TestUDPAssociationConnTunnelFailureIsIsolated(t *testing.T) {
var (
mu sync.Mutex
calls []*fakePacketConn
)
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) {
pc := newFakePacketConn()
mu.Lock()
calls = append(calls, pc)
mu.Unlock()
return pc, nil
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
addr1 := &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443}
addr2 := &net.UDPAddr{IP: net.ParseIP("8.8.8.8"), Port: 443}
if _, err := conn.WriteTo([]byte("first"), addr1); err != nil {
t.Fatalf("first WriteTo failed: %v", err)
}
if _, err := conn.WriteTo([]byte("second"), addr2); err != nil {
t.Fatalf("second WriteTo failed: %v", err)
}
mu.Lock()
first, second := calls[0], calls[1]
mu.Unlock()
first.queueReadError(errors.New("stream reset"))
select {
case <-first.closed:
case <-time.After(time.Second):
t.Fatal("failed tunnel was not removed")
}
second.queueRead([]byte("reply"), addr2)
buf := make([]byte, 32)
n, addr, err := conn.ReadFrom(buf)
if err != nil {
t.Fatalf("healthy tunnel failed after peer reset: %v", err)
}
if string(buf[:n]) != "reply" || addr.String() != addr2.String() {
t.Fatalf("unexpected healthy tunnel reply %q from %v", buf[:n], addr)
}
if _, err := conn.WriteTo([]byte("retry"), addr1); err != nil {
t.Fatalf("retry WriteTo failed: %v", err)
}
mu.Lock()
callCount := len(calls)
mu.Unlock()
if callCount != 3 {
t.Fatalf("expected failed destination to redial, got %d dials", callCount)
}
}
func TestUDPAssociationConnDialTimeout(t *testing.T) {
dialErr := make(chan error, 1)
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
20*time.Millisecond,
func(ctx context.Context, _ net.Addr) (net.PacketConn, error) {
<-ctx.Done()
dialErr <- ctx.Err()
return nil, ctx.Err()
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
payload := []byte("request")
n, err := conn.WriteTo(payload, &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443})
if err != nil || n != len(payload) {
t.Fatalf("timed out datagram was not dropped cleanly: n=%d err=%v", n, err)
}
if err := <-dialErr; !errors.Is(err, context.DeadlineExceeded) {
t.Fatalf("expected dial deadline, got %v", err)
}
}
func TestUDPAssociationConnDialDoesNotHoldAssociationLock(t *testing.T) {
started := make(chan struct{})
release := make(chan struct{})
conn := newUDPAssociationConn(
context.Background(),
&net.UDPAddr{},
0,
func(context.Context, net.Addr) (net.PacketConn, error) {
close(started)
<-release
return newFakePacketConn(), nil
},
nil,
nil,
)
t.Cleanup(func() { conn.Close() })
writeDone := make(chan struct{})
go func() {
conn.WriteTo([]byte("request"), &net.UDPAddr{IP: net.ParseIP("1.1.1.1"), Port: 443})
close(writeDone)
}()
<-started
deadlineDone := make(chan error, 1)
go func() {
deadlineDone <- conn.SetReadDeadline(time.Now().Add(time.Second))
}()
select {
case err := <-deadlineDone:
if err != nil {
t.Fatalf("SetReadDeadline failed: %v", err)
}
case <-time.After(time.Second):
t.Fatal("dial held the association lock")
}
close(release)
<-writeDone
}
type fakePacket struct {
data []byte
addr net.Addr
err error
}
type fakePacketConn struct {
reads chan fakePacket
closed chan struct{}
closeOnce sync.Once
mu sync.Mutex
writes [][]byte
}
func newFakePacketConn() *fakePacketConn {
return &fakePacketConn{
reads: make(chan fakePacket, 1),
closed: make(chan struct{}),
}
}
func (c *fakePacketConn) queueRead(data []byte, addr net.Addr) {
c.reads <- fakePacket{data: append([]byte(nil), data...), addr: addr}
}
func (c *fakePacketConn) queueReadError(err error) {
c.reads <- fakePacket{err: err}
}
func (c *fakePacketConn) writtenData() [][]byte {
c.mu.Lock()
defer c.mu.Unlock()
result := make([][]byte, len(c.writes))
for i, data := range c.writes {
result[i] = append([]byte(nil), data...)
}
return result
}
func (c *fakePacketConn) isClosed() bool {
select {
case <-c.closed:
return true
default:
return false
}
}
func (c *fakePacketConn) ReadFrom(b []byte) (int, net.Addr, error) {
select {
case packet := <-c.reads:
if packet.err != nil {
return 0, nil, packet.err
}
return copy(b, packet.data), packet.addr, nil
case <-c.closed:
return 0, nil, net.ErrClosed
}
}
func (c *fakePacketConn) WriteTo(b []byte, _ net.Addr) (int, error) {
select {
case <-c.closed:
return 0, net.ErrClosed
default:
}
c.mu.Lock()
c.writes = append(c.writes, append([]byte(nil), b...))
c.mu.Unlock()
return len(b), nil
}
func (c *fakePacketConn) Close() error {
c.closeOnce.Do(func() { close(c.closed) })
return nil
}
func (c *fakePacketConn) LocalAddr() net.Addr { return &net.UDPAddr{} }
func (c *fakePacketConn) SetDeadline(time.Time) error { return nil }
func (c *fakePacketConn) SetReadDeadline(time.Time) error { return nil }
func (c *fakePacketConn) SetWriteDeadline(time.Time) error { return nil }
+82 -15
View File
@@ -10,6 +10,7 @@ import (
"net/url"
"strconv"
"strings"
"sync"
"time"
"github.com/go-gost/core/connector"
@@ -19,6 +20,7 @@ import (
masque_dialer "github.com/go-gost/x/dialer/http3/masque"
masque_util "github.com/go-gost/x/internal/util/masque"
"github.com/go-gost/x/registry"
"github.com/quic-go/quic-go/http3"
)
func init() {
@@ -73,6 +75,16 @@ func (c *masqueConnector) Connect(ctx context.Context, conn net.Conn, network, a
return nil, fmt.Errorf("%w: %s", ErrUnsupportedNetwork, network)
}
func newTCPConnectRequest(address string) *http.Request {
return &http.Request{
Method: http.MethodConnect,
URL: &url.URL{},
Host: address,
Header: http.Header{},
ProtoMajor: 3,
}
}
func (c *masqueConnector) connectTCP(ctx context.Context, conn net.Conn, address string, log logger.Logger) (net.Conn, error) {
log.Debugf("connect-tcp %s", address)
@@ -103,13 +115,7 @@ func (c *masqueConnector) connectTCP(ctx context.Context, conn net.Conn, address
// Create standard HTTP/3 CONNECT request (RFC 9114)
// No :protocol pseudo-header for standard CONNECT
req := &http.Request{
Method: http.MethodConnect,
URL: &url.URL{},
Host: address, // Target address goes in :authority
Header: http.Header{},
Proto: "HTTP/3.0",
}
req := newTCPConnectRequest(address)
// Add proxy authentication if configured
if c.options.Auth != nil {
@@ -172,9 +178,69 @@ func (c *masqueConnector) connectUDP(ctx context.Context, conn net.Conn, address
return nil, ErrInvalidConnection
}
// Get pre-opened stream from dialer (stream opening happens there for dead connection detection)
reqStream := masqueConn.GetRequestStream()
proxyHost := masqueConn.GetHost()
if address == "" {
return c.connectUDPAssociation(ctx, masqueConn, log), nil
}
return c.connectUDPTarget(
ctx,
conn.LocalAddr(),
masqueConn.GetHost(),
masqueConn.GetRequestStream(),
address,
nil,
log,
)
}
func (c *masqueConnector) connectUDPAssociation(ctx context.Context, conn *masque_dialer.MasqueConn, log logger.Logger) net.Conn {
var (
mu sync.Mutex
firstStream = conn.GetRequestStream()
)
nextStream := func(ctx context.Context) (*http3.RequestStream, error) {
mu.Lock()
defer mu.Unlock()
if firstStream != nil {
stream := firstStream
firstStream = nil
return stream, nil
}
return conn.OpenRequestStream(ctx)
}
closeIdle := func() error {
mu.Lock()
defer mu.Unlock()
if firstStream == nil {
return nil
}
err := firstStream.Close()
firstStream = nil
return err
}
dial := func(ctx context.Context, addr net.Addr) (net.PacketConn, error) {
stream, err := nextStream(ctx)
if err != nil {
return nil, err
}
return c.connectUDPTarget(ctx, conn.LocalAddr(), conn.GetHost(), stream, addr.String(), addr, log)
}
return newUDPAssociationConn(ctx, conn.LocalAddr(), c.md.connectTimeout, dial, closeIdle, log)
}
func (c *masqueConnector) connectUDPTarget(
ctx context.Context,
localAddr net.Addr,
proxyHost string,
reqStream *http3.RequestStream,
address string,
remoteAddr net.Addr,
log logger.Logger,
) (*masque_util.DatagramConn, error) {
// Apply connect timeout to the actual stream
if c.md.connectTimeout > 0 {
@@ -259,14 +325,15 @@ func (c *masqueConnector) connectUDP(ctx context.Context, conn net.Conn, address
// Get the underlying HTTP/3 stream for datagrams
stream := reqStream
// Resolve target address
raddr, err := net.ResolveUDPAddr("udp", address)
if err != nil {
return nil, err
if remoteAddr == nil {
remoteAddr, err = net.ResolveUDPAddr("udp", address)
if err != nil {
return nil, err
}
}
// Create datagram connection wrapping the request stream
datagramConn := masque_util.NewDatagramConnFromRequestStream(stream, conn.LocalAddr(), raddr)
datagramConn := masque_util.NewDatagramConnFromRequestStream(stream, localAddr, remoteAddr)
success = true // Prevent defer from closing stream - datagramConn now owns it
return datagramConn, nil
+24
View File
@@ -0,0 +1,24 @@
package masque
import (
"net/http"
"testing"
)
func TestNewTCPConnectRequestUsesStandardConnect(t *testing.T) {
req := newTCPConnectRequest("example.com:443")
if req.Method != http.MethodConnect {
t.Fatalf("expected CONNECT method, got %s", req.Method)
}
if req.Host != "example.com:443" {
t.Fatalf("expected target authority, got %s", req.Host)
}
if req.Proto != "" {
// quic-go treats any non-empty Proto as an extended CONNECT :protocol.
t.Fatalf("standard CONNECT must not set :protocol, got %q", req.Proto)
}
if req.ProtoMajor != 3 {
t.Fatalf("expected HTTP/3 request, got HTTP/%d", req.ProtoMajor)
}
}
+7 -6
View File
@@ -9,7 +9,6 @@ import (
"strings"
"github.com/go-gost/core/common/bufpool"
"github.com/go-gost/core/connector"
md "github.com/go-gost/core/metadata"
"github.com/go-gost/go-shadowsocks2/core"
@@ -97,10 +96,13 @@ func (c *ssConnector) Connect(ctx context.Context, conn net.Conn, network, addre
log.Error(err)
return nil, err
}
rawaddr := bufpool.Get(512)
defer bufpool.Put(rawaddr)
// The wrapped conn stores this target and only writes it on first I/O,
// so it must be owned by this connection, not a shared pooled buffer
// (see go-gost/gost#896).
const maxAddrLen = 1 + 1 + 255 + 2 // SOCKS5 addr: atyp + len + host + port
n, err := addr.Encode(rawaddr)
target := make([]byte, maxAddrLen)
n, err := addr.Encode(target)
if err != nil {
log.Error("encoding addr: ", err)
return nil, err
@@ -111,8 +113,7 @@ func (c *ssConnector) Connect(ctx context.Context, conn net.Conn, network, addre
defer conn.SetDeadline(time.Time{})
}
target := socks.Addr(rawaddr[:n])
conn, err = c.client.WrapConn(conn, target)
conn, err = c.client.WrapConn(conn, socks.Addr(target[:n]))
if err != nil {
return nil, err
}
+81
View File
@@ -0,0 +1,81 @@
package ss
import (
"bytes"
"context"
"io"
"net"
"net/url"
"testing"
"time"
"github.com/go-gost/core/connector"
"github.com/go-gost/go-shadowsocks2/socks"
"github.com/go-gost/x/internal/util/ss/none"
xlogger "github.com/go-gost/x/logger"
mdx "github.com/go-gost/x/metadata"
)
// captureConn records everything written to it, so a test can inspect the
// bytes a connector actually puts on the wire.
type captureConn struct {
buf bytes.Buffer
}
func (c *captureConn) Read(b []byte) (int, error) { return 0, io.EOF }
func (c *captureConn) Write(b []byte) (int, error) { return c.buf.Write(b) }
func (c *captureConn) Close() error { return nil }
func (c *captureConn) LocalAddr() net.Addr { return &net.TCPAddr{} }
func (c *captureConn) RemoteAddr() net.Addr { return &net.TCPAddr{} }
func (c *captureConn) SetDeadline(time.Time) error { return nil }
func (c *captureConn) SetReadDeadline(time.Time) error { return nil }
func (c *captureConn) SetWriteDeadline(time.Time) error { return nil }
// TestConnectTargetNotOverwrittenByPool is a regression test for
// https://github.com/go-gost/gost/issues/896: the connector previously passed
// a slice aliasing the bufpool buffer to WrapConn, and the buffer was returned
// to the pool before the wrapped conn lazily wrote the target on first I/O. A
// second Connect could then overwrite the bytes and the first conn would send
// the second conn's target. The none cipher keeps the wire plaintext so the
// written target is directly assertable.
func TestConnectTargetNotOverwrittenByPool(t *testing.T) {
c := NewConnector(
connector.AuthOption(url.UserPassword("none", "x")),
connector.LoggerOption(xlogger.NewLogger()),
)
if err := c.Init(mdx.NewMetadata(nil)); err != nil {
t.Fatalf("Init: %v", err)
}
ctx := context.Background()
cap1 := &captureConn{}
conn1, err := c.Connect(ctx, cap1, "tcp", "1.2.3.4:80")
if err != nil {
t.Fatalf("first Connect: %v", err)
}
// Second Connect reuses the same 512-byte bufpool buffer that conn1's
// target used to alias (single pool tier, same goroutine).
if _, err := c.Connect(ctx, &captureConn{}, "tcp", "5.6.7.8:443"); err != nil {
t.Fatalf("second Connect: %v", err)
}
if _, err := conn1.Write([]byte("X")); err != nil {
t.Fatalf("first Write: %v", err)
}
// initWriter frames the target as [salt][2-byte length][addr].
data := cap1.buf.Bytes()
payload := data[none.Cipher.SaltSize():]
if len(payload) < 2 {
t.Fatalf("captured payload too short: %d bytes: %x", len(payload), data)
}
n := int(payload[0])<<8 | int(payload[1])
if len(payload) < 2+n {
t.Fatalf("captured payload too short: len=%d, need %d: %x", len(payload), 2+n, data)
}
addr := socks.SplitAddr(payload[2 : 2+n])
if got, want := addr.String(), "1.2.3.4:80"; got != want {
t.Fatalf("first conn wrote target %q, want %q (captured %x)", got, want, data)
}
}
+16
View File
@@ -1,6 +1,7 @@
package tunnel
import (
"context"
"encoding/binary"
"errors"
"fmt"
@@ -11,6 +12,7 @@ import (
"github.com/go-gost/core/common/bufpool"
mdata "github.com/go-gost/core/metadata"
"github.com/go-gost/relay"
ictx "github.com/go-gost/x/internal/ctx"
xrelay "github.com/go-gost/x/internal/util/relay"
)
@@ -94,6 +96,14 @@ func (c *bindConn) Metadata() mdata.Metadata {
return c.md
}
// Context implements xctx.Context, exposing the connection metadata (e.g. the
// tunneled hostname) to the handler's context. The service accept loop derives
// the handler context from a conn that implements this interface, which lets
// reverse-forward handlers (rtcp) route on the hostname carried by the tunnel.
func (c *bindConn) Context() context.Context {
return ictx.ContextWithMetadata(context.Background(), c.md)
}
type bindUDPConn struct {
net.Conn
localAddr net.Addr
@@ -162,6 +172,12 @@ func (c *bindUDPConn) Metadata() mdata.Metadata {
return c.md
}
// Context implements xctx.Context, exposing the connection metadata (e.g. the
// tunneled hostname) to the handler's context. See bindConn.Context.
func (c *bindUDPConn) Context() context.Context {
return ictx.ContextWithMetadata(context.Background(), c.md)
}
type bindAddr struct {
network string
addr string
+45
View File
@@ -0,0 +1,45 @@
package tunnel
import (
"net"
"testing"
mdata "github.com/go-gost/core/metadata"
ictx "github.com/go-gost/x/internal/ctx"
xmd "github.com/go-gost/x/metadata"
mdutil "github.com/go-gost/x/metadata/util"
)
// TestBindConnContext verifies that a bindConn exposes its connection metadata
// (the tunneled hostname) via the xctx.Context interface, so the service accept
// loop forwards it into the handler context for filter.host routing (gost#898).
func TestBindConnContext(t *testing.T) {
md := mdata.Metadata(xmd.NewMetadata(map[string]any{"host": "example.local:80"}))
cn := &bindConn{
Conn: &net.TCPConn{},
localAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0},
remoteAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 80},
md: md,
}
got := mdutil.GetString(ictx.MetadataFromContext(cn.Context()), "host")
if got != "example.local:80" {
t.Errorf("host from Context() = %q, want %q", got, "example.local:80")
}
}
// TestBindUDPConnContext is the same check for the UDP reverse-tunnel conn.
func TestBindUDPConnContext(t *testing.T) {
md := mdata.Metadata(xmd.NewMetadata(map[string]any{"host": "example.local:80"}))
cn := &bindUDPConn{
Conn: &net.TCPConn{},
localAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 0},
remoteAddr: &net.TCPAddr{IP: net.IPv4(127, 0, 0, 1), Port: 80},
md: md,
}
got := mdutil.GetString(ictx.MetadataFromContext(cn.Context()), "host")
if got != "example.local:80" {
t.Errorf("host from Context() = %q, want %q", got, "example.local:80")
}
}
+9
View File
@@ -102,6 +102,15 @@ func (c *MasqueConn) GetRequestStream() *http3.RequestStream {
return c.reqStream
}
// OpenRequestStream opens another request stream on the shared HTTP/3 connection.
// CONNECT-UDP associations use one stream per destination.
func (c *MasqueConn) OpenRequestStream(ctx context.Context) (*http3.RequestStream, error) {
if c.clientConn == nil {
return nil, errors.New("masque: connection is closed")
}
return c.clientConn.OpenRequestStream(ctx)
}
// GetHost returns the proxy host.
func (c *MasqueConn) GetHost() string {
return c.host
+1 -1
View File
@@ -9,7 +9,7 @@ require (
github.com/gin-contrib/cors v1.7.2
github.com/gin-gonic/gin v1.12.0
github.com/go-gost/core v0.6.0
github.com/go-gost/go-shadowsocks2 v0.1.3
github.com/go-gost/go-shadowsocks2 v0.1.4
github.com/go-gost/gosocks4 v0.1.0
github.com/go-gost/gosocks5 v0.5.0
github.com/go-gost/plugin v0.5.0
+2 -2
View File
@@ -53,8 +53,8 @@ github.com/gin-gonic/gin v1.12.0 h1:b3YAbrZtnf8N//yjKeU2+MQsh2mY5htkZidOM7O0wG8=
github.com/gin-gonic/gin v1.12.0/go.mod h1:VxccKfsSllpKshkBWgVgRniFFAzFb9csfngsqANjnLc=
github.com/go-gost/core v0.6.0 h1:Y7jG5bI0QxdzDlMOyEtAYxCz+0GxAHKmaei667pRPWc=
github.com/go-gost/core v0.6.0/go.mod h1:WGI43jOka7FAsSAwi/fSMaqxdR+E339ycb4NBGlFr6A=
github.com/go-gost/go-shadowsocks2 v0.1.3 h1:6CUZLp+mTWXnKP2aK8/Z9ZP+ERMX9gSbywmPu4kGX/A=
github.com/go-gost/go-shadowsocks2 v0.1.3/go.mod h1:866zFNNI3He6Wef1M/IvAjTal74WhcfKfBgRpTlkKys=
github.com/go-gost/go-shadowsocks2 v0.1.4 h1:n2Po4TDKdLp1PsvhSiWFB/6S4e/YKZfsKJkA0PUa168=
github.com/go-gost/go-shadowsocks2 v0.1.4/go.mod h1:866zFNNI3He6Wef1M/IvAjTal74WhcfKfBgRpTlkKys=
github.com/go-gost/gosocks4 v0.1.0 h1:eAzev6qw4fzkFQKC9uCHLVNnnPdHyqCggbnfNN80Pmk=
github.com/go-gost/gosocks4 v0.1.0/go.mod h1:hzVjwijJuZR1pp3GqpTj+AKcSGrx68RlWTrQMFMYBP0=
github.com/go-gost/gosocks5 v0.5.0 h1:YE37l1MJwde8diIQdynStqogMotG5enoTdborhA5yic=
+12 -38
View File
@@ -72,45 +72,19 @@ func (h *forwardHandler) sniffingDial(ctx context.Context, network, address stri
}
// handleSniffedProtocol dispatches a sniffed connection to the protocol-specific
// sniffer (HandleHTTP or HandleTLS). It returns (true, err) when the protocol was
// sniffer registered for proto. It returns (true, err) when the protocol was
// handled and (false, nil) when the caller should fall through to raw forwarding.
func (h *forwardHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS, sniffing.ProtoRedis:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithHTTPKeepalive(h.md.httpKeepalive),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
if proto == sniffing.ProtoTLS {
return true, sniffer.HandleTLS(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
return true, sniffer.HandleRedis(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
return forwarder.Dispatch(h.sniffer.Build(), ctx, conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithHop(h.getHop()),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithHTTPKeepalive(h.md.httpKeepalive),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+9
View File
@@ -93,6 +93,15 @@ func (h *forwardHandler) handleRawForwarding(ctx context.Context, conn net.Conn,
// metadata when set, then falling back to hop selection.
func (h *forwardHandler) selectTarget(ctx context.Context, proto string) *chain.Node {
if host := mdutil.GetString(ictx.MetadataFromContext(ctx), "host"); host != "" {
// Route the tunneled hostname through the hop so a node with a matching
// filter.host is selected (the node's addr is the real target). If the
// hop has no matching node (e.g. no filter configured), fall back to
// dialing the hostname directly — the pre-regression behavior.
if curHop := h.getHop(); curHop != nil {
if node := curHop.Select(ctx, hop.ProtocolSelectOption(proto), hop.HostSelectOption(host)); node != nil {
return node
}
}
return &chain.Node{Addr: host}
}
if curHop := h.getHop(); curHop != nil {
+46 -6
View File
@@ -201,7 +201,52 @@ func TestHandleRawForwarding_MarkerReset(t *testing.T) {
}
}
func TestHandleRawForwarding_ContextHost(t *testing.T) {
// TestHandleRawForwarding_ContextHostHop: a tunneled hostname with a hop whose
// node matches the host filter routes to that node's addr (the documented
// filter.host client routing). The node addr is the real target.
func TestHandleRawForwarding_ContextHostHop(t *testing.T) {
server, client := net.Pipe()
defer server.Close()
defer client.Close()
var gotHost string
h := newInitdHandler(withRouter(&mockRouter{
opts: &chain.RouterOptions{},
dialFn: func(ctx context.Context, network, address string) (net.Conn, error) {
return server, nil
},
}))
h.Forward(&mockHop{
selectFn: func(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
var so hop.SelectOptions
for _, o := range opts {
o(&so)
}
gotHost = so.Host
return &chain.Node{Name: "example", Addr: "10.0.0.2:80"}
},
})
ctx := ictx.ContextWithMetadata(context.Background(), xmd.NewMetadata(map[string]any{
"host": "example.local:80",
}))
conn := newStringConn(nil)
ro := &xrecorder.HandlerRecorderObject{}
err := h.handleRawForwarding(ctx, conn, ro, nopLog(), "tcp", "")
if err != nil {
t.Fatalf("unexpected error: %v", err)
}
if gotHost != "example.local:80" {
t.Errorf("expected host passed to hop selection, got '%s'", gotHost)
}
if ro.Host != "10.0.0.2:80" {
t.Errorf("expected node addr to be dialed, got '%s'", ro.Host)
}
}
// TestHandleRawForwarding_ContextHostNoHop: when there is no hop, the tunneled
// hostname is dialed directly (fallback preserving the pre-regression behavior).
func TestHandleRawForwarding_ContextHostNoHop(t *testing.T) {
server, client := net.Pipe()
defer server.Close()
defer client.Close()
@@ -212,11 +257,6 @@ func TestHandleRawForwarding_ContextHost(t *testing.T) {
return server, nil
},
}))
h.Forward(&mockHop{
selectFn: func(ctx context.Context, opts ...hop.SelectOption) *chain.Node {
return &chain.Node{Addr: "10.0.0.2:80"}
},
})
ctx := ictx.ContextWithMetadata(context.Background(), xmd.NewMetadata(map[string]any{
"host": "10.0.0.99:8080",
}))
+12 -38
View File
@@ -72,45 +72,19 @@ func (h *forwardHandler) sniffingDial(ctx context.Context, network, address stri
}
// handleSniffedProtocol dispatches a sniffed connection to the protocol-specific
// sniffer (HandleHTTP or HandleTLS). It returns (true, err) when the protocol was
// sniffer registered for proto. It returns (true, err) when the protocol was
// handled and (false, nil) when the caller should fall through to raw forwarding.
func (h *forwardHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS, sniffing.ProtoRedis:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithHTTPKeepalive(h.md.httpKeepalive),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
if proto == sniffing.ProtoTLS {
return true, sniffer.HandleTLS(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
}
return true, sniffer.HandleRedis(ctx, conn,
forwarder.WithService(h.options.Service),
forwarder.WithDial(dial),
forwarder.WithHop(h.getHop()),
forwarder.WithBypass(h.options.Bypass),
forwarder.WithRecorderObject(ro),
forwarder.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
return forwarder.Dispatch(h.sniffer.Build(), ctx, conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithHop(h.getHop()),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithHTTPKeepalive(h.md.httpKeepalive),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+8 -29
View File
@@ -128,35 +128,14 @@ func (h *httpHandler) sniffAndHandle(ctx context.Context, conn net.Conn, cc net.
sniffer := h.sniffer.Build()
conn = xnet.NewReadWriteConn(br, conn, conn)
switch proto {
case sniffing.ProtoHTTP:
return true, sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoTLS:
return true, sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoRedis:
return true, sniffer.HandleRedis(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
return false, nil
return sniffing.Dispatch(sniffer, ctx, "tcp", conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
// dial establishes an upstream connection through the proxy chain router.
+19 -17
View File
@@ -368,9 +368,6 @@ func (h *masqueHandler) handleConnectUDP(ctx context.Context, w http.ResponseWri
return ErrDatagramNotSupport
}
// Get the underlying HTTP/3 stream
stream := streamer.HTTPStream()
// Get target connection - either through router/chain or direct
var targetPC net.PacketConn
@@ -393,15 +390,17 @@ func (h *masqueHandler) handleConnectUDP(ctx context.Context, w http.ResponseWri
ro.SrcAddr = c.LocalAddr().String()
// The connection from router should be a PacketConn (e.g., from masque connector)
if pc, ok := c.(net.PacketConn); ok {
targetPC = pc
log.Debugf("relaying UDP to %s via chain", targetAddr)
} else {
// Wrap as PacketConn if it's a regular Conn
targetPC = &connPacketConn{Conn: c, raddr: raddr}
log.Debugf("relaying UDP to %s via chain (wrapped)", targetAddr)
}
// Always wrap the router-returned conn as a point-to-point PacketConn
// that writes via Conn.Write against the fixed target. A direct UDP
// dial returns a *net.UDPConn that satisfies net.PacketConn but is
// pre-connected, so calling WriteTo on it fails with "use of WriteTo
// with pre-connected connection" and no packet ever reaches the target.
// Routing writes through Conn.Write works for both direct dials and
// stream-based upstream MASQUE connectors. The MASQUE target is fixed
// for the connection lifetime (parsed from the request path), so a
// single remote address is always correct.
targetPC = &connPacketConn{Conn: c, raddr: raddr}
log.Debugf("relaying UDP to %s via chain", targetAddr)
} else {
// Direct connection - create local UDP socket
directConn, err := net.ListenPacket("udp", "")
@@ -424,14 +423,17 @@ func (h *masqueHandler) handleConnectUDP(ctx context.Context, w http.ResponseWri
// Wrap target with metrics
targetPC = metrics.WrapPacketConn(h.options.Service, targetPC)
// Send success response with capsule-protocol header
// Send the success response WITH the mandatory Capsule-Protocol header
// (RFC 9298), then hijack the stream. HTTPStream() flushes the response
// internally and commits the headers, so Capsule-Protocol MUST be set
// before it; otherwise the client never sees Capsule-Protocol: ?1 and
// rejects the tunnel. Dialing the target first (above) means we only
// respond 200 once the relay path is ready, matching handleConnectTCP.
w.Header().Set("Capsule-Protocol", "?1")
w.WriteHeader(http.StatusOK)
// Flush the response headers
if flusher, ok := w.(http.Flusher); ok {
flusher.Flush()
}
// Hijack the underlying HTTP/3 stream for datagram exchange.
stream := streamer.HTTPStream()
// Create datagram connection wrapping the HTTP/3 stream (client side)
datagramConn := masque_util.NewDatagramConn(stream, laddr, raddr)
+12 -28
View File
@@ -231,35 +231,19 @@ func (h *relayHandler) handleConnect(ctx context.Context, conn net.Conn, network
}
conn = xnet.NewReadWriteConn(br, conn, conn)
switch proto {
case sniffing.ProtoHTTP:
handled, err := sniffing.Dispatch(sniffer, ctx, "tcp", conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
if handled {
// Reset the recording start time: the sniffer handles the
// connection and manages ro.Time from here on.
ro.Time = time.Time{}
return sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoTLS:
ro.Time = time.Time{}
return sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
case sniffing.ProtoRedis:
ro.Time = time.Time{}
return sniffer.HandleRedis(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
return err
}
}
+14 -39
View File
@@ -90,49 +90,24 @@ func (h *sniHandler) sniffingDialTLS(ctx context.Context, network, address strin
}
// handleSniffedProtocol dispatches a sniffed connection to the
// protocol-specific sniffer ([sniffing.Sniffer.HandleHTTP] or
// [sniffing.Sniffer.HandleTLS]).
// protocol-specific sniffer registered for proto.
//
// It returns (true, err) when the protocol was handled and (false, nil)
// when the protocol is unrecognised and the caller should handle the
// connection directly (which for the SNI handler means silently dropping it).
func (h *sniHandler) handleSniffedProtocol(ctx context.Context, conn net.Conn, ro *xrecorder.HandlerRecorderObject, log logger.Logger, proto string) (handled bool, err error) {
switch proto {
case sniffing.ProtoHTTP, sniffing.ProtoTLS, sniffing.ProtoRedis:
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
dialTLS := func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error) {
return h.sniffingDialTLS(ctx, network, address, ro, cfg)
}
sniffer := h.sniffer.Build()
if proto == sniffing.ProtoHTTP {
return true, sniffer.HandleHTTP(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
if proto == sniffing.ProtoTLS {
return true, sniffer.HandleTLS(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
return true, sniffer.HandleRedis(ctx, "tcp", conn,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
default:
return false, nil
dial := func(ctx context.Context, network, address string) (net.Conn, error) {
return h.sniffingDial(ctx, network, address, ro)
}
dialTLS := func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error) {
return h.sniffingDialTLS(ctx, network, address, ro, cfg)
}
return sniffing.Dispatch(h.sniffer.Build(), ctx, "tcp", conn, proto,
sniffing.WithService(h.options.Service),
sniffing.WithDial(dial),
sniffing.WithDialTLS(dialTLS),
sniffing.WithBypass(h.options.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
)
}
+73 -89
View File
@@ -9,9 +9,7 @@ import (
"encoding/json"
"errors"
"io"
"net"
"sort"
"strings"
"sync"
"time"
@@ -22,6 +20,7 @@ import (
"github.com/go-gost/core/routing"
"github.com/go-gost/core/selector"
"github.com/go-gost/x/config"
xs "github.com/go-gost/x/selector"
mdutil "github.com/go-gost/x/metadata/util"
node_parser "github.com/go-gost/x/config/parsing/node"
"github.com/go-gost/x/internal/loader"
@@ -33,6 +32,12 @@ type options struct {
nodes []*chain.Node
bypass bypass.Bypass
selector selector.Selector[*chain.Node]
// failMaxFails / failTimeout mirror the FailFilter config used by the
// selector. Node selection consults them via selector.IsFailed to skip
// nodes currently marked failed (FailFilter itself no-ops on a single
// node, so the marker is checked here instead).
failMaxFails int
failTimeout time.Duration
fileLoader loader.Loader
redisLoader loader.Loader
httpLoader loader.Loader
@@ -71,6 +76,16 @@ func SelectorOption(s selector.Selector[*chain.Node]) Option {
}
}
// FailFilterSettingsOption sets the maxFails/failTimeout used by the hop's
// FailFilter. The pre-selection dead-node filter consults these to decide
// whether a node is currently marked failed.
func FailFilterSettingsOption(maxFails int, timeout time.Duration) Option {
return func(o *options) {
o.failMaxFails = maxFails
o.failTimeout = timeout
}
}
// ReloadPeriodOption sets the interval for periodic reloading of node lists.
func ReloadPeriodOption(period time.Duration) Option {
return func(opts *options) {
@@ -154,13 +169,12 @@ func (p *chainHop) Nodes() []*chain.Node {
// Selection pipeline:
//
// 1. Hop-level bypass — entire hop skipped if bypass matches (addr/host)
// 2. Per-node filter (pool) — each candidate must pass at least one gate:
// a. routing.Matcher — boolean expression (Host/Protocol/Method/Path/
// Query/Header/Body). Match → non-zero Priority.
// b. isEligible fallback — when no Matcher set, checks node Options.Filter:
// checkHost(Host), checkProtocol(Protocol),
// checkPath(Path prefix).
// Nodes that fail all gates or hit a node-level bypass are excluded.
// 2. Per-node matcher (pool) — each candidate must pass the gate:
// routing.Matcher — boolean expression (Host/Protocol/Method/
// Path/Query/Header/Body). Match → non-zero
// Priority. Nodes without a matcher are
// unconditional candidates.
// Nodes that fail the gate or hit a node-level bypass are excluded.
// 3. Priority short-circuit — if the top node has strictly higher Priority
// (>0) than the rest and no backup flag is present,
// it wins directly (selector skipped).
@@ -192,40 +206,26 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
maxPriCount int
maxPriNode *chain.Node
hasBackup bool
sawFailed bool
)
for _, node := range p.Nodes() {
if node == nil {
continue
}
// node level bypass
if node.Options().Bypass != nil &&
node.Options().Bypass.Contains(ctx, options.Network, options.Addr, bypass.WithHostOption(options.Host)) {
// A node currently marked failed (http.failCodes match, probe failure)
// is never selected, regardless of its priority or matcher. Checked
// before the eligibility gates so failed nodes skip matcher evaluation
// (body matching reads a large request prefix) and never reach the
// priority short-circuit.
if xs.IsFailed(node, p.options.failMaxFails, p.options.failTimeout) {
sawFailed = true
continue
}
if !p.nodeMatches(ctx, node, &options) {
continue
}
if matcher := node.Options().Matcher; matcher != nil {
// Gate 2a: routing.Matcher — boolean expression on HTTP-level fields.
// A match assigns a non-zero Priority (from rule specificity).
req := routing.Request{
ClientIP: options.ClientIP,
Network: options.Network,
Host: options.Host,
Protocol: options.Protocol,
Method: options.Method,
Path: options.Path,
Query: options.Query,
Header: options.Header,
Body: options.Body,
}
if !matcher.Match(&req) {
continue
}
log.Debugf("node %s match request %s %s, priority %d", node.Name, req.Protocol, req.Host, node.Options().Priority)
} else {
// Gate 2b: fallback eligibility — simple host/protocol/path filters.
if !p.isEligible(node, &options) {
continue
}
log.Debugf("node %s match request %s %s, priority %d", node.Name, options.Protocol, options.Host, node.Options().Priority)
}
pri := node.Options().Priority
@@ -244,6 +244,22 @@ func (p *chainHop) Select(ctx context.Context, opts ...hop.SelectOption) *chain.
nodes = append(nodes, node)
}
if len(nodes) == 0 {
if sawFailed {
// All candidates are marked failed. Best-effort last resort: return
// the first candidate that matches this request, so a transient
// failure (e.g. a single 429) doesn't take the hop fully down — a
// successful dial in the caller resets the marker, so the node
// recovers as soon as it works again. Requests matching no node
// still yield nil (no route).
for _, node := range p.Nodes() {
if node != nil &&
xs.IsFailed(node, p.options.failMaxFails, p.options.failTimeout) &&
p.nodeMatches(ctx, node, &options) {
log.Debugf("last resort: node %s selected (all candidates failed)", node.Name)
return node
}
}
}
return nil
}
if len(nodes) == 1 {
@@ -327,68 +343,36 @@ func isAllBackup(nodes []*chain.Node) bool {
return len(nodes) > 0
}
func (p *chainHop) isEligible(node *chain.Node, opts *hop.SelectOptions) bool {
// nodeMatches reports whether node passes the per-node bypass and matcher
// gates for the given selection options. Shared by the candidate-pool scan
// and the all-failed last-resort fallback. A node with no matcher is an
// unconditional candidate (legacy filter config is normalized to a matcher
// at parse time).
func (p *chainHop) nodeMatches(ctx context.Context, node *chain.Node, opts *hop.SelectOptions) bool {
if node == nil {
return false
}
if node.Options().Filter == nil {
return true
}
if !p.checkHost(opts.Host, node) || !p.checkProtocol(opts.Protocol, node) || !p.checkPath(opts.Path, node) {
if node.Options().Bypass != nil &&
node.Options().Bypass.Contains(ctx, opts.Network, opts.Addr, bypass.WithHostOption(opts.Host)) {
return false
}
if matcher := node.Options().Matcher; matcher != nil {
req := routing.Request{
ClientIP: opts.ClientIP,
Network: opts.Network,
Host: opts.Host,
Protocol: opts.Protocol,
Method: opts.Method,
Path: opts.Path,
Query: opts.Query,
Header: opts.Header,
Body: opts.Body,
}
return matcher.Match(&req)
}
return true
}
func (p *chainHop) checkHost(host string, node *chain.Node) bool {
var vhost string
if filter := node.Options().Filter; filter != nil {
vhost = filter.Host
}
if vhost == "" { // backup node
return true
}
if host == "" {
return false
}
if v, _, _ := net.SplitHostPort(host); v != "" {
host = v
}
if vhost == host || vhost[0] == '.' && strings.HasSuffix(host, vhost[1:]) {
return true
}
return false
}
func (p *chainHop) checkProtocol(protocol string, node *chain.Node) bool {
var prot string
if filter := node.Options().Filter; filter != nil {
prot = filter.Protocol
}
if prot == "" {
return true
}
return prot == protocol
}
func (p *chainHop) checkPath(path string, node *chain.Node) bool {
var pathFilter string
if filter := node.Options().Filter; filter != nil {
pathFilter = filter.Path
}
if pathFilter == "" {
return true
}
return strings.HasPrefix(path, pathFilter)
}
func (p *chainHop) periodReload(ctx context.Context) error {
if err := p.reload(ctx); err != nil {
p.logger.Warnf("reload: %v", err)
+165 -265
View File
@@ -19,9 +19,21 @@ import (
"github.com/go-gost/core/routing"
xmd "github.com/go-gost/x/metadata"
xlogger "github.com/go-gost/x/logger"
xselector "github.com/go-gost/x/selector"
xrouting "github.com/go-gost/x/routing"
"github.com/go-gost/x/registry"
)
// mustMatcher parses a matcher DSL rule, failing the test on error.
func mustMatcher(t *testing.T, rule string) routing.Matcher {
t.Helper()
m, err := xrouting.NewMatcher(rule)
if err != nil {
t.Fatalf("NewMatcher(%q): %v", rule, err)
}
return m
}
// --- Mock types ---
type testBypass struct {
@@ -548,6 +560,110 @@ func TestSelect_EqualPriorityDoesNotShortcut(t *testing.T) {
}
}
// TestSelect_FailCodesHighPriorityNode reproduces the reverse-proxy failCodes
// bug: a node with a stricter (longer) matcher wins by auto-computed priority,
// so the priority shortcut selected it directly and bypassed the FailFilter —
// a node marked failed by http.failCodes could never be excluded, so failover
// never happened. After the fix, marking the high-priority node makes Select
// fall through to the healthy lower-priority node.
func TestSelect_FailCodesHighPriorityNode(t *testing.T) {
// highPriority: stricter matcher -> longer rule -> higher auto priority.
highPriority := chain.NewNode("high", "127.0.0.1:8080",
chain.MatcherNodeOption(&testMatcher{match: true}),
chain.PriorityNodeOption(100), // simulates len(longer-rule)
)
// healthy: less specific matcher -> lower priority.
healthy := chain.NewNode("healthy", "127.0.0.1:9090",
chain.MatcherNodeOption(&testMatcher{match: true}),
chain.PriorityNodeOption(10),
)
sel := xselector.NewSelector[*chain.Node](
xselector.FIFOStrategy[*chain.Node](),
xselector.FailFilter[*chain.Node](1, 60*time.Second),
)
h := newTestHop(NodeOption(highPriority, healthy), SelectorOption(sel),
FailFilterSettingsOption(1, 60*time.Second))
defer h.Close()
// Before marking: high-priority node wins via the shortcut.
node := h.Select(context.Background())
if node == nil {
t.Fatal("expected node, got nil")
}
if node.Name != "high" {
t.Fatalf("expected 'high' (higher priority) before marking, got %q", node.Name)
}
// Simulate http.failCodes marking the high-priority node failed (429/5xx).
highPriority.Marker().Mark()
// After marking: Select must fall through to the healthy node.
node = h.Select(context.Background())
if node == nil {
t.Fatal("expected a node after marking, got nil")
}
if node.Name != "healthy" {
t.Fatalf("expected 'healthy' (marked node excluded by FailFilter), got %q", node.Name)
}
}
// TestSelect_SingleNodeFailedLastResort: with all candidates failed, Select
// falls back to the first matching node (best effort) instead of returning
// nil — a transient failure (e.g. one 429) must not take a lone-node hop
// fully down. A successful dial in the caller resets the marker.
func TestSelect_SingleNodeFailedLastResort(t *testing.T) {
only := chain.NewNode("only", "127.0.0.1:8080")
h := newTestHop(NodeOption(only))
defer h.Close()
only.Marker().Mark()
node := h.Select(context.Background())
if node == nil || node.Name != "only" {
t.Fatalf("expected last-resort node 'only', got %v", node)
}
}
// TestSelect_AllFailedLastResort: when every candidate is marked failed, the
// first matching node is still returned as a best-effort fallback.
func TestSelect_AllFailedLastResort(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
n1.Marker().Mark()
n2.Marker().Mark()
node := h.Select(context.Background())
if node == nil {
t.Fatal("expected a last-resort node, got nil")
}
if node.Name != "n1" && node.Name != "n2" {
t.Fatalf("expected n1 or n2, got %q", node.Name)
}
}
// TestSelect_AllFailedNoMatchReturnsNil: the last resort must still respect
// the matcher gates — a request matching no node yields nil even when nodes
// are marked failed (no route, not a fallback target).
func TestSelect_AllFailedNoMatchReturnsNil(t *testing.T) {
only := chain.NewNode("only", "127.0.0.1:8080",
chain.MatcherNodeOption(&testMatcher{match: false}),
)
h := newTestHop(NodeOption(only))
defer h.Close()
only.Marker().Mark()
node := h.Select(context.Background())
if node != nil {
t.Fatalf("expected nil (no node matches), got %q", node.Name)
}
}
func TestSelect_NilNodesSkipped(t *testing.T) {
h := newTestHop()
defer h.Close()
@@ -563,259 +679,12 @@ func TestSelect_NilNodesSkipped(t *testing.T) {
}
// =============================================================================
// isEligible tests
// Select with matcher integration tests
// =============================================================================
func TestIsEligible_NilNode(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
if ch.isEligible(nil, &hop.SelectOptions{}) {
t.Error("expected false for nil node")
}
}
func TestIsEligible_NoFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.isEligible(n, &hop.SelectOptions{}) {
t.Error("expected true when no filter")
}
}
func TestIsEligible_HostMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if !ch.isEligible(n, &hop.SelectOptions{Host: "example.com"}) {
t.Error("expected true for matching host")
}
}
func TestIsEligible_HostMismatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if ch.isEligible(n, &hop.SelectOptions{Host: "other.com"}) {
t.Error("expected false for mismatching host")
}
}
func TestIsEligible_ProtocolMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
if !ch.isEligible(n, &hop.SelectOptions{Protocol: "http"}) {
t.Error("expected true for matching protocol")
}
}
func TestIsEligible_ProtocolMismatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
if ch.isEligible(n, &hop.SelectOptions{Protocol: "socks5"}) {
t.Error("expected false for mismatching protocol")
}
}
func TestIsEligible_PathMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if !ch.isEligible(n, &hop.SelectOptions{Path: "/api/v1/users"}) {
t.Error("expected true for matching path prefix")
}
}
func TestIsEligible_PathMismatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if ch.isEligible(n, &hop.SelectOptions{Path: "/other/route"}) {
t.Error("expected false for mismatching path")
}
}
func TestIsEligible_AllFiltersMustPass(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{
Host: "example.com",
Protocol: "http",
}
if ch.isEligible(n, &hop.SelectOptions{Host: "example.com", Protocol: "socks5"}) {
t.Error("expected false: host matches but protocol doesn't")
}
}
// =============================================================================
// checkHost tests
// =============================================================================
func TestCheckHost_EmptyFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.checkHost("anything.com", n) {
t.Error("expected true when no host filter")
}
}
func TestCheckHost_ExactMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if !ch.checkHost("example.com", n) {
t.Error("expected true for exact host match")
}
}
func TestCheckHost_WildcardMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: ".example.com"}
if !ch.checkHost("sub.example.com", n) {
t.Error("expected true for wildcard host match")
}
}
func TestCheckHost_WildcardNoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: ".example.com"}
if ch.checkHost("other.net", n) {
t.Error("expected false for non-matching wildcard")
}
}
func TestCheckHost_EmptyInputHost(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if ch.checkHost("", n) {
t.Error("expected false when input host is empty and filter is set")
}
}
func TestCheckHost_StripPort(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if !ch.checkHost("example.com:443", n) {
t.Error("expected true after stripping port")
}
}
func TestCheckHost_IPWithPort(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "192.168.1.1"}
if !ch.checkHost("192.168.1.1:3128", n) {
t.Error("expected true for IP with port")
}
}
func TestCheckHost_IPv6WithPort(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "::1"}
if !ch.checkHost("[::1]:8080", n) {
t.Error("expected true for bracketed IPv6 with port")
}
}
func TestCheckHost_NoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
if ch.checkHost("other.com", n) {
t.Error("expected false for non-matching host")
}
}
// =============================================================================
// checkProtocol tests
// =============================================================================
func TestCheckProtocol_EmptyFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.checkProtocol("http", n) {
t.Error("expected true when no protocol filter")
}
}
func TestCheckProtocol_Match(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
if !ch.checkProtocol("http", n) {
t.Error("expected true for matching protocol")
}
}
func TestCheckProtocol_NoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Protocol: "socks5"}
if ch.checkProtocol("http", n) {
t.Error("expected false for non-matching protocol")
}
}
// =============================================================================
// checkPath tests
// =============================================================================
func TestCheckPath_EmptyFilter(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
if !ch.checkPath("/any/path", n) {
t.Error("expected true when no path filter")
}
}
func TestCheckPath_PrefixMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if !ch.checkPath("/api/v1/users", n) {
t.Error("expected true for prefix match")
}
}
func TestCheckPath_ExactMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if !ch.checkPath("/api", n) {
t.Error("expected true for exact match (prefix of self)")
}
}
func TestCheckPath_NoMatch(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if ch.checkPath("/other", n) {
t.Error("expected false for non-prefix match")
}
}
func TestCheckPath_ShorterInput(t *testing.T) {
ch := &chainHop{logger: xlogger.Nop()}
n := chain.NewNode("n1", "127.0.0.1:8080")
n.Options().Filter = &chain.NodeFilterSettings{Path: "/api"}
if ch.checkPath("/ap", n) {
t.Error("expected false when input is shorter than filter prefix")
}
}
// =============================================================================
// Select with filter integration tests
// =============================================================================
func TestSelect_FilterHost_Match(t *testing.T) {
func TestSelect_MatcherHost_Match(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`)")
h := newTestHop(NodeOption(n1))
defer h.Close()
@@ -825,9 +694,9 @@ func TestSelect_FilterHost_Match(t *testing.T) {
}
}
func TestSelect_FilterHost_NoMatch(t *testing.T) {
func TestSelect_MatcherHost_NoMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: "example.com"}
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
@@ -841,9 +710,9 @@ func TestSelect_FilterHost_NoMatch(t *testing.T) {
}
}
func TestSelect_FilterProtocol_Match(t *testing.T) {
func TestSelect_MatcherProtocol_Match(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
n1.Options().Matcher = mustMatcher(t, "Proto(`http`)")
h := newTestHop(NodeOption(n1))
defer h.Close()
@@ -853,9 +722,9 @@ func TestSelect_FilterProtocol_Match(t *testing.T) {
}
}
func TestSelect_FilterProtocol_NoMatch(t *testing.T) {
func TestSelect_MatcherProtocol_NoMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Protocol: "http"}
n1.Options().Matcher = mustMatcher(t, "Proto(`http`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
@@ -869,6 +738,39 @@ func TestSelect_FilterProtocol_NoMatch(t *testing.T) {
}
}
func TestSelect_MatcherWildcardHost_MatchesApexAndSubdomain(t *testing.T) {
// filter `.example.com` (legacy wildcard) matches apex AND subdomains;
// the converter emits `Host(example.com) || Host(.example.com)`.
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`) || Host(`.example.com`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
for _, host := range []string{"example.com", "sub.example.com", "a.b.example.com"} {
node := h.Select(context.Background(), hop.HostSelectOption(host))
if node == nil || node.Name != "n1" {
t.Errorf("expected n1 for host %q, got %v", host, node)
}
}
}
func TestSelect_MatcherWildcardHost_NoSubdomainMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Matcher = mustMatcher(t, "Host(`example.com`) || Host(`.example.com`)")
n2 := chain.NewNode("n2", "127.0.0.1:9090")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
node := h.Select(context.Background(), hop.HostSelectOption("other.net"))
if node == nil {
t.Fatal("expected node, got nil")
}
if node.Name != "n2" {
t.Errorf("expected 'n2' (n1 host mismatch), got %q", node.Name)
}
}
// =============================================================================
// Select with select options
// =============================================================================
@@ -1315,14 +1217,13 @@ func TestSelect_OnlyNegativePriorities(t *testing.T) {
}
// =============================================================================
// Select with backup node (empty host filter means backup)
// Select with catch-all node (no matcher = unconditional candidate)
// =============================================================================
func TestSelect_BackupNode_HostFilterEmpty(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: ""}
func TestSelect_CatchAllNode_TakesOverOnNoMatch(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080") // no matcher: catch-all
n2 := chain.NewNode("n2", "127.0.0.1:9090")
n2.Options().Filter = &chain.NodeFilterSettings{Host: "specific.com"}
n2.Options().Matcher = mustMatcher(t, "Host(`specific.com`)")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
@@ -1331,15 +1232,14 @@ func TestSelect_BackupNode_HostFilterEmpty(t *testing.T) {
t.Fatal("expected node, got nil")
}
if node.Name != "n1" {
t.Errorf("expected 'n1' (backup), got %q", node.Name)
t.Errorf("expected 'n1' (catch-all), got %q", node.Name)
}
}
func TestSelect_SpecificHostNodeWins(t *testing.T) {
n1 := chain.NewNode("n1", "127.0.0.1:8080")
n1.Options().Filter = &chain.NodeFilterSettings{Host: ""}
n1 := chain.NewNode("n1", "127.0.0.1:8080") // no matcher: catch-all
n2 := chain.NewNode("n2", "127.0.0.1:9090")
n2.Options().Filter = &chain.NodeFilterSettings{Host: "specific.com"}
n2.Options().Matcher = mustMatcher(t, "Host(`specific.com`)")
h := newTestHop(NodeOption(n1, n2))
defer h.Close()
+50
View File
@@ -0,0 +1,50 @@
package forwarder
import (
"context"
"net"
"sync"
"github.com/go-gost/x/internal/util/sniffing"
)
// HandleFunc handles a connection whose protocol has been detected as proto,
// mirroring sniffing.HandleFunc but without the network parameter. The
// receiver s carries the per-connection Sniffer state.
type HandleFunc func(s *Sniffer, ctx context.Context, conn net.Conn, opts ...sniffing.HandleOption) error
var protoHandlers sync.Map // map[string]HandleFunc
// Register associates a protocol with its handler. Duplicate registration
// panics, mirroring the house registry pattern.
func Register(proto string, fn HandleFunc) {
if _, dup := protoHandlers.LoadOrStore(proto, fn); dup {
panic("forwarder: duplicate protocol handler: " + proto)
}
}
// Get returns the handler registered for proto.
func Get(proto string) (HandleFunc, bool) {
v, ok := protoHandlers.Load(proto)
if !ok {
return nil, false
}
return v.(HandleFunc), true
}
// Dispatch routes a detected protocol to its registered handler. It returns
// (false, nil) when no handler is registered for proto, signalling the caller
// to fall through to raw forwarding.
func Dispatch(s *Sniffer, ctx context.Context, conn net.Conn, proto string, opts ...sniffing.HandleOption) (handled bool, err error) {
fn, ok := Get(proto)
if !ok {
return false, nil
}
return true, fn(s, ctx, conn, opts...)
}
func init() {
Register(sniffing.ProtoHTTP, (*Sniffer).HandleHTTP)
Register(sniffing.ProtoTLS, (*Sniffer).HandleTLS)
Register(sniffing.ProtoRedis, (*Sniffer).HandleRedis)
}
+86
View File
@@ -0,0 +1,86 @@
package forwarder
import (
"context"
"net"
"testing"
"github.com/go-gost/x/internal/util/sniffing"
)
// TestRegisterGet verifies a handler can be registered and looked up by proto.
func TestRegisterGet(t *testing.T) {
const proto = "testproto-get"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ net.Conn, _ ...sniffing.HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
fn, ok := Get(proto)
if !ok {
t.Fatal("Get: handler not found")
}
if err := fn(&Sniffer{}, context.Background(), nil); err != nil {
t.Fatalf("fn() error = %v", err)
}
if !called {
t.Fatal("registered handler was not called")
}
if _, ok := Get("no-such-proto"); ok {
t.Fatal("Get: unexpected handler for unknown proto")
}
}
// TestRegisterDuplicate verifies registering the same proto twice panics.
func TestRegisterDuplicate(t *testing.T) {
const proto = "testproto-dup"
Register(proto, func(*Sniffer, context.Context, net.Conn, ...sniffing.HandleOption) error {
return nil
})
defer protoHandlers.Delete(proto)
defer func() {
if recover() == nil {
t.Fatal("expected panic on duplicate registration")
}
}()
Register(proto, func(*Sniffer, context.Context, net.Conn, ...sniffing.HandleOption) error {
return nil
})
}
// TestDispatch_Unknown verifies Dispatch falls through (false, nil) for an
// unregistered proto.
func TestDispatch_Unknown(t *testing.T) {
handled, err := Dispatch(&Sniffer{}, context.Background(), nil, "no-such-proto")
if handled {
t.Fatal("Dispatch: unknown proto handled = true, want false")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
}
// TestDispatch_Known verifies Dispatch routes a registered proto to its handler.
func TestDispatch_Known(t *testing.T) {
const proto = "testproto-known"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ net.Conn, _ ...sniffing.HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
handled, err := Dispatch(&Sniffer{}, context.Background(), nil, proto)
if !handled {
t.Fatal("Dispatch: known proto handled = false, want true")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
if !called {
t.Fatal("Dispatch: registered handler was not called")
}
}
+8 -76
View File
@@ -1,7 +1,6 @@
package forwarder
import (
"context"
"crypto"
"crypto/tls"
"crypto/x509"
@@ -11,13 +10,11 @@ import (
"github.com/go-gost/core/bypass"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/recorder"
"github.com/go-gost/x/config"
tls_util "github.com/go-gost/x/internal/util/tls"
"github.com/go-gost/x/internal/util/httpcache"
xrecorder "github.com/go-gost/x/recorder"
"github.com/go-gost/x/internal/util/sniffing"
tls_util "github.com/go-gost/x/internal/util/tls"
)
const (
@@ -33,77 +30,12 @@ const DefaultReadTimeout = 30 * time.Second
// DefaultCertPool is the default in-memory certificate pool used for TLS MITM.
var DefaultCertPool = tls_util.NewMemoryCertPool()
// HandleOptions holds configuration options for sniffing handlers.
type HandleOptions struct {
service string
dial func(ctx context.Context, network, address string) (net.Conn, error)
httpKeepalive bool
readTimeout time.Duration
node *chain.Node
hop hop.Hop
bypass bypass.Bypass
recorderObject *xrecorder.HandlerRecorderObject
log logger.Logger
}
// HandleOptions aliases sniffing.HandleOptions, the unified option set shared
// by the sniffing and forwarder Sniffer implementations.
type HandleOptions = sniffing.HandleOptions
// HandleOption configures HandleOptions for sniffing handlers.
type HandleOption func(opts *HandleOptions)
// WithService sets the service name for bypass and selection lookups.
func WithService(service string) HandleOption {
return func(opts *HandleOptions) {
opts.service = service
}
}
// WithDial sets the dial function used to establish upstream connections.
func WithDial(dial func(ctx context.Context, network, address string) (net.Conn, error)) HandleOption {
return func(opts *HandleOptions) {
opts.dial = dial
}
}
// WithHTTPKeepalive enables or disables HTTP keep-alive on the upstream connection.
func WithHTTPKeepalive(keepalive bool) HandleOption {
return func(opts *HandleOptions) {
opts.httpKeepalive = keepalive
}
}
// WithNode sets a pre-resolved chain node to connect to, bypassing hop selection.
func WithNode(node *chain.Node) HandleOption {
return func(opts *HandleOptions) {
opts.node = node
}
}
// WithHop sets the hop used for node selection when routing requests.
func WithHop(h hop.Hop) HandleOption {
return func(opts *HandleOptions) {
opts.hop = h
}
}
// WithBypass sets the bypass rules for filtering requests by host.
func WithBypass(bypass bypass.Bypass) HandleOption {
return func(opts *HandleOptions) {
opts.bypass = bypass
}
}
// WithRecorderObject sets the recorder object for capturing traffic metadata.
func WithRecorderObject(ro *xrecorder.HandlerRecorderObject) HandleOption {
return func(opts *HandleOptions) {
opts.recorderObject = ro
}
}
// WithLog sets the logger for the handler.
func WithLog(log logger.Logger) HandleOption {
return func(opts *HandleOptions) {
opts.log = log
}
}
type HandleOption = sniffing.HandleOption
// Sniffer handles HTTP and TLS traffic sniffing, recording, and MITM TLS
// termination for protocol-aware forwarding. It can intercept HTTP requests,
@@ -171,8 +103,8 @@ func normalizeHost(host, defaultPort string) string {
// effectiveReadTimeout returns the read timeout from options, falling back to
// the Sniffer's ReadTimeout, then to DefaultReadTimeout.
func (h *Sniffer) effectiveReadTimeout(ho *HandleOptions) time.Duration {
if ho.readTimeout > 0 {
return ho.readTimeout
if ho.ReadTimeout > 0 {
return ho.ReadTimeout
}
if h.ReadTimeout > 0 {
return h.ReadTimeout
+2 -2
View File
@@ -32,8 +32,8 @@ func (h *Sniffer) serveH2(ctx context.Context, conn net.Conn, ho *HandleOptions)
return errors.New("h2: invalid client preface")
}
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
ro.Time = time.Time{}
+32 -32
View File
@@ -51,7 +51,7 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
for _, opt := range opts {
opt(&ho)
}
ho.readTimeout = h.effectiveReadTimeout(&ho)
ho.ReadTimeout = h.effectiveReadTimeout(&ho)
pStats := xstats.Stats{}
conn = stats_wrapper.WrapConn(conn, &pStats)
@@ -62,13 +62,13 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
return err
}
log := ho.log
log := ho.Log
if log.IsLevelEnabled(logger.TraceLevel) {
dump, _ := httputil.DumpRequest(req, false)
log.Trace(string(dump))
}
ro := ho.recorderObject
ro := ho.RecorderObject
// Copy ro so that all internal recording (cache-hit serveCachedResponse
// and cache-miss httpRoundTrip) happens on a local clone, preventing
@@ -76,7 +76,7 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
ro2 := &xrecorder.HandlerRecorderObject{}
*ro2 = *ro
ro = ro2
ho.recorderObject = ro2
ho.RecorderObject = ro2
if clientIP := xhttp.GetClientIP(req); clientIP != nil {
clientAddr := &net.TCPAddr{IP: clientIP}
@@ -140,8 +140,8 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
}
upstreamHost = normalizeHost(ro.HTTP.Host, "80")
ho.log = log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log = ho.log
ho.Log = log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log = ho.Log
log.Debugf("connected to node %s(%s)", node.Name, node.Addr)
ro.SrcAddr = cc.LocalAddr().String()
@@ -179,7 +179,7 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, conn net.Conn, opts ...HandleO
ro.Host = reqHost
log = log.WithFields(map[string]any{"host": reqHost})
ho.log = log
ho.Log = log
}
}
}
@@ -227,19 +227,19 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
StatusCode: http.StatusServiceUnavailable,
}
if ho.bypass != nil &&
ho.bypass.Contains(ctx, "tcp", host,
bypass.WithService(ho.service),
if ho.Bypass != nil &&
ho.Bypass.Contains(ctx, "tcp", host,
bypass.WithService(ho.Service),
bypass.WithPathOption(req.RequestURI)) {
ho.log.Debugf("bypass: %s %s", host, req.RequestURI)
ho.Log.Debugf("bypass: %s %s", host, req.RequestURI)
res.StatusCode = http.StatusForbidden
return nil, res, xbypass.ErrBypass
}
node = &chain.Node{}
if ho.hop != nil {
if ho.Hop != nil {
var clientIP net.IP
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.recorderObject.ClientAddr); clientAddr != nil {
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.RecorderObject.ClientAddr); clientAddr != nil {
clientIP = clientAddr.IP
}
@@ -248,7 +248,7 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
// without consuming the stream that is still forwarded below.
var bodyPrefix []byte
var maxBodySize int
if nl, ok := ho.hop.(hop.NodeList); ok {
if nl, ok := ho.Hop.(hop.NodeList); ok {
for _, n := range nl.Nodes() {
if n == nil {
continue
@@ -271,7 +271,7 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
}
}
node = ho.hop.Select(ctx,
node = ho.Hop.Select(ctx,
hop.ClientIPSelectOption(clientIP),
hop.ProtocolSelectOption(sniffing.ProtoHTTP),
hop.HostSelectOption(host),
@@ -283,11 +283,11 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
)
}
if node == nil {
ho.log.Warnf("node for %s not found", host)
ho.Log.Warnf("node for %s not found", host)
res.StatusCode = http.StatusBadGateway
return nil, res, errors.New("node not available")
}
ho.recorderObject.Node = node.Name
ho.RecorderObject.Node = node.Name
if node.Addr == "" {
node = &chain.Node{
Name: node.Name,
@@ -299,21 +299,21 @@ func resolveHTTPNode(ctx context.Context, host string, req *http.Request, ho *Ha
// dial selects a node, establishes a connection, and sends the request upstream.
func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho *HandleOptions) (node *chain.Node, cc net.Conn, err error) {
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
if node = ho.node; node != nil {
if node = ho.Node; node != nil {
cc, err = dial(ctx, "tcp", node.Addr)
return
}
ro := ho.recorderObject
ro := ho.RecorderObject
host := normalizeHost(req.Host, "80")
if host != "" {
ro.Host = host
ho.log = ho.log.WithFields(map[string]any{
ho.Log = ho.Log.WithFields(map[string]any{
"host": host,
})
}
@@ -322,7 +322,7 @@ func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho
if resolveErr != nil {
ro.HTTP.StatusCode = res.StatusCode
if werr := res.Write(conn); werr != nil {
ho.log.Warnf("write error response: %v", werr)
ho.Log.Warnf("write error response: %v", werr)
}
return nil, nil, resolveErr
}
@@ -336,20 +336,20 @@ func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho
}
ro.Host = node.Addr
ho.log = ho.log.WithFields(map[string]any{
ho.Log = ho.Log.WithFields(map[string]any{
"node": node.Name,
"dst": node.Addr,
})
ho.log.Debugf("find node for host %s -> %s(%s)", host, node.Name, node.Addr)
ho.Log.Debugf("find node for host %s -> %s(%s)", host, node.Name, node.Addr)
cc, err = dial(ctx, "tcp", node.Addr)
if err != nil {
if marker := node.Marker(); marker != nil {
marker.Mark()
}
ho.log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
ho.Log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
if werr := res.Write(conn); werr != nil {
ho.log.Warnf("write error response: %v", werr)
ho.Log.Warnf("write error response: %v", werr)
}
return
}
@@ -368,9 +368,9 @@ func (h *Sniffer) dial(ctx context.Context, conn net.Conn, req *http.Request, ho
func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser, node *chain.Node, req *http.Request, pStats stats.Stats, ho *HandleOptions, staleResp *http.Response) (shouldClose bool, err error) {
shouldClose = true
log := ho.log
log := ho.Log
ro := &xrecorder.HandlerRecorderObject{}
*ro = *ho.recorderObject
*ro = *ho.RecorderObject
ro.Time = time.Now()
log.Infof("%s <-> %s", ro.RemoteAddr, req.Host)
@@ -439,7 +439,7 @@ func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser,
if httpSettings != nil {
if auther := httpSettings.Auther; auther != nil {
username, password, _ := req.BasicAuth()
id, ok := auther.Authenticate(ctx, username, password, auth.WithService(ho.service))
id, ok := auther.Authenticate(ctx, username, password, auth.WithService(ho.Service))
if !ok {
res.StatusCode = http.StatusUnauthorized
ro.HTTP.StatusCode = res.StatusCode
@@ -534,7 +534,7 @@ func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser,
br := bufio.NewReader(cc)
var resp *http.Response
for {
xio.SetReadDeadline(cc, time.Now().Add(ho.readTimeout))
xio.SetReadDeadline(cc, time.Now().Add(ho.ReadTimeout))
resp, err = http.ReadResponse(br, req)
if err != nil {
log.Errorf("read response: %v", err)
@@ -600,7 +600,7 @@ func (h *Sniffer) httpRoundTrip(ctx context.Context, rw, cc io.ReadWriteCloser,
resp.ProtoMinor = req.ProtoMinor
}
if !ho.httpKeepalive {
if !ho.HTTPKeepalive {
resp.Header.Set("Connection", "close")
}
@@ -679,7 +679,7 @@ func (h *Sniffer) serveStale(rw io.Writer, staleResp *http.Response, ro *xrecord
if h.Cache == nil || staleResp == nil || !h.Cache.ServeStale() {
return false
}
if !ho.httpKeepalive {
if !ho.HTTPKeepalive {
staleResp.Header.Set("Connection", "close")
}
ro.HTTP.StatusCode = staleResp.StatusCode
+14 -14
View File
@@ -25,20 +25,20 @@ func (h *Sniffer) HandleRedis(ctx context.Context, conn net.Conn, opts ...Handle
}
buf := new(bytes.Buffer)
if err := sniffing.ParseRedisMetadata(io.TeeReader(conn, buf), ho.recorderObject); err != nil {
if err := sniffing.ParseRedisMetadata(io.TeeReader(conn, buf), ho.RecorderObject); err != nil {
return err
}
ro := ho.recorderObject
ro := ho.RecorderObject
node, cc, err := h.dialRedis(ctx, &ho)
if err != nil {
return err
}
defer cc.Close()
ho.node = node
ho.Node = node
log := ho.log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log := ho.Log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log.Debugf("connected to node %s(%s)", node.Name, node.Addr)
ro.SrcAddr = cc.LocalAddr().String()
@@ -63,12 +63,12 @@ func (h *Sniffer) HandleRedis(ctx context.Context, conn net.Conn, opts ...Handle
// and ClientIPSelectOption.
func resolveRedisNode(ctx context.Context, ho *HandleOptions) (node *chain.Node, err error) {
node = &chain.Node{}
if ho.hop != nil {
if ho.Hop != nil {
var clientIP net.IP
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.recorderObject.ClientAddr); clientAddr != nil {
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.RecorderObject.ClientAddr); clientAddr != nil {
clientIP = clientAddr.IP
}
node = ho.hop.Select(ctx,
node = ho.Hop.Select(ctx,
hop.ClientIPSelectOption(clientIP),
hop.ProtocolSelectOption(sniffing.ProtoRedis),
)
@@ -76,19 +76,19 @@ func resolveRedisNode(ctx context.Context, ho *HandleOptions) (node *chain.Node,
if node == nil {
return nil, fmt.Errorf("node not available")
}
ho.recorderObject.Node = node.Name
ho.RecorderObject.Node = node.Name
return node, nil
}
// dialRedis selects a node via resolveRedisNode and establishes a TCP
// connection to the node's address.
func (h *Sniffer) dialRedis(ctx context.Context, ho *HandleOptions) (node *chain.Node, cc net.Conn, err error) {
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
if node = ho.node; node != nil {
if node = ho.Node; node != nil {
cc, err = dial(ctx, "tcp", node.Addr)
return
}
@@ -98,7 +98,7 @@ func (h *Sniffer) dialRedis(ctx context.Context, ho *HandleOptions) (node *chain
return
}
ro := ho.recorderObject
ro := ho.RecorderObject
addr := node.Addr
network := "tcp"
if opts := node.Options(); opts != nil {
@@ -117,18 +117,18 @@ func (h *Sniffer) dialRedis(ctx context.Context, ho *HandleOptions) (node *chain
}
ro.Host = addr
ho.log = ho.log.WithFields(map[string]any{
ho.Log = ho.Log.WithFields(map[string]any{
"node": node.Name,
"dst": fmt.Sprintf("%s/%s", addr, network),
})
ho.log.Debugf("find node for redis -> %s(%s)", node.Name, addr)
ho.Log.Debugf("find node for redis -> %s(%s)", node.Name, addr)
cc, err = dial(ctx, network, addr)
if err != nil {
if marker := node.Marker(); marker != nil {
marker.Mark()
}
ho.log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
ho.Log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
return
}
+18 -6
View File
@@ -497,10 +497,22 @@ func (b *rewriteBody) apply(body []byte, opts ...rewriter.RewriteOption) ([]byte
body = rewritten
}
} else if rw.Pattern != nil {
if strings.HasPrefix(rw.Type, "json:") {
if strings.HasPrefix(rw.Type, "json:") || strings.HasPrefix(rw.Type, "json-:") {
del := strings.HasPrefix(rw.Type, "json-:")
path := rw.Type[5:]
if del {
path = rw.Type[6:]
}
if rw.Pattern.MatchString(gjson.GetBytes(body, path).String()) {
replaced, err := sjson.SetBytes(body, path, string(rw.Replacement))
var (
replaced []byte
err error
)
if del {
replaced, err = sjson.DeleteBytes(body, path)
} else {
replaced, err = sjson.SetBytes(body, path, string(rw.Replacement))
}
if err == nil {
body = replaced
}
@@ -514,9 +526,9 @@ func (b *rewriteBody) apply(body []byte, opts ...rewriter.RewriteOption) ([]byte
}
// shouldApply reports whether a rewrite rule's configured type matches the
// actual request Content-Type. A type prefix "json:" matches only
// "application/json" content. An empty type defaults to "text/html", or "*"
// when a Rewriter is set.
// actual request Content-Type. JSON rules ("json:" or "json-:" Type prefixes)
// match only "application/json" content. An empty type defaults to
// "text/html", or "*" when a Rewriter is set.
func shouldApply(configuredType, actualContentType string, hasRewriter bool) bool {
t := configuredType
if t == "" {
@@ -526,7 +538,7 @@ func shouldApply(configuredType, actualContentType string, hasRewriter bool) boo
t = "text/html"
}
}
if strings.HasPrefix(t, "json:") {
if strings.HasPrefix(t, "json:") || strings.HasPrefix(t, "json-:") {
return actualContentType == "application/json"
}
return t == "*" || strings.Contains(t, actualContentType)
+182 -69
View File
@@ -6,6 +6,7 @@ import (
"compress/gzip"
"compress/zlib"
"context"
"encoding/json"
"io"
"net"
"net/http"
@@ -21,9 +22,11 @@ import (
"github.com/go-gost/core/hop"
"github.com/go-gost/core/recorder"
"github.com/go-gost/core/rewriter"
"github.com/klauspost/compress/zstd"
xlogger "github.com/go-gost/x/logger"
xrecorder "github.com/go-gost/x/recorder"
"github.com/klauspost/compress/zstd"
"github.com/go-gost/x/internal/util/sniffing"
)
// =============================================================================
@@ -36,6 +39,16 @@ func (n *noopRecorder) Record(_ context.Context, _ []byte, _ ...recorder.RecordO
return nil
}
// captureRecorder records every payload written through recorder.Recorder.
type captureRecorder struct {
data [][]byte
}
func (r *captureRecorder) Record(_ context.Context, b []byte, _ ...recorder.RecordOption) error {
r.data = append(r.data, append([]byte(nil), b...))
return nil
}
// mockBypass is a configurable bypass.Bypass for tests.
type mockBypass struct {
contains bool
@@ -230,7 +243,6 @@ func TestRewriteReqBody_Gzip(t *testing.T) {
})
}
func TestClampBodySize(t *testing.T) {
tests := []struct {
name string
@@ -297,7 +309,7 @@ func TestEffectiveReadTimeout(t *testing.T) {
for _, tt := range tests {
t.Run(tt.name, func(t *testing.T) {
h := &Sniffer{ReadTimeout: tt.snifferTO}
ho := &HandleOptions{readTimeout: tt.hoTO}
ho := &HandleOptions{ReadTimeout: tt.hoTO}
got := h.effectiveReadTimeout(ho)
if got != tt.want {
t.Errorf("effectiveReadTimeout() = %v, want %v", got, tt.want)
@@ -541,9 +553,10 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
}))
defer upstream.Close()
rec := &captureRecorder{}
h := &Sniffer{
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
Recorder: rec,
}
ro := &xrecorder.HandlerRecorderObject{}
@@ -555,12 +568,12 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithService("test-svc"),
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithService("test-svc"),
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return net.Dial("tcp", upstream.Listener.Addr().String())
}),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -581,17 +594,35 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
t.Errorf("status = %d, want %d", resp.StatusCode, http.StatusOK)
}
clientConn.Close()
if err := <-errCh; err != nil {
t.Logf("HandleHTTP returned: %v", err)
// Drain the body so HandleHTTP's response write completes before the
// client side closes (net.Pipe writes block until read).
body, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatalf("read body: %v", err)
}
if string(body) != "OK from upstream" {
t.Errorf("body = %q, want %q", body, "OK from upstream")
}
// Verify recorder object was populated.
if ro.SrcAddr == "" {
t.Error("SrcAddr should be populated")
clientConn.Close()
if err := <-errCh; err != nil {
t.Errorf("HandleHTTP returned: %v", err)
}
if ro.DstAddr == "" {
t.Error("DstAddr should be populated")
// HandleHTTP records on an internal clone of ro and publishes it via
// h.Recorder, so assert on the captured payload.
if len(rec.data) != 1 {
t.Fatalf("expected 1 recorder event, got %d", len(rec.data))
}
var recorded xrecorder.HandlerRecorderObject
if err := json.Unmarshal(rec.data[0], &recorded); err != nil {
t.Fatalf("unmarshal recorder event: %v", err)
}
if recorded.SrcAddr == "" {
t.Error("recorded srcAddr should be populated")
}
if recorded.DstAddr == "" {
t.Error("recorded dstAddr should be populated")
}
}
@@ -612,9 +643,9 @@ func TestHandleHTTP_HTTP2Detection(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithService("test-svc"),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithService("test-svc"),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -637,16 +668,16 @@ func TestHandleHTTP_HTTP2Detection(t *testing.T) {
func TestWithService(t *testing.T) {
opts := &HandleOptions{}
WithService("mysvc")(opts)
if opts.service != "mysvc" {
t.Errorf("service = %q, want %q", opts.service, "mysvc")
sniffing.WithService("mysvc")(opts)
if opts.Service != "mysvc" {
t.Errorf("service = %q, want %q", opts.Service, "mysvc")
}
}
func TestWithHTTPKeepalive(t *testing.T) {
opts := &HandleOptions{}
WithHTTPKeepalive(true)(opts)
if !opts.httpKeepalive {
sniffing.WithHTTPKeepalive(true)(opts)
if !opts.HTTPKeepalive {
t.Errorf("httpKeepalive = false, want true")
}
}
@@ -654,8 +685,8 @@ func TestWithHTTPKeepalive(t *testing.T) {
func TestWithNode(t *testing.T) {
opts := &HandleOptions{}
node := &chain.Node{Name: "test-node", Addr: "127.0.0.1:8080"}
WithNode(node)(opts)
if opts.node != node {
sniffing.WithNode(node)(opts)
if opts.Node != node {
t.Errorf("node not set")
}
}
@@ -663,8 +694,8 @@ func TestWithNode(t *testing.T) {
func TestWithHop(t *testing.T) {
opts := &HandleOptions{}
mh := &mockHop{}
WithHop(mh)(opts)
if opts.hop != mh {
sniffing.WithHop(mh)(opts)
if opts.Hop != mh {
t.Errorf("hop not set")
}
}
@@ -672,8 +703,8 @@ func TestWithHop(t *testing.T) {
func TestWithBypass(t *testing.T) {
opts := &HandleOptions{}
bp := &mockBypass{contains: true}
WithBypass(bp)(opts)
if opts.bypass != bp {
sniffing.WithBypass(bp)(opts)
if opts.Bypass != bp {
t.Errorf("bypass not set")
}
}
@@ -681,8 +712,8 @@ func TestWithBypass(t *testing.T) {
func TestWithRecorderObject(t *testing.T) {
opts := &HandleOptions{}
ro := &xrecorder.HandlerRecorderObject{}
WithRecorderObject(ro)(opts)
if opts.recorderObject != ro {
sniffing.WithRecorderObject(ro)(opts)
if opts.RecorderObject != ro {
t.Errorf("recorderObject not set")
}
}
@@ -690,8 +721,8 @@ func TestWithRecorderObject(t *testing.T) {
func TestWithLog(t *testing.T) {
opts := &HandleOptions{}
log := xlogger.Nop()
WithLog(log)(opts)
if opts.log != log {
sniffing.WithLog(log)(opts)
if opts.Log != log {
t.Errorf("log not set")
}
}
@@ -732,11 +763,11 @@ func TestHandleHTTP_DialError(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return nil, io.ErrUnexpectedEOF
}),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -782,11 +813,11 @@ func TestHandleHTTP_HTTP10Request(t *testing.T) {
errCh := make(chan error, 1)
go func() {
errCh <- h.HandleHTTP(context.Background(), serverConn,
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return net.Dial("tcp", upstream.Listener.Addr().String())
}),
WithRecorderObject(ro),
WithLog(xlogger.Nop()),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(xlogger.Nop()),
)
}()
@@ -926,10 +957,10 @@ var _ hop.Hop = (*configurableHop)(nil)
func TestResolveHTTPNode_Bypass(t *testing.T) {
ho := &HandleOptions{
service: "test-svc",
bypass: &mockBypass{contains: true},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Service: "test-svc",
Bypass: &mockBypass{contains: true},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -950,8 +981,8 @@ func TestResolveHTTPNode_Bypass(t *testing.T) {
func TestResolveHTTPNode_NoHop(t *testing.T) {
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -972,9 +1003,9 @@ func TestResolveHTTPNode_NoHop(t *testing.T) {
func TestResolveHTTPNode_HopReturnsNil(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: nil},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: nil},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -999,9 +1030,9 @@ func TestResolveHTTPNode_HopReturnsNil(t *testing.T) {
func TestResolveHTTPNode_HopReturnsValidNode(t *testing.T) {
expectedNode := &chain.Node{Name: "backend", Addr: "10.0.0.1:8080"}
ho := &HandleOptions{
hop: &configurableHop{node: expectedNode},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: expectedNode},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -1019,9 +1050,9 @@ func TestResolveHTTPNode_HopReturnsValidNode(t *testing.T) {
func TestResolveHTTPNode_HopReturnsNodeWithoutAddr(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: &chain.Node{Name: "backend"}},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: &chain.Node{Name: "backend"}},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
req, _ := http.NewRequest("GET", "http://example.com/path", nil)
@@ -1049,8 +1080,8 @@ func TestResolveHTTPNode_HopReturnsNodeWithoutAddr(t *testing.T) {
func TestResolveTLSNode_NoHop(t *testing.T) {
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1067,9 +1098,9 @@ func TestResolveTLSNode_NoHop(t *testing.T) {
func TestResolveTLSNode_HopReturnsNil(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: nil},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: nil},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1087,9 +1118,9 @@ func TestResolveTLSNode_HopReturnsNil(t *testing.T) {
func TestResolveTLSNode_HopReturnsValidNode(t *testing.T) {
expectedNode := &chain.Node{Name: "backend", Addr: "10.0.0.1:443"}
ho := &HandleOptions{
hop: &configurableHop{node: expectedNode},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: expectedNode},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1103,9 +1134,9 @@ func TestResolveTLSNode_HopReturnsValidNode(t *testing.T) {
func TestResolveTLSNode_HopReturnsNodeWithoutAddr(t *testing.T) {
ho := &HandleOptions{
hop: &configurableHop{node: &chain.Node{Name: "backend"}},
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Hop: &configurableHop{node: &chain.Node{Name: "backend"}},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
node, err := resolveTLSNode(context.Background(), "example.com", ho)
@@ -1720,8 +1751,8 @@ func TestServeH2_InvalidPreface(t *testing.T) {
Recorder: &noopRecorder{},
}
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
clientConn, serverConn := net.Pipe()
@@ -1744,7 +1775,6 @@ func TestServeH2_InvalidPreface(t *testing.T) {
// SSE (Server-Sent Events) Tests
// =============================================================================
func TestRewriteRespBody_SSE_ContentType(t *testing.T) {
// rewriteRespBody now rewrites SSE per-event via the universal wrapper.
resp := &http.Response{
@@ -1960,6 +1990,89 @@ func TestNewRewriteBody(t *testing.T) {
})
}
func TestRewriteJSONDelete(t *testing.T) {
makeRewrite := func(rewriteType, match, replace string) chain.HTTPBodyRewriteSettings {
return chain.HTTPBodyRewriteSettings{
Pattern: regexp.MustCompile(match),
Type: rewriteType,
Replacement: []byte(replace),
}
}
ctx := context.Background()
t.Run("delete matched field", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json-:user.id", "123", ""),
}, "application/json", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{},"keep":1}` {
t.Errorf("body = %q, want %q", string(got), `{"user":{},"keep":1}`)
}
})
t.Run("no match leaves body unchanged", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json-:user.id", "^9$", ""),
}, "application/json", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{"id":123},"keep":1}` {
t.Errorf("body = %q, want %q (unchanged)", string(got), `{"user":{"id":123},"keep":1}`)
}
})
t.Run("non-json content type filtered out", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json-:user.id", "123", ""),
}, "text/plain", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{"id":123},"keep":1}` {
t.Errorf("body = %q, want %q (unchanged)", string(got), `{"user":{"id":123},"keep":1}`)
}
})
t.Run("json: prefix still replaces", func(t *testing.T) {
src := io.NopCloser(strings.NewReader(`{"user":{"id":123},"keep":1}`))
body, err := newRewriteBody(ctx, src, []chain.HTTPBodyRewriteSettings{
makeRewrite("json:user.id", "123", "999"),
}, "application/json", "", -1, "", "")
if err != nil {
t.Fatal(err)
}
if body == nil {
t.Fatal("expected non-nil body")
}
got, _ := io.ReadAll(body)
body.Close()
if string(got) != `{"user":{"id":"999"},"keep":1}` {
t.Errorf("body = %q, want %q", string(got), `{"user":{"id":"999"},"keep":1}`)
}
})
}
func TestRewriteRespBody_CompressedEncodings(t *testing.T) {
tests := []struct {
name string
+26 -26
View File
@@ -32,7 +32,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
for _, opt := range opts {
opt(&ho)
}
ho.readTimeout = h.effectiveReadTimeout(&ho)
ho.ReadTimeout = h.effectiveReadTimeout(&ho)
buf := new(bytes.Buffer)
clientHello, err := dissector.ParseClientHello(io.TeeReader(conn, buf))
@@ -40,7 +40,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
return err
}
ro := ho.recorderObject
ro := ho.RecorderObject
ro.TLS = &xrecorder.TLSRecorderObject{
ServerName: clientHello.ServerName,
ClientHello: hex.EncodeToString(buf.Bytes()),
@@ -51,14 +51,14 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
host := normalizeHost(clientHello.ServerName, "443")
if host == "" {
if ho.log != nil {
ho.log.Debugf("no sni in clienthello from %s", conn.RemoteAddr())
if ho.Log != nil {
ho.Log.Debugf("no sni in clienthello from %s", conn.RemoteAddr())
}
} else {
ro.Host = host
}
if ho.bypass != nil && ho.bypass.Contains(ctx, "tcp", host, bypass.WithService(ho.service)) {
if ho.Bypass != nil && ho.Bypass.Contains(ctx, "tcp", host, bypass.WithService(ho.Service)) {
return xbypass.ErrBypass
}
@@ -67,9 +67,9 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
return err
}
defer cc.Close()
ho.node = node
ho.Node = node
log := ho.log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log := ho.Log.WithFields(map[string]any{"src": cc.LocalAddr().String(), "dst": cc.RemoteAddr().String()})
log.Debugf("connected to node %s(%s)", node.Name, node.Addr)
ro.SrcAddr = cc.LocalAddr().String()
@@ -89,7 +89,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
return err
}
xio.SetReadDeadline(cc, time.Now().Add(ho.readTimeout))
xio.SetReadDeadline(cc, time.Now().Add(ho.ReadTimeout))
serverHello, serverHelloErr := dissector.ParseServerHello(io.TeeReader(cc, buf))
xio.SetReadDeadline(cc, time.Time{})
@@ -128,12 +128,12 @@ func (h *Sniffer) HandleTLS(ctx context.Context, conn net.Conn, opts ...HandleOp
// resolveTLSNode selects a node for a TLS connection by applying hop selection.
func resolveTLSNode(ctx context.Context, host string, ho *HandleOptions) (node *chain.Node, err error) {
node = &chain.Node{}
if ho.hop != nil {
if ho.Hop != nil {
var clientIP net.IP
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.recorderObject.ClientAddr); clientAddr != nil {
if clientAddr, _ := net.ResolveTCPAddr("tcp", ho.RecorderObject.ClientAddr); clientAddr != nil {
clientIP = clientAddr.IP
}
node = ho.hop.Select(ctx,
node = ho.Hop.Select(ctx,
hop.ClientIPSelectOption(clientIP),
hop.HostSelectOption(host),
hop.ProtocolSelectOption(sniffing.ProtoTLS),
@@ -142,7 +142,7 @@ func resolveTLSNode(ctx context.Context, host string, ho *HandleOptions) (node *
if node == nil {
return nil, errors.New("node not available")
}
ho.recorderObject.Node = node.Name
ho.RecorderObject.Node = node.Name
if node.Addr == "" {
node = &chain.Node{
Name: node.Name,
@@ -154,12 +154,12 @@ func resolveTLSNode(ctx context.Context, host string, ho *HandleOptions) (node *
// dialTLS selects a node and establishes a TLS connection.
func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (node *chain.Node, cc net.Conn, err error) {
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
if node = ho.node; node != nil {
if node = ho.Node; node != nil {
cc, err = dial(ctx, "tcp", node.Addr)
return
}
@@ -169,7 +169,7 @@ func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (
return
}
ro := ho.recorderObject
ro := ho.RecorderObject
addr := node.Addr
network := "tcp"
if opts := node.Options(); opts != nil {
@@ -188,19 +188,19 @@ func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (
}
ro.Host = addr
ho.log = ho.log.WithFields(map[string]any{
ho.Log = ho.Log.WithFields(map[string]any{
"host": host,
"node": node.Name,
"dst": fmt.Sprintf("%s/%s", addr, network),
})
ho.log.Debugf("find node for host %s -> %s(%s)", host, node.Name, addr)
ho.Log.Debugf("find node for host %s -> %s(%s)", host, node.Name, addr)
cc, err = dial(ctx, network, addr)
if err != nil {
if marker := node.Marker(); marker != nil {
marker.Mark()
}
ho.log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
ho.Log.Warnf("connect to node %s(%s) failed: %v", node.Name, node.Addr, err)
return
}
@@ -217,8 +217,8 @@ func (h *Sniffer) dialTLS(ctx context.Context, host string, ho *HandleOptions) (
// dynamically generated certificate. The decrypted traffic is then handled
// as HTTP.
func (h *Sniffer) terminateTLS(ctx context.Context, conn, cc net.Conn, clientHello *dissector.ClientHelloInfo, ho *HandleOptions) error {
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
nextProtos := clientHello.SupportedProtos
if h.NegotiatedProtocol != "" {
@@ -317,14 +317,14 @@ func (h *Sniffer) terminateTLS(ctx context.Context, conn, cc net.Conn, clientHel
}
opts := []HandleOption{
WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
sniffing.WithDial(func(ctx context.Context, network, address string) (net.Conn, error) {
return clientConn, nil
}),
WithHTTPKeepalive(true),
WithNode(ho.node),
WithBypass(ho.bypass),
WithRecorderObject(ro),
WithLog(log),
sniffing.WithHTTPKeepalive(true),
sniffing.WithNode(ho.Node),
sniffing.WithBypass(ho.Bypass),
sniffing.WithRecorderObject(ro),
sniffing.WithLog(log),
}
return h.HandleHTTP(ctx, serverConn, opts...)
}
+48
View File
@@ -0,0 +1,48 @@
package sniffing
import (
"context"
"net"
"sync"
)
// HandleFunc handles a connection whose protocol has been detected as proto.
// The receiver s carries the per-connection Sniffer state (MITM certificate,
// HTTP cache, recorder, etc.).
type HandleFunc func(s *Sniffer, ctx context.Context, network string, conn net.Conn, opts ...HandleOption) error
var protoHandlers sync.Map // map[string]HandleFunc
// Register associates a protocol with its handler. Duplicate registration
// panics, mirroring the house registry pattern.
func Register(proto string, fn HandleFunc) {
if _, dup := protoHandlers.LoadOrStore(proto, fn); dup {
panic("sniffing: duplicate protocol handler: " + proto)
}
}
// Get returns the handler registered for proto.
func Get(proto string) (HandleFunc, bool) {
v, ok := protoHandlers.Load(proto)
if !ok {
return nil, false
}
return v.(HandleFunc), true
}
// Dispatch routes a detected protocol to its registered handler. It returns
// (false, nil) when no handler is registered for proto, signalling the caller
// to fall through to raw forwarding.
func Dispatch(s *Sniffer, ctx context.Context, network string, conn net.Conn, proto string, opts ...HandleOption) (handled bool, err error) {
fn, ok := Get(proto)
if !ok {
return false, nil
}
return true, fn(s, ctx, network, conn, opts...)
}
func init() {
Register(ProtoHTTP, (*Sniffer).HandleHTTP)
Register(ProtoTLS, (*Sniffer).HandleTLS)
Register(ProtoRedis, (*Sniffer).HandleRedis)
}
+84
View File
@@ -0,0 +1,84 @@
package sniffing
import (
"context"
"net"
"testing"
)
// TestRegisterGet verifies a handler can be registered and looked up by proto.
func TestRegisterGet(t *testing.T) {
const proto = "testproto-get"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ string, _ net.Conn, _ ...HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
fn, ok := Get(proto)
if !ok {
t.Fatal("Get: handler not found")
}
if err := fn(&Sniffer{}, context.Background(), "tcp", nil); err != nil {
t.Fatalf("fn() error = %v", err)
}
if !called {
t.Fatal("registered handler was not called")
}
if _, ok := Get("no-such-proto"); ok {
t.Fatal("Get: unexpected handler for unknown proto")
}
}
// TestRegisterDuplicate verifies registering the same proto twice panics.
func TestRegisterDuplicate(t *testing.T) {
const proto = "testproto-dup"
Register(proto, func(*Sniffer, context.Context, string, net.Conn, ...HandleOption) error {
return nil
})
defer protoHandlers.Delete(proto)
defer func() {
if recover() == nil {
t.Fatal("expected panic on duplicate registration")
}
}()
Register(proto, func(*Sniffer, context.Context, string, net.Conn, ...HandleOption) error {
return nil
})
}
// TestDispatch_Unknown verifies Dispatch falls through (false, nil) for an
// unregistered proto.
func TestDispatch_Unknown(t *testing.T) {
handled, err := Dispatch(&Sniffer{}, context.Background(), "tcp", nil, "no-such-proto")
if handled {
t.Fatal("Dispatch: unknown proto handled = true, want false")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
}
// TestDispatch_Known verifies Dispatch routes a registered proto to its handler.
func TestDispatch_Known(t *testing.T) {
const proto = "testproto-known"
called := false
Register(proto, func(s *Sniffer, _ context.Context, _ string, _ net.Conn, _ ...HandleOption) error {
called = true
return nil
})
defer protoHandlers.Delete(proto)
handled, err := Dispatch(&Sniffer{}, context.Background(), "tcp", nil, proto)
if !handled {
t.Fatal("Dispatch: known proto handled = false, want true")
}
if err != nil {
t.Fatalf("Dispatch: err = %v, want nil", err)
}
if !called {
t.Fatal("Dispatch: registered handler was not called")
}
}
+44 -13
View File
@@ -10,6 +10,8 @@ import (
"time"
"github.com/go-gost/core/bypass"
"github.com/go-gost/core/chain"
"github.com/go-gost/core/hop"
"github.com/go-gost/core/logger"
"github.com/go-gost/core/recorder"
"github.com/go-gost/x/internal/util/httpcache"
@@ -33,58 +35,87 @@ const (
var DefaultCertPool = tls_util.NewMemoryCertPool()
// HandleOptions holds configuration options for sniffing handlers.
// Fields are exported because forwarder's protocol handlers set them
// across the package boundary.
type HandleOptions struct {
service string
dial func(ctx context.Context, network, address string) (net.Conn, error)
dialTLS func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error)
Service string
Dial func(ctx context.Context, network, address string) (net.Conn, error)
DialTLS func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error)
bypass bypass.Bypass
recorderObject *xrecorder.HandlerRecorderObject
log logger.Logger
HTTPKeepalive bool
ReadTimeout time.Duration
Node *chain.Node
Hop hop.Hop
Bypass bypass.Bypass
RecorderObject *xrecorder.HandlerRecorderObject
Log logger.Logger
}
// HandleOption configures HandleOptions for sniffing handlers.
type HandleOption func(opts *HandleOptions)
// WithService sets the service name for bypass lookups.
// WithService sets the service name for bypass and selection lookups.
func WithService(service string) HandleOption {
return func(opts *HandleOptions) {
opts.service = service
opts.Service = service
}
}
// WithDial sets the dial function used to establish upstream connections.
func WithDial(dial func(ctx context.Context, network, address string) (net.Conn, error)) HandleOption {
return func(opts *HandleOptions) {
opts.dial = dial
opts.Dial = dial
}
}
// WithDialTLS sets the dial function used for TLS-wrapped upstream connections.
func WithDialTLS(dialTLS func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error)) HandleOption {
return func(opts *HandleOptions) {
opts.dialTLS = dialTLS
opts.DialTLS = dialTLS
}
}
// WithHTTPKeepalive enables or disables HTTP keep-alive on the upstream connection.
func WithHTTPKeepalive(keepalive bool) HandleOption {
return func(opts *HandleOptions) {
opts.HTTPKeepalive = keepalive
}
}
// WithNode sets a pre-resolved chain node to connect to, bypassing hop selection.
func WithNode(node *chain.Node) HandleOption {
return func(opts *HandleOptions) {
opts.Node = node
}
}
// WithHop sets the hop used for node selection when routing requests.
func WithHop(h hop.Hop) HandleOption {
return func(opts *HandleOptions) {
opts.Hop = h
}
}
// WithBypass sets the bypass rules for filtering requests by host.
func WithBypass(bypass bypass.Bypass) HandleOption {
return func(opts *HandleOptions) {
opts.bypass = bypass
opts.Bypass = bypass
}
}
// WithRecorderObject sets the recorder object for capturing traffic metadata.
func WithRecorderObject(ro *xrecorder.HandlerRecorderObject) HandleOption {
return func(opts *HandleOptions) {
opts.recorderObject = ro
opts.RecorderObject = ro
}
}
// WithLog sets the logger for the handler.
func WithLog(log logger.Logger) HandleOption {
return func(opts *HandleOptions) {
opts.log = log
opts.Log = log
}
}
+3 -3
View File
@@ -33,14 +33,14 @@ func (h *Sniffer) serveH2(ctx context.Context, network string, conn net.Conn, ho
return errors.New("h2: invalid client preface")
}
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
ro.Time = time.Time{}
tr := &http2.Transport{
DialTLSContext: func(ctx context.Context, nw, addr string, cfg *tls.Config) (net.Conn, error) {
if dial := ho.dialTLS; dial != nil {
if dial := ho.DialTLS; dial != nil {
return dial(ctx, network, addr, cfg)
}
+4 -4
View File
@@ -44,13 +44,13 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, network string, conn net.Conn,
return err
}
log := ho.log
log := ho.Log
if log.IsLevelEnabled(logger.TraceLevel) {
dump, _ := httputil.DumpRequest(req, false)
log.Trace(string(dump))
}
ro := ho.recorderObject
ro := ho.RecorderObject
// Copy ro so that all internal recording (cache-hit serveCachedResponse
// and cache-miss httpRoundTrip) happens on a local clone, preventing
@@ -74,12 +74,12 @@ func (h *Sniffer) HandleHTTP(ctx context.Context, network string, conn net.Conn,
ro.Host = host
log = log.WithFields(map[string]any{"host": host})
if ho.bypass != nil && ho.bypass.Contains(ctx, network, host, bypass.WithService(ho.service)) {
if ho.Bypass != nil && ho.Bypass.Contains(ctx, network, host, bypass.WithService(ho.Service)) {
return xbypass.ErrBypass
}
}
dialFn := ho.dial
dialFn := ho.Dial
if dialFn == nil {
dialFn = (&net.Dialer{}).DialContext
}
+4 -4
View File
@@ -153,14 +153,14 @@ func (h *Sniffer) HandleRedis(ctx context.Context, network string, conn net.Conn
}
buf := new(bytes.Buffer)
if err := ParseRedisMetadata(io.TeeReader(conn, buf), ho.recorderObject); err != nil {
if err := ParseRedisMetadata(io.TeeReader(conn, buf), ho.RecorderObject); err != nil {
return err
}
log := ho.log
ro := ho.recorderObject
log := ho.Log
ro := ho.RecorderObject
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
+55 -24
View File
@@ -5,6 +5,7 @@ import (
"bytes"
"context"
"crypto/tls"
"encoding/json"
"errors"
"io"
"net"
@@ -30,6 +31,16 @@ func (n *noopRecorder) Record(_ context.Context, _ []byte, _ ...recorder.RecordO
return nil
}
// captureRecorder records every payload written through recorder.Recorder.
type captureRecorder struct {
data [][]byte
}
func (r *captureRecorder) Record(_ context.Context, b []byte, _ ...recorder.RecordOption) error {
r.data = append(r.data, append([]byte(nil), b...))
return nil
}
type mockBypass struct {
contains bool
whitelist bool
@@ -177,8 +188,8 @@ func TestWrapErr(t *testing.T) {
func TestWithService(t *testing.T) {
opts := &HandleOptions{}
WithService("mysvc")(opts)
if opts.service != "mysvc" {
t.Errorf("service = %q, want %q", opts.service, "mysvc")
if opts.Service != "mysvc" {
t.Errorf("service = %q, want %q", opts.Service, "mysvc")
}
}
@@ -188,7 +199,7 @@ func TestWithDial(t *testing.T) {
return nil, nil
}
WithDial(dial)(opts)
if opts.dial == nil {
if opts.Dial == nil {
t.Error("dial should be set")
}
}
@@ -199,7 +210,7 @@ func TestWithDialTLS(t *testing.T) {
return nil, nil
}
WithDialTLS(dialTLS)(opts)
if opts.dialTLS == nil {
if opts.DialTLS == nil {
t.Error("dialTLS should be set")
}
}
@@ -208,7 +219,7 @@ func TestWithBypass(t *testing.T) {
opts := &HandleOptions{}
bp := &mockBypass{contains: true}
WithBypass(bp)(opts)
if opts.bypass != bp {
if opts.Bypass != bp {
t.Errorf("bypass not set")
}
}
@@ -217,7 +228,7 @@ func TestWithRecorderObject(t *testing.T) {
opts := &HandleOptions{}
ro := &xrecorder.HandlerRecorderObject{}
WithRecorderObject(ro)(opts)
if opts.recorderObject != ro {
if opts.RecorderObject != ro {
t.Errorf("recorderObject not set")
}
}
@@ -226,7 +237,7 @@ func TestWithLog(t *testing.T) {
opts := &HandleOptions{}
log := xlogger.Nop()
WithLog(log)(opts)
if opts.log != log {
if opts.Log != log {
t.Errorf("log not set")
}
}
@@ -242,9 +253,10 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
}))
defer upstream.Close()
rec := &captureRecorder{}
h := &Sniffer{
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
Recorder: rec,
}
ro := &xrecorder.HandlerRecorderObject{}
@@ -278,16 +290,35 @@ func TestHandleHTTP_BasicProxy(t *testing.T) {
t.Errorf("status = %d, want %d", resp.StatusCode, http.StatusOK)
}
clientConn.Close()
if err := <-errCh; err != nil {
t.Logf("HandleHTTP returned: %v", err)
// Drain the body so HandleHTTP's response write completes before the
// client side closes (net.Pipe writes block until read).
body, err := io.ReadAll(resp.Body)
if err != nil {
t.Fatalf("read body: %v", err)
}
if string(body) != "OK from upstream" {
t.Errorf("body = %q, want %q", body, "OK from upstream")
}
if ro.SrcAddr == "" {
t.Error("SrcAddr should be populated")
clientConn.Close()
if err := <-errCh; err != nil {
t.Errorf("HandleHTTP returned: %v", err)
}
if ro.DstAddr == "" {
t.Error("DstAddr should be populated")
// HandleHTTP records on an internal clone of ro and publishes it via
// h.Recorder, so assert on the captured payload.
if len(rec.data) != 1 {
t.Fatalf("expected 1 recorder event, got %d", len(rec.data))
}
var recorded xrecorder.HandlerRecorderObject
if err := json.Unmarshal(rec.data[0], &recorded); err != nil {
t.Fatalf("unmarshal recorder event: %v", err)
}
if recorded.SrcAddr == "" {
t.Error("recorded srcAddr should be populated")
}
if recorded.DstAddr == "" {
t.Error("recorded dstAddr should be populated")
}
}
@@ -416,8 +447,8 @@ func TestCopyWebsocketFrame_Basic(t *testing.T) {
}
var frame bytes.Buffer
frame.WriteByte(0x81) // FIN + text opcode
frame.WriteByte(0x82) // MASK + len=2
frame.WriteByte(0x81) // FIN + text opcode
frame.WriteByte(0x82) // MASK + len=2
frame.Write(mask)
frame.Write(maskedPayload)
@@ -664,8 +695,8 @@ func TestServeH2_InvalidPreface(t *testing.T) {
Recorder: &noopRecorder{},
}
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
clientConn, serverConn := net.Pipe()
@@ -692,8 +723,8 @@ func TestServeH2_ShortRead(t *testing.T) {
Recorder: &noopRecorder{},
}
ho := &HandleOptions{
log: xlogger.Nop(),
recorderObject: &xrecorder.HandlerRecorderObject{},
Log: xlogger.Nop(),
RecorderObject: &xrecorder.HandlerRecorderObject{},
}
clientConn, serverConn := net.Pipe()
@@ -1058,9 +1089,9 @@ func TestHandleTLS_NonTLSData(t *testing.T) {
func TestHandleUpgradeResponse_WebsocketEnabled(t *testing.T) {
h := &Sniffer{
Websocket: true,
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
Websocket: true,
ReadTimeout: 5 * time.Second,
Recorder: &noopRecorder{},
}
ro := &xrecorder.HandlerRecorderObject{}
+8 -8
View File
@@ -36,9 +36,9 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
return err
}
log := ho.log
log := ho.Log
ro := ho.recorderObject
ro := ho.RecorderObject
ro.TLS = &xrecorder.TLSRecorderObject{
ServerName: clientHello.ServerName,
ClientHello: hex.EncodeToString(buf.Bytes()),
@@ -56,11 +56,11 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
ro.Host = host
}
if ho.bypass != nil && ho.bypass.Contains(ctx, network, host, bypass.WithService(ho.service)) {
if ho.Bypass != nil && ho.Bypass.Contains(ctx, network, host, bypass.WithService(ho.Service)) {
return xbypass.ErrBypass
}
dial := ho.dial
dial := ho.Dial
if dial == nil {
dial = (&net.Dialer{}).DialContext
}
@@ -79,7 +79,7 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
if host == "" {
host = ro.Host
}
if h.MitmBypass == nil || !h.MitmBypass.Contains(ctx, network, host, bypass.WithService(ho.service)) {
if h.MitmBypass == nil || !h.MitmBypass.Contains(ctx, network, host, bypass.WithService(ho.Service)) {
return h.terminateTLS(ctx, network, xnet.NewReadWriteConn(io.MultiReader(buf, conn), conn, conn), cc, clientHello, &ho)
}
}
@@ -128,8 +128,8 @@ func (h *Sniffer) HandleTLS(ctx context.Context, network string, conn net.Conn,
// dynamically generated certificate. The decrypted traffic is then handled
// as HTTP.
func (h *Sniffer) terminateTLS(ctx context.Context, network string, conn, cc net.Conn, clientHello *dissector.ClientHelloInfo, ho *HandleOptions) error {
ro := ho.recorderObject
log := ho.log
ro := ho.RecorderObject
log := ho.Log
nextProtos := clientHello.SupportedProtos
if h.NegotiatedProtocol != "" {
@@ -234,7 +234,7 @@ func (h *Sniffer) terminateTLS(ctx context.Context, network string, conn, cc net
WithDialTLS(func(ctx context.Context, network, address string, cfg *tls.Config) (net.Conn, error) {
return clientConn, nil
}),
WithBypass(ho.bypass),
WithBypass(ho.Bypass),
WithRecorderObject(ro),
WithLog(log),
}
+19 -6
View File
@@ -2,6 +2,7 @@ package ws
import (
"context"
"io"
"net"
"sync"
"time"
@@ -21,7 +22,7 @@ type WebsocketConn interface {
type websocketConn struct {
*websocket.Conn
rb []byte
r io.Reader
ctx context.Context
mux sync.Mutex
}
@@ -44,12 +45,24 @@ func ContextConn(ctx context.Context, conn *websocket.Conn) WebsocketConn {
}
func (c *websocketConn) Read(b []byte) (n int, err error) {
if len(c.rb) == 0 {
_, c.rb, err = c.Conn.ReadMessage()
for {
if c.r == nil {
_, c.r, err = c.Conn.NextReader()
if err != nil {
return 0, err
}
}
n, err = c.r.Read(b)
if err == io.EOF {
c.r = nil
if n > 0 {
return n, nil
}
continue
}
return n, err
}
n = copy(b, c.rb)
c.rb = c.rb[n:]
return
}
func (c *websocketConn) Write(b []byte) (n int, err error) {
+6
View File
@@ -31,6 +31,12 @@ func (l *redirectListener) listenUDP(addr string) (*net.UDPConn, error) {
lc := net.ListenConfig{
Control: func(network, address string, c syscall.RawConn) error {
return c.Control(func(fd uintptr) {
// Must match dialUDP's socket so per-packet tproxy sockets can
// bind a destination port that overlaps this listener without
// "address already in use" (gost#413).
if err := unix.SetsockoptInt(int(fd), unix.SOL_SOCKET, unix.SO_REUSEADDR, 1); err != nil {
l.logger.Errorf("SetsockoptInt(SOL_SOCKET, SO_REUSEADDR, 1): %v", err)
}
if err := unix.SetsockoptInt(int(fd), unix.SOL_IP, unix.IP_TRANSPARENT, 1); err != nil {
l.logger.Errorf("SetsockoptInt(SOL_IP, IP_TRANSPARENT, 1): %v", err)
}
+37 -7
View File
@@ -62,8 +62,17 @@ func LevelOption(level logger.LogLevel) Option {
// slogLogger implements logger.Logger backed by log/slog.
type slogLogger struct {
// base is the pristine slog.Logger before any WithFields attrs are
// applied. It is used to rebuild logger after each WithFields call.
base *slog.Logger
// logger is the effective slog.Logger used for emitting entries.
logger *slog.Logger
level *slog.LevelVar
// args holds the accumulated WithFields key-value pairs in insertion
// order, deduplicated by key (last write wins). slog.Logger.With does not
// deduplicate, so chaining WithFields with a repeated key would otherwise
// emit duplicate JSON keys.
args []any
level *slog.LevelVar
}
// NewLogger creates a new logger.Logger backed by log/slog.
@@ -90,6 +99,7 @@ func NewLogger(opts ...Option) logger.Logger {
}
return &slogLogger{
base: sl,
logger: sl,
level: levelVar,
}
@@ -186,18 +196,38 @@ func revertLevel(lvl slog.Level) logger.LogLevel {
}
// WithFields returns a new Logger with the given fields attached to every
// subsequent log entry.
// subsequent log entry. A field whose key was already set by an earlier
// WithFields call is overwritten (last write wins) rather than duplicated.
func (l *slogLogger) WithFields(fields map[string]any) logger.Logger {
attrs := make([]any, 0, len(fields)*2)
for k, v := range fields {
attrs = append(attrs, k, v)
}
args := mergeArgs(l.args, fields)
return &slogLogger{
logger: l.logger.With(attrs...),
base: l.base,
logger: l.base.With(args...),
args: args,
level: l.level,
}
}
// mergeArgs merges fields into args (a flat [key, value, ...] slice),
// overwriting any existing key in place to preserve insertion order.
func mergeArgs(args []any, fields map[string]any) []any {
index := make(map[string]int, len(args)/2+len(fields))
out := make([]any, 0, len(args)+len(fields)*2)
for i := 0; i < len(args); i += 2 {
index[args[i].(string)] = len(out)
out = append(out, args[i], args[i+1])
}
for k, v := range fields {
if i, ok := index[k]; ok {
out[i+1] = v
} else {
index[k] = len(out)
out = append(out, k, v)
}
}
return out
}
// Trace logs at Trace level.
func (l *slogLogger) Trace(args ...any) {
l.log(levelTrace, args...)
+20
View File
@@ -4,6 +4,7 @@ import (
"bytes"
"log/slog"
"regexp"
"strings"
"testing"
"time"
@@ -184,6 +185,25 @@ func TestWithFields(t *testing.T) {
}
}
func TestWithFieldsDeduplicatesKeys(t *testing.T) {
var buf bytes.Buffer
l := NewLogger(
OutputOption(&buf),
LevelOption(logger.InfoLevel),
FormatOption(logger.JSONFormat),
)
l = l.WithFields(map[string]any{"kind": "service", "service": "s"}).WithFields(map[string]any{"kind": "handler"})
l.Info("hello")
out := buf.String()
// "kind" must appear exactly once, with the last value winning.
if strings.Count(out, `"kind"`) != 1 {
t.Errorf("expected one %q key, got %q", "kind", out)
}
if !strings.Contains(out, `"kind":"handler"`) {
t.Errorf("expected kind=handler, got %q", out)
}
}
func TestIsLevelEnabled(t *testing.T) {
l := NewLogger(LevelOption(logger.InfoLevel))
if !l.IsLevelEnabled(logger.WarnLevel) {
+39 -28
View File
@@ -30,39 +30,50 @@ func (f *failFilter[T]) Filter(ctx context.Context, vs ...T) []T {
}
var l []T
for _, v := range vs {
maxFails := f.maxFails
failTimeout := f.failTimeout
if mi, _ := any(v).(metadata.Metadatable); mi != nil {
if md := mi.Metadata(); md != nil {
if md.IsExists(labelMaxFails) {
maxFails = mdutil.GetInt(md, labelMaxFails)
}
if md.IsExists(labelFailTimeout) {
failTimeout = mdutil.GetDuration(md, labelFailTimeout)
}
}
if !IsFailed(v, f.maxFails, f.failTimeout) {
l = append(l, v)
}
if maxFails <= 0 {
maxFails = 1
}
if failTimeout <= 0 {
failTimeout = DefaultFailTimeout
}
if mi, _ := any(v).(selector.Markable); mi != nil {
if marker := mi.Marker(); marker != nil {
if marker.Count() < int64(maxFails) ||
time.Since(marker.Time()) >= failTimeout {
l = append(l, v)
}
continue
}
}
l = append(l, v)
}
return l
}
// IsFailed reports whether v is currently marked failed: its failure count
// has reached maxFails and the last failure is still within failTimeout.
// Per-item metadata keys "maxFails" and "failTimeout" override the given
// defaults. Items without a failure marker are never considered failed.
func IsFailed[T any](v T, maxFails int, failTimeout time.Duration) bool {
maxFails, failTimeout = effectiveFailSettings(v, maxFails, failTimeout)
mi, _ := any(v).(selector.Markable)
if mi == nil || mi.Marker() == nil {
return false
}
marker := mi.Marker()
return !(marker.Count() < int64(maxFails) || time.Since(marker.Time()) >= failTimeout)
}
// effectiveFailSettings resolves the maxFails/failTimeout for v, honoring
// per-item metadata overrides and defaulting to 1 / DefaultFailTimeout.
func effectiveFailSettings[T any](v T, maxFails int, failTimeout time.Duration) (int, time.Duration) {
if mi, _ := any(v).(metadata.Metadatable); mi != nil {
if md := mi.Metadata(); md != nil {
if md.IsExists(labelMaxFails) {
maxFails = mdutil.GetInt(md, labelMaxFails)
}
if md.IsExists(labelFailTimeout) {
failTimeout = mdutil.GetDuration(md, labelFailTimeout)
}
}
}
if maxFails <= 0 {
maxFails = 1
}
if failTimeout <= 0 {
failTimeout = DefaultFailTimeout
}
return maxFails, failTimeout
}
type backupFilter[T any] struct{}
// BackupFilter filters the backup objects.
+30
View File
@@ -264,3 +264,33 @@ func TestFilterMetadataLabels(t *testing.T) {
t.Fatalf("failTimeout: expected 30s, got %v", ft)
}
}
// --- IsFailed shared helper tests ---
func TestIsFailed(t *testing.T) {
item := newTestMarkable(nil)
if IsFailed(item, 1, 10*time.Second) {
t.Fatal("unmarked item should not be failed")
}
item.Marker().Mark()
if !IsFailed(item, 1, 10*time.Second) {
t.Fatal("marked item with maxFails=1 should be failed")
}
if IsFailed(item, 2, 10*time.Second) {
t.Fatal("single mark below maxFails=2 should not be failed")
}
// Per-item metadata override beats the given defaults.
item2 := newTestMarkable(map[string]any{"maxFails": 1, "failTimeout": "60s"})
item2.Marker().Mark()
if !IsFailed(item2, 5, 10*time.Second) {
t.Fatal("metadata maxFails override should take precedence")
}
// Non-markable items are never failed.
if IsFailed(42, 1, time.Second) {
t.Fatal("non-markable item should not be failed")
}
}