46 Commits
Author SHA1 Message Date
ginuerzh 3e533c61c7 Merge pull request #126: emit periodic session snapshots and byte deltas 2026-10-07 02:08:41 +00:00
Mike-7777777 dc9c7444b9 sniffing,forwarder: authorize every request against the bypass
The bypass was evaluated once per connection, which authorizes the first
request and nothing after it. A keep-alive request naming another host
re-dialed without a check, a request served from the HTTP cache needed no
dial and so passed no check, a request arriving after a reload was still
judged by the policy that had been in force when the connection opened,
and an HTTP/2 :authority was never checked at all. With MITM enabled the
same holes applied to HTTPS, since the decrypted stream is handed to the
HTTP path.

State the invariant instead of patching the sites one by one: every
request forwarded to an upstream is authorized against the bypass in
effect at the time of that request. CheckBypass is that one check, called
from the four request-handling paths in the two sniffer copies. In the
HTTP loop it replaces the pre-loop check rather than adding to it, so the
file ends up with one check site instead of two.

A reload now reaches established connections at their next request. That
is a behaviour change, and it is what closes the cache and reload paths.
Tunnels stay authorized at establishment: a TLS connection without MITM,
a websocket after upgrade and a CONNECT tunnel have no later request to
check.

CheckBypass is nil-receiver safe because h2Handler can be built without
handle options, and a data path should not panic over that.

Measured cost against a 269-pattern bypass: 645 ns for a host that
matches nothing (full wildcard scan), 149 ns for an exact hit.
2026-10-03 18:46:11 +02:00
Leonov Vladislav 1c42c638a6 recorder: emit periodic session snapshots and byte deltas
Handlers write one recorder entry when a session ends, so long-lived
SOCKS5 tunnels, streaming HTTP responses and UDP associations stay
invisible while they run and report their traffic as one late record.

With recorder.period set, a session now emits interim records on the
interval while it is active and one final record when it ends. Each
record carries the cumulative counters alongside the increments since the
previous one, so a consumer sums records instead of replacing them, plus
sessionID, recordIndex and phase. A session that ends inside one period
still writes a single record, as it does today.

One goroutine per handler samples every live session on the tick and
writes its record straight to the recorder. There is no queue and no
retry loop, and none is needed: a session moves its cursor past a record
only once the recorder has taken it, so a refused write costs no bytes.
They stay in the delta and go out with the next record. Holding on to
what the sink cannot take right now is the sink's job.

Interim and Finish take a context and return an error, so a handler
learns what became of every record it asked for. Start writes nothing, so
it takes neither. Close stops the sampling and reports the sessions its
owner had not finished; nothing is buffered, so there is nothing to
drain.

With recorder.period unset or non-positive the previous behaviour is
kept: one record per session, written inline, in the old shape. An object
its handler never stamped, and one whose record mode is off, is dropped
in both modes.
2026-09-15 15:05:02 +03:00
ginuerzh 4ea4efaba9 fix(forwarder, sniffing): assert BasicProxy recording via captured output
Since 486a7766 HandleHTTP records on an internal clone of the passed
recorder object, TestHandleHTTP_BasicProxy's assertions on the original
ro were always empty, failing every run. Assert SrcAddr/DstAddr on the
captured recorder payload instead, and drain the response body before
closing the net.Pipe so HandleHTTP completes the write and exits cleanly.
2026-08-25 22:17:05 +08:00
ginuerzh a38a701dc4 refactor(sniffing): register protocol handlers via init() registries
Replace the 5 duplicated switch statements that dispatch sniffed
connections (forward/local, forward/remote, http CONNECT, sni, relay)
with per-package registries keyed by proto string. Protocol handlers
self-register via init(), following the house registration pattern, so
adding a protocol no longer requires touching every dispatch site.

Unify the two HandleOptions types (forwarder lacked dialTLS; sniffing
lacked hop/node/httpKeepalive) into one exported struct in sniffing;
forwarder aliases it. Unknown protocols fall through to raw forwarding
exactly as before; SSH remains detection-only. redirect/tcp's special
fallback-replay dispatch is intentionally left untouched.

Adds Dispatch helpers and 8 registry tests (register/get/duplicate/
unknown/known) in both packages. Net -151 lines.
2026-08-25 21:35:45 +08:00
ginuerzh b2d901e70f x: add Redis TCP sniffing with protocol-based routing
Add RESP protocol detection to the sniffing framework so plaintext
Redis traffic on a port-reuse listener can be identified and routed by
protocol. Sniff() now peeks 1 byte first to match RESP type bytes
(*+-:$) without blocking, then falls through to the existing 5-byte
TLS/HTTP/SSH detection.

ParseRedisMetadata consumes exactly one RESP message via io.TeeReader
and records command/key in HandlerRecorderObject.Redis. HandleRedis in
both the sniffing (injected-dial) and forwarder (hop-selected) packages
replays the captured first message and pipes the rest, so pipelined
commands beyond the first are forwarded transparently.

Wired into all 5 dispatch sites: sni, forward/local, forward/remote,
http CONNECT, and relay tunnel handlers. forwarder paths route via
resolveRedisNode using ProtocolSelectOption(ProtoRedis), enabling
`protocol: redis` hop selection.

Part of two-phase plan (2026-07-29-sniffing-redis-postgres.md);
PostgreSQL deferred to phase 2.
2026-08-01 16:03:23 +08:00
ginuerzh edb0b5d651 x: extract shared QUIC sniffing helpers, add redirect UDP QUIC sniff
Extract PopulateQUICClientHello/PopulateQUICServerHello into
internal/util/sniffing/quic.go to deduplicate the QUIC ClientHello/ServerHello
population logic that was copy-pasted across forward/local (Handle, handleRawDatagram)
and redirect/udp handlers (3 sites → 2 helpers).

- handler/redirect/udp: add QUIC ClientHello + ServerHello sniff with
  sniffing/sniffing.timeout metadata, populating ro.Proto/ro.TLS for recording.
- handler/forward/local: replace inline QUIC blocks with shared helpers;
  use h.md.bufferSize instead of hardcoded 8192 for sniff reads.
- internal/util/sniffing: add ProtoQUIC constant.

Bump quic-dissector dependency to v0.1.0.
2026-08-01 16:03:23 +08:00
ginuerzh 6b1c6757f6 x: record cache-hit status in HTTP recorder object 2026-07-29 23:02:18 +08:00
ginuerzh 88670baeae x: don't overwrite ro.Host when TLS SNI is empty
The previous empty-SNI fix clobbered ro.Host with the listener's local
address. For handlers that set ro.Host before HandleTLS (SOCKS5 via the
CONNECT address, SS via target), that destroyed the real destination used
for recording and bypass. Now ro.Host is only set when SNI is present;
the caller's pre-set value is left intact otherwise.
2026-07-29 21:42:56 +08:00
ginuerzh 226420a622 x: allow empty TLS SNI in sniffing handlers
Clients connecting to HTTPS by raw IP (e.g. https://1.1.1.1) don't
send SNI per RFC 6066. Both HandleTLS methods errored immediately
on empty SNI, closing the connection. Now they fall through to the
normal routing path — hop selection in the forwarder, or the caller's
dial closure in the sniffing sniffer (e.g. HTTP CONNECT) — which
handles the empty host gracefully or fails naturally.
2026-07-28 23:52:29 +08:00
ginuerzh 486a77665d fix: prevent double-recording when HTTP response cache is enabled
The cache-hit path (serveCachedResponse) recorded on the same HandlerRecorderObject
as the calling handler's deferred Record(), producing two records per request.
Copy ro at HandleHTTP entry so the sniffer's internal recording stays isolated
from the caller's defer. Also zero ro.Time when the sniffer takes over, so the
handler's defer Record() skips the now-redundant connection-level record.

Fixes 10 files across sniffer and all 8 handlers that support sniffed HTTP forwarding.
2026-07-26 20:11:36 +08:00
ginuerzh 3474501784 x: add generic cache component and HTTP response caching
Implements the cache.Cache interface with in-memory backend (TTL,
maxSize/maxBytes bounds, oldest/LRU eviction, background cleanup).
Adds httpcache layer for HTTP response policy (per-status TTL,
serve-stale, method/body filtering). Wires into config parser,
loader, registry, and all four sniffing handlers (forward/local,
forward/remote, sni, http) so HTTP responses are cached before
dialing upstream.

Bumps core dependency to v0.6.0 for the new cache.Cache interface.
2026-07-26 17:26:00 +08:00
ginuerzh 7952b4f55d x: add Gost-Record header for client-controlled recording level
Supports three modes via Gost-Record request header:
- (absent)/full: full recording with body (existing behavior)
- headers: metadata only, skip body/payload capture
- off: no recording at all

Header is stripped before forwarding. Per-request granularity
via HandlerRecorderObject.RecordMode field propagating through
the ro clone chain to all sub-paths (WS frames, H2, sniffing).
2026-07-17 15:39:07 +08:00
ginuerzh a32ffd5608 fix(sniffing): pass bypass option to sniffer HandleHTTP/HandleTLS calls
When sniffing is enabled, the sniffer extracts the real domain from HTTP
Host header or TLS SNI and checks bypass rules against it. However, most
handlers were not passing WithBypass(...) to the sniffer, so the sniffer
internal bypass check was always skipped.

Additionally, the TLS-to-HTTP fallback paths in internal util sniffer
and forwarder also omitted bypass when constructing options for the
recursive HandleHTTP call after decrypting TLS.

Add WithBypass to all sniffer call sites that were missing it across
handlers (http, relay, socks4, socks5, ss, sshd, unix) and internal
TLS fallback paths.

Also fix import ordering in handler/http/connect.go.

Fixes go-gost/gost#874
2026-06-23 20:54:26 +08:00
ginuerzh e814852ca1 fix(sniffing): re-dial upstream on HTTP Host change + surface no-SNI errors
sniffer_http:
When DNS override directs multiple domains to the same proxy IP, the
browser may reuse a keep-alive connection for a different host. The
HTTP keep-alive loop now detects Host header changes and re-dials a
new upstream connection with correct node selection, preventing CDN
errors (Fastly unknown domain, CloudFront 403).

sniffer_tls:
Return a descriptive error when TLS ClientHello has no SNI, instead
of silently returning nil. This makes the connection drop visible in
logs and recorder output.

Fixes go-gost/gost#479
2026-06-21 16:37:47 +08:00
ginuerzh f09a7a69b3 fix(sniffing): return early on empty SNI to avoid 'missing port in address' errors (#645)
When TLS ClientHello has no SNI extension (ServerName is empty),
normalizeHost returns empty string. Both sniffer_tls.go paths (sniffing
and forwarder packages) previously still called dial with an empty host,
cascading 'missing port in address' errors through connector, router,
and service layers.

Now returns early with a debug-level log, silently closing the
connection instead of attempting to dial upstream.
2026-06-05 20:46:00 +08:00
ginuerzh 382d47ea12 fix: nil HTTP recorder ref in WebSocket goroutines, add tunnel loop detection
- handler/http/websocket, forwarder/sniffer_ws, sniffing/sniffer_ws: nil
  ro2.HTTP in WebSocket copy goroutines to avoid data races on the shared
  HTTP recorder object
- handler/tunnel/bind: remove ingress rule check that incorrectly
  overrode the endpoint host for non-matching ingress rules
- handler/tunnel/entrypoint: add forwarding loop detection by checking
  Gost-Forwarded-Node header for the entrypoint's own node ID
2026-06-01 18:28:01 +08:00
ginuerzh cf89192211 feat(forwarder/sniffer): add HTTP request body rewriting, deprecate rewriteBody in favor of rewriteResponseBody
- Add RewriteRequestBody field to HTTPNodeConfig (yaml: rewriteRequestBody)
- Add RewriteRequestBody field to HTTPNodeSettings in core chain
- Add rewriteReqBody function symmetric to rewriteRespBody in sniffer_rewrite.go
- Wire up rewriteReqBody call in httpRoundTrip before body wrapping for recording
- Add new RewriteResponseBody config field, deprecate old RewriteBody
- Parse both RewriteRequestBody and RewriteResponseBody in node config parsing
- Add 10 test cases covering nil, skip, content-type filter, and chained rewrites
- Bump core dependency to v0.4.1
- Fix: clone response header map in internal/util/sniffing/sniffer_http.go
2026-05-31 19:46:41 +08:00
ginuerzh 722dde5cfc fix(handler/forward): split readTimeout from pipe idleTimeout to prevent 15s download abort
The readTimeout field (default 15s) was being applied via xnet.Pipe to both
directions of a bidirectional proxy connection. During asymmetric transfers
(e.g. HTTP file download), the direction reading from the tunnel sees no
data after the initial request is forwarded, causing SetReadDeadline to fire
after 15s and abort the entire transfer.

Fix: add a separate idleTimeout field (default 0=disabled) to the metadata
structs in both forward/local and forward/remote handlers, and switch
xnet.Pipe to use idleTimeout instead of readTimeout. The readTimeout field
now only applies to the initial protocol sniffing/handshake phase.

Also document readTimeout vs idleTimeout semantics across all 24 locations
in the x/ module where these timeouts appear:
- readTimeout: handshake sniffing deadline (handlers), upstream response
  header timeout (http.Transport), or transport-level read deadline
- idleTimeout: idle read deadline per Pipe direction (0=disabled)
- ReadTimeout on Sniffer/SnifferBuilder: upstream response header/TLS
  handshake read timeout during sniffing
2026-05-31 17:05:21 +08:00
ginuerzh 23b58ddd23 refactor(handler/http): extract Authenticator, SnifferBuilder, and normalizeRequest as testable units
Extract three pure-logic components from httpHandler to eliminate I/O
coupling from auth and sniffing construction, enabling synchronous unit
tests and reducing per-request allocation in sniffAndHandle:

- Authenticator struct with AuthResult return type — auth decisions no
  longer write to the connection; callers handle response I/O. Auth
  tests drop net.Pipe/goroutines for direct return-value assertions.
- SnifferBuilder pre-built in Init and reused per-connection via Build(),
  replacing inline sniffing.Sniffer{} construction in sniffAndHandle.
- normalizeRequest extracted to util.go with NormalizedRequest type,
  collapsing 20 lines of inline URL/normalisation into a single call.
- knockMatch extracted as standalone pure function.
- clampBodySize exported as ClampBodySize for cross-package use.
- util_test.go with 22 tests covering utility functions.
- helpers_test.go consolidates shared test fakes (logger, observer, conn).
2026-05-29 00:14:44 +08:00
ginuerzh a2cef9639d test(sniffing): add 20 unit tests, coverage from 34.2% to 58.9%
Add tests for Sniff/isHTTP protocol detection, setHeader, h2Handler.ServeHTTP
(mock RoundTripper), HandleHTTP bypass/body-recording, HandleTLS error path,
handleUpgradeResponse websocket path, and copyWebsocketFrame read error.
2026-05-27 21:30:52 +08:00
ginuerzh 4f526c2aee refactor(sniffing): split sniffer monolith into focused files, extract testable helpers
Split ~850-line sniffer.go into 6 focused files (http, h2, tls, ws,
rewrite, test), extract pure helpers (clampBodySize, normalizeHost,
effectiveReadTimeout), fix "DeafultSampleRate" typo, add doc comments
on all exported symbols, and add 22 unit tests.
2026-05-27 21:21:40 +08:00
ginuerzh e2072399dd fix sniffing for websocket 2025-09-04 21:30:49 +08:00
ginuerzh c7d16962ec add service option for plugin 2025-08-29 23:36:31 +08:00
ginuerzh 52289bea6c add network param for sniffing 2025-08-13 21:25:35 +08:00
ginuerzh b597467858 add context for conn 2025-08-04 19:29:38 +08:00
ginuerzh db21de831a fix xnet.Pipe 2025-08-01 23:00:50 +08:00
ginuerzh 208d18125c replace xnet.Transport by xnet.Pipe 2025-07-28 20:52:15 +08:00
ginuerzh 35a049fb03 rename vtun to tungo 2025-07-26 16:17:04 +08:00
ginuerzh d9a2f44a78 add src field for logger and recorder 2025-07-02 21:35:02 +08:00
ginuerzh d819fd3041 support http-100 continue for sniffer 2025-06-24 21:15:38 +08:00
ginuerzh 06d993023f add traffic reset for stats 2024-12-24 20:11:22 +08:00
ginuerzh a12870e766 add compression flag option for http request 2024-11-02 20:41:23 +08:00
ginuerzh 88ba01b3dd add redirect for handler recorder object 2024-10-31 19:14:21 +08:00
ginuerzh a97ac1f30a fix http body recorder 2024-10-22 22:51:33 +08:00
ginuerzh 3ca253537b fix race condition for sniffing 2024-10-20 19:19:16 +08:00
ginuerzh 41f527aa46 sniffing: full websocket frame recording 2024-10-19 20:40:07 +08:00
ginuerzh 6b932e35bf sniffing websocket frame 2024-10-19 19:36:06 +08:00
ginuerzh 7e51404ae5 fix handler sniffing 2024-10-16 23:01:12 +08:00
ginuerzh 51ca9f620a add transport for tunnel entrypoint 2024-10-15 18:47:36 +08:00
ginuerzh 5bba004324 add keepalive option for tunnel entrypoint 2024-10-14 23:09:07 +08:00
ginuerzh 0ffed81601 add sniffer for forwarding handler 2024-10-14 21:18:38 +08:00
ginuerzh 27242d0b66 recorder: add input/output traffic stats for handler recorder object 2024-10-11 20:32:56 +08:00
ginuerzh 8f994ab632 add handle options for sniffer 2024-10-10 22:49:56 +08:00
ginuerzh c42a44abb6 add sniffer utility 2024-10-02 22:51:23 +08:00
ginuerzh 3c8add4b82 add traffic sniffing for handler 2024-09-27 15:55:28 +08:00