45 Commits
Author SHA1 Message Date
Leonov Vladislav 1c42c638a6 recorder: emit periodic session snapshots and byte deltas
Handlers write one recorder entry when a session ends, so long-lived
SOCKS5 tunnels, streaming HTTP responses and UDP associations stay
invisible while they run and report their traffic as one late record.

With recorder.period set, a session now emits interim records on the
interval while it is active and one final record when it ends. Each
record carries the cumulative counters alongside the increments since the
previous one, so a consumer sums records instead of replacing them, plus
sessionID, recordIndex and phase. A session that ends inside one period
still writes a single record, as it does today.

One goroutine per handler samples every live session on the tick and
writes its record straight to the recorder. There is no queue and no
retry loop, and none is needed: a session moves its cursor past a record
only once the recorder has taken it, so a refused write costs no bytes.
They stay in the delta and go out with the next record. Holding on to
what the sink cannot take right now is the sink's job.

Interim and Finish take a context and return an error, so a handler
learns what became of every record it asked for. Start writes nothing, so
it takes neither. Close stops the sampling and reports the sessions its
owner had not finished; nothing is buffered, so there is nothing to
drain.

With recorder.period unset or non-positive the previous behaviour is
kept: one record per session, written inline, in the old shape. An object
its handler never stamped, and one whose record mode is off, is dropped
in both modes.
2026-09-15 15:05:02 +03:00
ginuerzh b2d901e70f x: add Redis TCP sniffing with protocol-based routing
Add RESP protocol detection to the sniffing framework so plaintext
Redis traffic on a port-reuse listener can be identified and routed by
protocol. Sniff() now peeks 1 byte first to match RESP type bytes
(*+-:$) without blocking, then falls through to the existing 5-byte
TLS/HTTP/SSH detection.

ParseRedisMetadata consumes exactly one RESP message via io.TeeReader
and records command/key in HandlerRecorderObject.Redis. HandleRedis in
both the sniffing (injected-dial) and forwarder (hop-selected) packages
replays the captured first message and pipes the rest, so pipelined
commands beyond the first are forwarded transparently.

Wired into all 5 dispatch sites: sni, forward/local, forward/remote,
http CONNECT, and relay tunnel handlers. forwarder paths route via
resolveRedisNode using ProtocolSelectOption(ProtoRedis), enabling
`protocol: redis` hop selection.

Part of two-phase plan (2026-07-29-sniffing-redis-postgres.md);
PostgreSQL deferred to phase 2.
2026-08-01 16:03:23 +08:00
ginuerzh 6b1c6757f6 x: record cache-hit status in HTTP recorder object 2026-07-29 23:02:18 +08:00
ginuerzh 7952b4f55d x: add Gost-Record header for client-controlled recording level
Supports three modes via Gost-Record request header:
- (absent)/full: full recording with body (existing behavior)
- headers: metadata only, skip body/payload capture
- off: no recording at all

Header is stripped before forwarding. Per-request granularity
via HandlerRecorderObject.RecordMode field propagating through
the ro clone chain to all sub-paths (WS frames, H2, sniffing).
2026-07-17 15:39:07 +08:00
ginuerzh 4bc3886825 feat(forwarder): record pre-rewrite HTTP originals in recorder object
Capture client-sent Host/URI/headers/body before node HTTP rewrites are applied, so the recorder sees both the original and forwarded values. Adds a borrowBodyPrefix helper that inspects a body prefix without consuming the stream. Also serializes file recorder writes unconditionally (the underlying io.WriteCloser is not safe for concurrent Write) and drops a debug log from the bodyRegexp matcher.
2026-07-03 20:13:11 +08:00
ginuerzh d4d823df45 fix(recorder): forward per-service recorder metadata to plugin recorders (#412)
User-configured metadata on recorder objects (recorders[].metadata in YAML)
was silently discarded — MetadataRecordOption existed but was never called
anywhere in the codebase, causing plugin (HTTP/gRPC) recorders to always
receive null metadata.

Fix: wrap recorders that carry config metadata in MetadataRecorder, which
auto-appends the metadata to every Record() call. This covers all paths
(handler, router, sniffer, service) with zero call-site changes.
2026-06-21 13:25:27 +08:00
Denis Galeev 9f610fd163 feat service: introduce labels for logs/recorder (#104) 2026-06-13 13:05:14 +08:00
ginuerzh b1b7500c2f test(recorder): add 55 unit tests for file, HTTP, TCP recorders and HandlerRecorderObject
Covers Record, Close idempotency, nil guards, write errors, concurrent writes,
context propagation, short-write loop, JSON round-trip, and all option functions.
2026-05-25 00:15:35 +08:00
ginuerzh 64a16fafbc fix(recorder): propagate context, handle short writes, add nil guards, idempotent Close, and doc comments
- httpRecorder: use http.NewRequestWithContext instead of http.NewRequest
- tcpRecorder: loop Write to avoid short-write data loss
- fileRecorder: guard against nil out in Record and Close
- plugin/grpc, plugin/http: handle json.Marshal errors instead of ignoring
- fileRecorder, redis*Recorder: idempotent Close via sync.Once
- All exported symbols: add doc comments
2026-05-24 23:58:28 +08:00
Simon Hang d2cc07d3ac Pass username to new redis client 2025-10-30 13:26:11 +08:00
Denis Galeev bf70f07419 fix recorder: writes order 2025-08-25 10:03:26 +08:00
ginuerzh b597467858 add context for conn 2025-08-04 19:29:38 +08:00
ginuerzh db21de831a fix xnet.Pipe 2025-08-01 23:00:50 +08:00
ginuerzh d9a2f44a78 add src field for logger and recorder 2025-07-02 21:35:02 +08:00
Denis Galeev 1eaf08c9c1 fix recorders 2025-06-24 11:01:26 +08:00
Denis Galeev e4b1c3c2b5 feat recorder: add rotate options 2025-02-18 15:30:42 +08:00
ginuerzh 489a762811 service graceful shutdown 2025-01-08 23:12:06 +08:00
ginuerzh 06d993023f add traffic reset for stats 2024-12-24 20:11:22 +08:00
ginuerzh 88ba01b3dd add redirect for handler recorder object 2024-10-31 19:14:21 +08:00
ginuerzh 6b932e35bf sniffing websocket frame 2024-10-19 19:36:06 +08:00
ginuerzh 27242d0b66 recorder: add input/output traffic stats for handler recorder object 2024-10-11 20:32:56 +08:00
ginuerzh c42a44abb6 add sniffer utility 2024-10-02 22:51:23 +08:00
ginuerzh 3c8add4b82 add traffic sniffing for handler 2024-09-27 15:55:28 +08:00
ginuerzh dfb6cb95d0 add username for redis 2024-09-25 22:05:19 +08:00
ginuerzh e22aa91571 add route for recorder 2024-09-24 20:20:15 +08:00
ginuerzh e09ec08881 add sid for connector log 2024-09-20 18:20:57 +08:00
ginuerzh e37213ac01 add http body for handler recorder 2024-09-16 19:44:15 +08:00
ginuerzh b39aa63f75 add clientIP for handler recorder object 2024-09-15 18:28:24 +08:00
ginuerzh f681f7aa03 update handler recorder object 2024-09-15 10:51:23 +08:00
ginuerzh 69455ace9d add handler recorder 2024-09-14 23:21:25 +08:00
ginuerzh 118ee91c95 fix error response for web APIs 2024-06-04 22:05:13 +08:00
ginuerzh c959fc2f73 add observer 2024-01-03 20:55:06 +08:00
ginuerzh a7166b8206 add sd for tunnel 2023-10-31 22:59:14 +08:00
ginuerzh 0bef7c0cdf improve tunnel handler 2023-10-27 22:11:11 +08:00
ginuerzh e5fa371586 add options for components 2023-10-26 22:20:10 +08:00
ginuerzh 497915f465 add tunnel handler and connector 2023-10-15 15:39:25 +08:00
ginuerzh 5488076801 add recorder for relay tunnel endpoint 2023-10-14 21:06:03 +08:00
ginuerzh ea585fc25d add reload and plugin support for hop 2023-09-28 21:04:15 +08:00
ginuerzh 1760151500 add HTTP plugin 2023-09-20 22:56:43 +08:00
ginuerzh 674a70cd23 add client ID for plugin service 2023-09-19 22:31:37 +08:00
ginuerzh a743862f23 add recorder for serial handler 2023-09-18 21:13:11 +08:00
ginuerzh a623232cc1 add recorder for serial handler 2023-09-18 09:43:01 +08:00
ginuerzh 32c8188351 add plugin system 2023-04-18 20:52:56 +08:00
ginuerzh ee2c05494c add more options for grpc 2022-12-21 18:17:46 +08:00
ginuerzh 4808441fb3 add recorder 2022-04-11 23:14:20 +08:00