feat: add database monitoring and Redis RabbitMQ readonly support
This commit is contained in:
曾志威
@@ -203,7 +203,7 @@ powershell.exe -NoProfile -ExecutionPolicy Bypass -File scripts/package.ps1
|
||||
|
||||
最终交付目录 `bin/releases/<时间与唯一标识>/` 只包含 `linkport.exe`，直接双击即可运行。网页已嵌入程序，不再输出 ZIP、额外工具、脚本、说明文件或校验文件；旧 `-Compression` 参数已移除。版本、提交和构建时间可通过 `linkport.exe --version --json` 查看，EXE 的 SHA-256 显示在打包日志中。前端按源码、配置、依赖锁文件、Node 版本及产物内容校验缓存，变化或缺失时自动重建；依赖变化自动执行 `npm ci`。可用 `.\package.cmd -RebuildWeb` 强制重建前端。缓存位于 `bin/build-cache`。Go 每次构建当前代码并注入版本信息；构建和验证均在临时构建目录完成，只有验证成功才生成最终交付目录。
|
||||
|
||||
`start-dev.cmd` 用于直接启动。源码目录默认快速构建：复用前端依赖和构建缓存，跳过启动前的 Go 全量测试；Go 每次编译当前源码、注入最新构建信息，并复用 Go 编译缓存。先生成 `bin/dev-build/linkport.exe`，成功后才停止当前目录中的旧服务并等待退出，将整个 `data` 备份到 `bin/backups/<时间>/`，再替换并运行 `bin/linkport.exe`。构建失败时保留原服务运行；数据备份始终在停服后进行。日志显示前端、测试（启用时）、Go 构建和总耗时。
|
||||
`start-dev.cmd` 用于直接启动。源码目录默认快速构建：复用前端依赖和构建缓存，跳过启动前的 Go 全量测试；Go 每次检查当前源码和内嵌资源，未变化时直接复用候选 EXE。开发模式保留版本与提交，构建时间显示 `unknown`，避免每次启动因时间戳变化强制重新链接；发布打包仍记录实际构建时间。先生成 `bin/dev-build/linkport.exe`，成功后才停止当前目录中的旧服务并等待退出，将整个 `data` 备份到 `bin/backups/<时间>/`，再替换并运行 `bin/linkport.exe`。构建失败时保留原服务运行；数据备份始终在停服后进行。日志显示前端、测试（启用时）、Go 构建和总耗时。
|
||||
|
||||
```powershell
|
||||
.\start-dev.cmd # 缓存构建并启动
|
||||
|
||||
@@ -12,6 +12,16 @@ Drone 将 linux/amd64、linux/arm64、windows/amd64、windows/arm64、darwin/amd
|
||||
|
||||
`bin/backups` 包含用户数据，不是可随意删除的编译缓存。旧 `artifacts/` 中的测试产物和备份仍保留，不在合并时搬移或清理。验证可使用 `artifacts/` 下的独立程序、临时端口和临时数据库，不覆盖运行中的 `bin/linkport.exe`。
|
||||
|
||||
## 开发构建复用（2026-10-15）
|
||||
|
||||
用户反馈前端缓存命中后，Go 构建仍耗时 90.5 秒。热缓存复测为 12.85 秒，跟踪记录没有编译或下载命令，但仍调用一次链接器：每次注入当前构建时间会使链接参数变化，无法复用现有 EXE。
|
||||
|
||||
`start-dev.cmd` 现在向构建脚本传入 `-Development`，保留版本和提交，将构建时间固定为 `unknown`。每次仍执行 `go build`，由 Go 检查源码、依赖、构建参数及内嵌前端资源；内容未变化时复用 `bin/dev-build/linkport.exe`，变化时正常重新构建。`-FullCheck` 继续运行完整 Go 测试，开发模式同样复用稳定链接参数。直接运行 `scripts/build.ps1` 的默认行为、发布打包及其实际构建时间保持不变，无配置或数据迁移。首次构建和源码变化后的编译、链接仍需耗时。
|
||||
|
||||
实际验证：Go 1.27.1 下首次切换开发参数时 Go 构建 17.1 秒、构建总计 18.5 秒；未修改源码再次运行 `start-dev.ps1 -BuildOnly`，Go 2.7 秒、构建总计 4.0 秒，包含 PowerShell 启动的入口墙钟为 8.54 秒。EXE SHA-256 相同；另一次 `go build -x` 复核为 3.09 秒、零编译和零链接命令。启动脚本隔离回归通过，覆盖开发参数在默认/FullCheck 下传递、开发版本/提交保留和发布实际时间，以及原有锁、失败隔离、停服所有权和备份保护；其中服务程序均为临时 fixture。前端生产构建和 `go vet ./...` 通过，`web/dist/.gitkeep` 已恢复。
|
||||
|
||||
全量 Go 测试仅复现已记录的 cluster MySQL 认证 fixture 空指针及 pgproxy/TestPreparedRejectsDifferentIdentity/account 失败；后者单独复跑通过（6.103 秒），保留全量失败记录。Playwright 使用新构建的 `bin/dev-build/linkport.exe`、18861/18862 端口和临时 SQLite：全量发现 173 项，以 `--max-failures=1 --timeout=15000` 执行，在旧 account-editor 的 `.target-card / 配置目标` 定位器失败后停止，172 项未运行；补测当前 React 主题与认证 2 项通过（10.8 秒）。这些测试不连接真实协议服务器，未运行发布打包或替换正在运行的服务，未修改真实数据。日志与构建跟踪在 `bin/dev-build/build-speed/`。
|
||||
|
||||
## 相对路径与跨机器使用
|
||||
|
||||
脚本不保存开发者盘符、用户目录或固定系统安装位置。项目根目录从脚本自身位置推导，与从哪个目录调用无关；复制整个项目到另一台电脑或带空格的其他目录后，不需要改脚本中的路径。
|
||||
|
||||
@@ -1,5 +1,53 @@
|
||||
# 变更原因与设计记录
|
||||
|
||||
## Redis / RabbitMQ 严格只读与 RabbitMQ 双端口（2026-10-15）
|
||||
|
||||
原因：用户要求 Redis、RabbitMQ 真正支持只读，明确 RabbitMQ 不允许消费或确认消息，并要求同一网关同时提供 AMQP（5672）与管理页面/API（15672）。此前只隐藏不支持的选项不足以满足新需求。
|
||||
|
||||
结果：Redis 单节点和 Cluster 使用命令/子命令白名单，拒绝写入、脚本、事务、订阅和未知命令；单节点只读请求逐条收取响应，保证混合流水线回复顺序。RabbitMQ AMQP 仅允许连接控制、心跳和 passive 检查，拒绝消费、确认、发布、声明/绑定/删除及内容帧，返回 403 并关闭连接。后端账号/网关校验及共用前端选项同时开放 Redis/RabbitMQ 只读；拒绝进入审计和拒绝计数。
|
||||
|
||||
双端口：网关新增可选 managementListenPort，目标新增 managementPort/managementTLS。复用 HTTP 反向代理和 Basic 凭据替换，按网关账号选择对应目标的管理端口；严格只读仅允许内置信息 GET/HEAD 和静态页面，拒绝取消息、写操作及会写入的 aliveness-test。两个监听共同启停，启动或持久化失败回滚端口，管理连接不复用 AMQP 隧道。界面可配置端口/上游 HTTPS，诊断显示管理地址。
|
||||
|
||||
兼容：旧配置管理监听默认关闭，目标管理端口 0 表示 15672，无数据库迁移；旧读写账号不自动改权限。修改运行中的管理监听端口或共用监听地址需先停止；目标设置对新连接生效。管理入口提供 HTTP，目标可用独立 HTTPS；目标需启用 management 插件并具有管理权限。管理接口不做 AMQP 集群发现；启用管理端口时逐个创建网关。Redis 只读回复沿用 Cluster 解析大小限制，未放行的合法扩展命令也会被拒绝。详见 docs/readonly.md。
|
||||
|
||||
验证：协议测试使用 RESP/AMQP/HTTP fixtures 和临时数据库，不是真实 Redis/RabbitMQ。Redis/RabbitMQ/HTTP、model、gateway 专项通过，Node 访问模式/草稿 18 项通过，前端生产构建及最终 go vet 通过。Go 1.27.1 全量测试中 API（含端口持久化）、gateway、全部协议包通过，仍在已有 cluster MySQL 公钥认证 fixture 空指针处失败；本轮 pgproxy 通过。新嵌入程序 bin/dev-build/readonly/linkport.exe 上，React/UI domain 和 Redis 界面 16 项通过；RabbitMQ 浏览器测试最初重复新增账号留下空默认账号，修正 fixture 后 Redis/RabbitMQ 两项均通过（15.8 秒）。全量 Playwright 175 项在已有 account-editor 旧定位器处停止，174 项未运行。日志见 bin/dev-build/readonly。未部署、重启或修改真实数据。
|
||||
|
||||
## Redis 网关访问模式与协议能力对齐（2026-10-15）
|
||||
|
||||
原因：用户创建 Redis 网关时，POST /api/v1/proxies 返回 readonly is currently supported only for MySQL and MongoDB。前端访问模式选项对所有非 SSH/TCP 协议都提供了只读，和后端协议能力不一致。
|
||||
|
||||
结果与范围：共用访问模式选项仅向 MySQL/MongoDB 提供只读，Redis、PostgreSQL、SQL Server、RabbitMQ、HTTP、HTTPS 只提供读写；账号字段校验复用同一组选项，提交前定位不受支持的模式，后端同类错误转为中文说明。MySQL 的全放行和 MongoDB 的只读选项保留；不自动改写草稿权限，服务端限制不变。无需数据迁移，重新构建并重启后使用新前端。详见 readonly.md。
|
||||
|
||||
实际验证：新增回归先复现三处问题，修复后访问模式和草稿 Node 测试 17 项通过；TypeScript/Vite 及 Go 1.27.1 新嵌入构建通过，恢复 web/dist/.gitkeep。新程序在隔离 SQLite、18901/18902 端口的 16 项 Playwright 专项全部通过（46.0 秒），包含 Redis 账号界面与真实 POST 201/readwrite 回读，以及 MySQL 只读账号保存和秘密保留。全量 174 项仍在既有 account-editor 定位器处失败，173 项未运行。后端源码未改，未重复本会话已运行的 Go 全量/vet；既有后端失败记录保留。测试未连接真实协议服务器，未部署或修改真实配置；日志见 bin/dev-build/redis-access-mode。
|
||||
|
||||
## 开发启动避免时间戳强制重新链接（2026-10-15）
|
||||
|
||||
原因：用户报告前端缓存命中后，Go 构建耗时 90.5 秒。热缓存复测 12.85 秒，跟踪显示零编译、零下载，仍执行一次链接；每次生成的新构建时间会改变链接参数，使未变化的候选 EXE 也无法复用。
|
||||
|
||||
行为与范围：Windows 开发启动向 build.ps1 传入 Development，由共用版本信息函数保留版本和提交、将开发构建时间设为 unknown。仍调用原生 go build 检查所有编译输入，未变化时复用候选 EXE，变化时正常重新构建；FullCheck 仍执行完整测试。发布打包和直接构建默认继续注入实际时间，未增加独立源码缓存或绕过编译检查。
|
||||
|
||||
兼容与迁移：开发版版本信息的 buildTime 改为 unknown，首次或输入变化时仍有编译链接开销；无配置、协议、数据结构或数据库迁移。仅构建到候选路径，不替换运行实例或操作真实数据。相关说明见 build-output.md。
|
||||
|
||||
实际验证：Go 1.27.1 下未变化源码的重复构建为 4.0 秒，其中 Go 2.7 秒；完整入口墙钟 8.54 秒，原生跟踪确认零编译、零链接、EXE SHA-256 相同。启动脚本 fixture 回归、前端生产构建和 go vet ./... 通过；新 EXE 的 React 主题/认证 Playwright 2 项通过。全量 Go 测试仍有已记录的 cluster MySQL 认证 fixture 空指针及 pgproxy 身份隔离 fixture 失败，pgproxy 单独复跑通过；浏览器全量在旧 account-editor 定位器失败后停止，172 项未运行。未把 fixture 当作真实协议服务器验证，未运行发布打包。细节和日志见 build-output.md 的开发构建复用记录。
|
||||
|
||||
## 历史慢 SQL 只读函数补齐（2026-10-14）
|
||||
|
||||
原因：历史慢 SQL 使用 UNIX_TIMESTAMP、TIME_TO_SEC、SEC_TO_TIME、LEFT，先前白名单未覆盖，返回 readonly_statement。结果：仅补齐这四个内置函数，保留全部参数递归检查、限定名自定义函数拒绝和 EXPLAIN ANALYZE 拒绝；新增实际参数化慢 SQL 执行计划及嵌套危险函数回归。无配置或数据迁移，升级并重启程序后生效。CSV 日志表索引问题由监控端单独提示，不调整数据库、网关访问模式或扫描保护。
|
||||
|
||||
实际验证：Go 1.27.1 下 mysqlproxy 全包、go vet 和新程序构建通过；全量 Go 测试仅 cluster 已记录的 MySQL 认证 fixture 空指针失败，日志见 bin/slow-sql-go-test.log。已更新本机运行实例，管理健康检查通过，真实 MySQL 的四个最小函数查询及慢 SQL 执行计划成功返回；仅做 SELECT/EXPLAIN，未执行慢日志全表读取或数据库结构修改。LinkPort 前端无改动，沿用本轮会话已完成的前端构建，未重复执行已确认存在旧界面定位失败的浏览器套件；Shopee Sentinel 的新增列表和提示另经浏览器验证。
|
||||
|
||||
## MySQL 只读监控函数兼容（2026-10-14）
|
||||
|
||||
原因：Shopee Sentinel 的时间范围查询使用 FROM_UNIXTIME，权限元数据查询使用 CHAR 和 SUBSTRING_INDEX。三个函数被只读策略拒绝，返回 Error 1290 (HY000): linkport rejected request: readonly_statement。现场最小查询确认普通 EXPLAIN 和 EXPLAIN FORMAT=TRADITIONAL 本身可通过，实际缺口是函数检查。
|
||||
|
||||
行为与范围：MySQL 只读函数白名单增加 FROM_UNIXTIME、SUBSTRING_INDEX，并允许 Vitess 专用 CharExpr 节点，覆盖 CHAR 的 USING 字符集形式。继续递归检查所有参数、子查询和内部 EXPLAIN 语句；嵌套 SLEEP、LOAD_FILE、GET_LOCK、变量赋值、带库名前缀的自定义函数、混入写入的批次及 EXPLAIN ANALYZE 仍拒绝。常用元数据 SELECT 无需单独绕过策略。详见 readonly.md。
|
||||
|
||||
兼容：仅扩展三个只读内置函数的语法支持，无配置或数据库迁移；现有运行实例需更新程序并重启生效。
|
||||
|
||||
验证：先运行新增回归复现项目原始 SQL 被拒绝，再修改策略；修复后 TestReadOnly 系列通过，覆盖原始参数化波次查询及两种 EXPLAIN、权限和索引元数据查询、服务器变量和危险嵌套参数。前端构建与新可执行文件构建通过。以上策略回归不连接真实数据库；未替换运行实例或修改真实目标数据。
|
||||
|
||||
完整检查结果：Go 1.27.1 下 mysqlproxy 全包及 go vet ./... 通过。全量 Go 测试仍遇到已记录的 cluster/TestMySQLReplicationAndGroupDiscovery 认证 fixture 空指针；outbound 的 TestSingBoxProcessManagerSharesAndCleansUp 首次超时，独立复跑通过。Playwright 使用新构建 bin/linkport-readonly-monitoring.exe、18761/18762 端口和临时 SQLite；全量运行在账号编辑旧定位器失败后被会话中断，未完成 173 项。补跑 mysql-access-modes.spec.ts 也在旧界面“按账号设置”文本断言处失败，当前界面使用新的列表/编辑入口，未进入协议查询验证；未扩大本次改动去修复无关旧 UI 测试。日志位于 bin/readonly-monitoring-{go-test,vet,browser,browser-focused}.log。协议测试使用 fixtures，修复版本尚未对真实上游进行部署验证。
|
||||
|
||||
## 网关 PUT 非法字段定位与账号结构修复（2026-10-10）
|
||||
|
||||
原因：用户报告 PUT /api/v1/proxies/:id 返回 invalid configuration fields。代码确认可复现路径为编辑器新增账号时混用了目标与网关账号字段，前端只筛选顶层字段，未筛选账号对象；另外小数最大连接数能通过原有范围校验，却无法解码成后端整数。
|
||||
@@ -1489,6 +1537,14 @@ Actual validation: the Windows PowerShell startup regression suite passed with c
|
||||
## Drone 克隆绕过代理（2026-10-13）
|
||||
|
||||
原因：默认克隆使用流水线代理访问 Git 服务，CONNECT 被代理以 407 拒绝，用户要求 Git 不走代理。最小调整为在 no_proxy/NO_PROXY 中同时加入 git.freeicu.top；保留默认克隆及其他下载代理，不改应用与数据库。本地断言检查两份配置和 git diff --check；未连接远端 Drone，直连可达性需实际构建确认，Runner 自身覆盖代理时也需同步直连名单。详见 docs/build-output.md。
|
||||
## 数据库健康、网关慢请求和压力（2026-10-15）
|
||||
|
||||
原因：需要在同一运维页面判断所有已支持数据库服务是否健康、是否积压以及经过网关的请求是否缓慢。复用现有健康探测，增加 MySQL、PostgreSQL、SQL Server、MongoDB、Redis、RabbitMQ 的固定只读压力采集，以及协议响应完成事件。界面支持手动采集和页面可见时每 15 秒采集；权限不足显示说明。
|
||||
|
||||
按用户确认，慢请求仅来自 LinkPort 流量，不读取上游慢日志。阈值 1 秒，进程内最多 500 条，不存 SQL 参数、键值或消息正文；无明确完成响应的流式、异步请求不计时。压力是数据库进程/统计视图可见值，不等于宿主机系统监控。协议边界和权限见 `gateway-operations.md`。
|
||||
|
||||
影响范围为六种协议、网关内存统计、监控 API 和运维页面；旧健康 API 保持兼容，监控快照仅新增字段，无数据迁移，不修改生产数据或当前运行服务。验证使用本地协议测试桩、隔离 SQLite 和浏览器模拟压力响应，不代表真实服务验收。具体本轮测试结果记录于 `gateway-operations.md`。
|
||||
|
||||
## Drone 移除代理配置（2026-10-13）
|
||||
|
||||
原因：用户已不再使用代理。删除 Drone 流水线全局的六个大小写 HTTP/HTTPS 代理及绕过名单变量，默认克隆、依赖下载和 IPFS 上传均依赖构建环境直接联网。同步更新当前 CI 与 IPFS 网络说明；保留历史记录，不影响应用运行、数据格式或数据库迁移。Runner、Docker daemon 若另有代理配置，需在其部署环境单独清除。验证检查流水线代理字段和差异；未运行远端 Drone 构建。
|
||||
|
||||
@@ -25,8 +25,8 @@ ID 与用户名在同一网关内唯一；密码可以相同。账号 ID 是凭
|
||||
SSH 可使用账号公钥代替密码。
|
||||
|
||||
旧 `gatewayUser/gatewayPassword` 配置仍可使用；一旦配置账号列表，认证仅
|
||||
使用列表，不再回退到旧账号。MySQL、MongoDB 支持账号只读权限，账号无法
|
||||
提升网关本身的只读限制。其他协议不提供只读模式。账号来源限制与网关来源
|
||||
使用列表，不再回退到旧账号。MySQL、MongoDB、Redis、RabbitMQ 支持账号只读权限，账号无法
|
||||
提升网关本身的只读限制。RabbitMQ 严格只读禁止消费，具体边界见 `readonly.md`。账号来源限制与网关来源
|
||||
限制同时生效。每个真实网关账号必须用自己的 `targetId/targetAccountId` 选择同协议目标及其账号，不再从网关默认目标继承。旧顶层映射仍可读取；在界面编辑保存时会展开到各真实账号并清空顶层字段。Redis、MongoDB、TCP 的空网关账号继续使用顶层字段保存，空账号本身就是明确端点。解除网关关联会清空全部账号映射。
|
||||
|
||||
网关账号和目标账号支持 `expiresAt`（RFC3339）及 `rotationReminderDays`（0 默认 7 天，最多 365 天）。到期后拒绝新的认证，不主动终止既有会话。凭据仍需手动轮换；运行中保存轮换结果后，新连接使用新凭据，已有会话不主动终止。
|
||||
@@ -105,6 +105,22 @@ TCP 连接和协议认证分别计数。手动健康检查最多尝试 3 次，
|
||||
|
||||
## 验证范围
|
||||
|
||||
### 数据库健康、慢请求与压力（2026-10-15）
|
||||
|
||||
运维监控增加数据库选择、目标账号选择和“采集健康与压力”。覆盖 MySQL、PostgreSQL、SQL Server、MongoDB、Redis、RabbitMQ；复用现有健康检查，压力采集使用独立连接，最多等待 10 秒。可选每 15 秒采集，仅页面可见时发起，不重叠请求。已有后台健康检查间隔不受影响。
|
||||
|
||||
压力采集仅执行固定只读统计请求：MySQL 的 GLOBAL STATUS/VARIABLES（连接、流量、锁和缓冲池），PostgreSQL 的 pg_stat_database/pg_stat_activity（会话、事务、缓存、锁等待），SQL Server DMV（会话、请求、调度队列、进程内存），MongoDB serverStatus（连接、内存、操作和锁队列），Redis INFO（连接、内存、吞吐、累计 CPU 秒数），RabbitMQ 管理 API overview/nodes（积压、内存、磁盘、文件描述符及告警）。RabbitMQ 使用目标管理端口，默认 15672，可独立启用 TLS，不要求开启网关管理监听。
|
||||
|
||||
统计权限由目标账号决定；MongoDB 通常需要 clusterMonitor，PostgreSQL 完整统计需要 pg_monitor/pg_read_all_stats，SQL Server 需要对应版本的 VIEW SERVER STATE 或 VIEW SERVER PERFORMANCE STATE，RabbitMQ 节点指标需要 monitoring/administrator。缺失指标给出提示，不伪造为零，不自动授予权限或修改配置。数据库统计不等于宿主机 CPU 利用率；累计值及集群单节点范围在界面注明。
|
||||
|
||||
慢请求只统计经过 LinkPort 且已收到完整响应的操作，阈值固定为 1000 ms，进程内保留最近 500 条，重启清空。仅保存时间、网关/目标/账号标识、协议、操作类别和耗时；不保存 SQL 参数、Redis 键值或消息正文，不读取上游慢日志。耗时包含网关、网络、排队及响应传输，不能当作纯数据库执行时间。
|
||||
|
||||
MySQL 按查询响应、MongoDB 按请求响应、Redis 按有序回复、SQL Server 按 SQL batch/RPC 响应计时。PostgreSQL 简单查询按 Query 周期、扩展查询按首个 Parse/Bind/Execute 到 ReadyForQuery 的周期计时（包含客户端发送 Sync 前的等待）。RabbitMQ 只计量有强制回复的通道开关及队列/交换机声明；异步发布、消费、推送、无回复操作和管理 HTTP 请求不在慢请求计时范围。订阅模式及无法可靠关联的响应停止计时，保持协议转发；每条连接待匹配队列上限 4096。
|
||||
|
||||
接口为 `POST /api/v1/monitoring/database`（targetId、accountId），返回 health 和 diagnostics；原 `GET /api/v1/monitoring` 新增按目标范围筛选的 slowRequests。新增响应字段兼容旧客户端，无数据库迁移，不修改已有目标数据。
|
||||
|
||||
本轮实际验证：Go 1.27.1 协议、model、gateway、api 测试通过，新增 MySQL 完成计时、Redis 流水线/推送/未知 RESP 保真、TDS 消息边界、RabbitMQ 通道关联和只读管理采集测试通过，API 验证指标不可用时仍保留健康结果且不泄露凭据。全量 Go 测试仅 `internal/cluster/TestMySQLReplicationAndGroupDiscovery/mysql-replication` 的既有测试桩在 go-mysql 公钥认证处空指针失败，其他包通过；`go vet ./...` 通过。前端生产构建、18 项 Node 测试通过。新构建 `bin/dev-build/monitoring/linkport.exe` 的监控、Redis/RabbitMQ 只读与管理端口专项浏览器回归通过；监控数据为模拟接口，协议测试为本地 fixture，未连接真实数据库。完整 Playwright 在既有 account-editor 测试的 `.target-card`/“配置目标”定位器处失败，设置首次失败停止，余下 175 项未运行；日志在 `bin/dev-build/monitoring`。未部署或重启当前服务。
|
||||
|
||||
TCP 端口映射为无协议账号的独立服务类型，默认监听 `0.0.0.0`；来源白名单和连接数限制仍生效。目标需同为 TCP 类型，画板通过网关“空账号”连接到目标“空账号”，不使用服务卡片建立绑定。完整操作与安全边界见 `tcp-port-mapping.md`。
|
||||
|
||||
新增后端测试使用临时 SQLite 和本地协议测试桩，前端测试使用独立数据目录
|
||||
|
||||
+40
-2
@@ -1,6 +1,12 @@
|
||||
# 可选只读模式
|
||||
|
||||
MySQL 新建及默认配置为安全读写，MongoDB 默认读写。每条代理可独立开启只读；SSH、Redis、RabbitMQ、PostgreSQL 当前没有此选项。切换前必须停止并断开现有连接，再启动使用新策略。
|
||||
MySQL 新建及默认配置为安全读写，MongoDB、Redis、RabbitMQ 默认读写。这四种协议支持账号只读模式；RabbitMQ 为严格只读，禁止消费和确认。SSH、PostgreSQL 等协议不提供只读选项。切换前必须停止并断开现有连接，再启动使用新策略。
|
||||
|
||||
账号访问模式按协议提供：MySQL 为只读、读写、全放行，MongoDB、Redis、RabbitMQ 为只读、读写；PostgreSQL、SQL Server、HTTP、HTTPS 仅提供读写。2026-10-15 先修复了无协议策略却错误显示“只读”选项的问题，随后补齐 Redis 和 RabbitMQ 的实际拦截才重新开放选项。已有草稿若含不受支持的账号模式，保存前在对应账号字段提示重新选择，不自动将只读改成读写；服务端返回同类错误时显示中文原因与操作指引。无配置或数据迁移，需重新构建并重启后生效。
|
||||
|
||||
实际验证：先用新增回归复现错误选项、漏校验和英文提示，修复后访问模式/草稿 Node 测试 17 项通过。前端 TypeScript/Vite 构建通过，恢复 `web/dist/.gitkeep`；Go 1.27.1 新嵌入程序为 `bin/dev-build/redis-access-mode/linkport.exe`。该程序使用 18901/18902 端口和临时 SQLite 的 Playwright 专项 16 项通过（46.0 秒），覆盖 Redis 界面只提供读写、真实 POST 返回 201 并回读 `readwrite`、MySQL 只读账号保存及空密码保留、React 与 UI domain 回归。未连接真实 Redis/MySQL/MongoDB 服务器或修改真实配置。
|
||||
|
||||
全量 Playwright 发现 174 项，以 `--max-failures=1 --timeout=15000` 运行后，在既有 account-editor 的 `.target-card / 配置目标` 定位器超时处停止，173 项未运行，不记为全套通过。后端源码未变，未重复本会话已完成的 Go 全量和 vet；先前全量的 cluster MySQL fixture 空指针、pgproxy 身份隔离 fixture 间歇失败仍保留记录。构建与浏览器日志位于 `bin/dev-build/redis-access-mode/`；本次未部署或重启现有实例。
|
||||
|
||||
只读是在协议网关对请求执行白名单，并不是目标数据库账号的权限。无法解析或不在白名单的请求不会转发上游；已允许的视图、服务器扩展内部仍可能有副作用。需要数据库级绝对写权限隔离时，应同时使用目标数据库的只读账号。
|
||||
|
||||
@@ -10,6 +16,12 @@ MySQL 账号可选择只读、安全读写（默认）和普通读写。安全
|
||||
|
||||
使用 AST 递归检查 SELECT、CTE、UNION、子查询、表达式和函数，允许计划中的常见 SHOW、DESCRIBE、普通 EXPLAIN SELECT 和有限会话/事务控制。包括 SHOW TABLE STATUS（可带数据库、LIKE、WHERE）、SHOW [FULL] PROCESSLIST、SHOW VARIABLES LIKE 和 information_schema.ENGINES 查询。
|
||||
|
||||
监控查询支持 `FROM_UNIXTIME`、`CHAR`（含 `USING` 字符集）和 `SUBSTRING_INDEX`，用于时间范围过滤及 `information_schema.USER_PRIVILEGES` 的当前账号匹配。普通 `EXPLAIN SELECT` 和 `EXPLAIN FORMAT=TRADITIONAL SELECT` 对内部查询执行相同递归检查；`EXPLAIN ANALYZE` 仍拒绝。允许的函数参数中包含未知函数、有副作用的函数或用户变量赋值时仍拒绝，带数据库限定名的自定义函数不因名称相同而放行。
|
||||
|
||||
此兼容调整无需数据迁移；现有运行实例需更新程序并重启后生效。
|
||||
|
||||
历史慢 SQL 监控另支持 `UNIX_TIMESTAMP`、`TIME_TO_SEC`、`SEC_TO_TIME`、`LEFT`，并检查其全部参数；允许 `CONVERT(... USING utf8mb4)` 与 `CAST(... AS CHAR)` 组合。放行函数不代表日志表具有查询索引，监控应用仍需单独检查执行计划，CSV 慢日志源仍可能无法安全读取。
|
||||
|
||||
允许最多 100 条符合上述策略的批量语句，先检查整个批次，再在同一目标会话逐条执行并返回多个结果集；批量写入不支持，读写模式仍允许单条写入。数据库运行错误发生后停止剩余语句，已执行语句不会自动回滚。
|
||||
|
||||
拒绝写入、DDL、管理操作、CALL、动态 PREPARE/EXECUTE、可执行版本注释、SELECT INTO、锁定读取、用户变量和未知函数。二进制预编译在 prepare 时审批，后续仅允许该连接中已批准的 statement ID，重置后清空。
|
||||
@@ -26,6 +38,32 @@ MySQL 账号可选择只读、安全读写（默认）和普通读写。安全
|
||||
|
||||
完整设计白名单见 [开发计划](development-plan.md#4-可选只读模式)，实际策略位于 `internal/protocol/mysqlproxy/policy.go` 和 `internal/protocol/mongoproxy/policy.go`，由测试覆盖。白名单不支持通过管理界面扩展。
|
||||
|
||||
## Redis
|
||||
|
||||
只读策略在单节点、Sentinel/复制解析后的连接和 Cluster 路由前执行同一命令白名单。允许常用字符串、哈希、列表、集合、有序集合读取、SCAN、TTL、流读取及有限元数据查询。AUTH/HELLO 继续映射网关凭据，SELECT 和有限 CLIENT 会话信息命令可用。
|
||||
|
||||
拒绝写入、发布、脚本（含 EVAL_RO）、事务、模块/未知命令、订阅推送、CLIENT TRACKING/REPLY、带写副作用的读取变体。具体白名单见 `internal/protocol/redisproxy/policy.go`；不根据命令名称前缀猜测权限。Redis 的 READONLY 命令不是权限控制，不能代替此策略。
|
||||
|
||||
单节点只读连接逐条读取完整 RESP 回复再处理下一条请求，混合流水线按原顺序返回读取结果和 NOPERM，拒绝的请求不会到达上游。单条回复沿用 Cluster 解析器的 32 MiB、65536 个数组项及 64 层嵌套限制。单节点已连接会话不能通过重新认证切换只读/读写模式，需重新连接；读写连接保持流式转发。默认模式和已有账号不自动改变，无数据迁移。Redis 自身的过期清理等内部维护仍正常运行。
|
||||
|
||||
## RabbitMQ
|
||||
|
||||
按用户选择实现 **严格只读**，不是消费者权限。AMQP 0-9-1 只允许连接/通道控制、心跳，以及带 passive 标志的已有队列/交换机检查。队列检查可以返回消息数和消费者数，但不读取消息正文。不允许空名称、主动声明、exclusive/auto-delete 等声明标志。
|
||||
|
||||
拒绝 basic.consume、basic.get、ack/reject/nack/recover、publish、队列/交换机创建、绑定、清空、删除、事务和未知方法；也拒绝客户端直接发送内容头/正文帧。拒绝先记录审计，再返回 AMQP connection.close 403 并断开整个连接，避免流水线发布的剩余内容帧穿透。客户端自动声明/绑定也会被拒绝，需使用被动检查。
|
||||
|
||||
账号默认仍为读写，无数据迁移；界面在选择 RabbitMQ 只读时显示禁止消费/确认的说明。
|
||||
|
||||
### AMQP 与管理页面双端口
|
||||
|
||||
同一个 RabbitMQ 网关可同时监听 AMQP 和 HTTP 管理端口。网关的 `listenPort` 为 AMQP；`managementListenPort` 为管理 HTTP，默认 0（关闭），可以设为 15672。两个端口使用同一个 `listenHost`、来源白名单、连接数上限、账号映射和审计。目标服务的 `targetPort` 通常为 5672；`managementPort` 默认 0 时使用 15672，`managementTLS` 可单独启用目标 HTTPS（例如 15671），共用目标 CA。管理页面地址为 `http://<网关主机>:<管理监听端口>/`，用网关账号登录；上游需要启用 management 插件并授予目标账号相应管理权限。
|
||||
|
||||
管理 HTTP 复用现有反向代理与 Basic 凭据映射：只读账号仅允许内置信息 API 的 GET/HEAD 及页面静态资源。拒绝 POST/PUT/PATCH/DELETE、消息获取接口、定义导入/导出、未知插件路径、协议升级和 CONNECT。特别拒绝 GET `aliveness-test`，因为它会创建队列并发送/消费探测消息。拒绝返回 HTTP 403，不转发目标。读写账号保留上游权限决定的管理功能。管理接口连接目标配置的主机和管理端口，不使用 AMQP 集群发现或 AMQP 持久连接；SSH/Kubernetes 出口使用管理端口独立拨号。
|
||||
|
||||
双端口全部绑定成功后才发布运行状态；管理端口占用或持久化失败会关闭本次新建监听。停止/关闭程序释放两个端口。已运行时修改管理监听端口或启用管理监听时的监听地址，必须先停网关；目标管理端口配置更新对新连接生效。诊断页面展示管理监听地址。启用管理端口时逐个创建网关，不支持批量自动分配两个端口。配置使用现有 JSON 存储，无数据库迁移；老配置不自动开放新端口。
|
||||
|
||||
## 错误与记录
|
||||
|
||||
被拒绝请求使用数据库原生错误返回，普通策略拒绝不主动断开连接。连接事件记录规则编号；命令审计加密记录 SQL/BSON 请求内容（参数和正文覆盖范围见开发计划）。直连、SOCKS5 与 SSH 转发出口执行相同规则。
|
||||
2026-10-15 Redis/RabbitMQ 实现验证：Go 协议、model、gateway 和 API 专项及全量中的这些包通过；Node 18 项、TypeScript/Vite、go vet 和新嵌入程序构建通过。浏览器 React/UI domain 及 Redis 16 项通过，修正 RabbitMQ 测试默认账号后两协议保存回归 2 项通过。全量仍有既有 cluster MySQL fixture 崩溃和 account-editor 浏览器旧定位器失败，详情见 change-rationale.md；日志及程序位于 `bin/dev-build/readonly/`。未使用真实 Redis/RabbitMQ，也未部署到现有实例。
|
||||
|
||||
被拒绝请求使用协议原生错误返回；RabbitMQ 拒绝时关闭连接，其余协议普通策略拒绝不主动断开连接。命令审计记录拒绝，SQL/BSON 请求内容覆盖范围见开发计划。直连、SOCKS5 与 SSH 转发出口执行相同规则。
|
||||
|
||||
@@ -62,7 +62,13 @@ func (s *Server) monitoring(w http.ResponseWriter, r *http.Request, path string)
|
||||
}
|
||||
out = append(out, map[string]any{"id": p.ID, "metrics": metrics, "health": latest, "accountHealth": accountHealth})
|
||||
}
|
||||
respond(w, 200, map[string]any{"gateways": out, "health": health, "intervalSeconds": interval, "auditQueue": s.store.AuditQueueSnapshot()})
|
||||
slow := []gateway.SlowRequest{}
|
||||
for _, entry := range s.manager.SlowSnapshot() {
|
||||
if scopes == nil || scopes.Matches("targets", entry.TargetID, requestScope(r)) {
|
||||
slow = append(slow, entry)
|
||||
}
|
||||
}
|
||||
respond(w, 200, map[string]any{"gateways": out, "health": health, "slowRequests": slow, "intervalSeconds": interval, "auditQueue": s.store.AuditQueueSnapshot()})
|
||||
return
|
||||
}
|
||||
if r.Method == "POST" && path == "monitoring/settings" {
|
||||
@@ -79,7 +85,7 @@ func (s *Server) monitoring(w http.ResponseWriter, r *http.Request, path string)
|
||||
respond(w, 200, map[string]int{"intervalSeconds": body.IntervalSeconds})
|
||||
return
|
||||
}
|
||||
if path != "monitoring/check" {
|
||||
if path != "monitoring/check" && path != "monitoring/database" {
|
||||
fail(w, 404, "not found")
|
||||
return
|
||||
}
|
||||
@@ -117,5 +123,13 @@ func (s *Server) monitoring(w http.ResponseWriter, r *http.Request, path string)
|
||||
fail(w, 400, err.Error())
|
||||
return
|
||||
}
|
||||
if path == "monitoring/database" {
|
||||
diagnostics, e := s.manager.DatabaseMonitoring(ctx, p)
|
||||
if e != nil {
|
||||
diagnostics.Notices = append(diagnostics.Notices, e.Error())
|
||||
}
|
||||
respond(w, 200, map[string]any{"health": result, "diagnostics": diagnostics})
|
||||
return
|
||||
}
|
||||
respond(w, 200, result)
|
||||
}
|
||||
|
||||
@@ -27,6 +27,19 @@ func TestMonitoringTargetAccountAndSettings(t *testing.T) {
|
||||
if strings.Contains(w.Body.String(), "never-expose-this") {
|
||||
t.Fatal("health response leaked credentials")
|
||||
}
|
||||
w = f.request("POST", "/api/v1/monitoring/database", map[string]any{"targetId": target.ID, "accountId": "first"}, true)
|
||||
requireStatus(t, w, 200)
|
||||
var diagnostics struct {
|
||||
Health gateway.HealthResult `json:"health"`
|
||||
Diagnostics model.DatabaseMonitoring `json:"diagnostics"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &diagnostics); err != nil || !diagnostics.Health.Healthy || len(diagnostics.Diagnostics.Notices) == 0 {
|
||||
t.Fatalf("unavailable pressure must retain health and a notice: %s", w.Body)
|
||||
}
|
||||
if strings.Contains(w.Body.String(), "never-expose-this") {
|
||||
t.Fatal("diagnostics leaked credentials")
|
||||
}
|
||||
requireStatus(t, f.request("POST", "/api/v1/monitoring/database", map[string]any{"targetId": target.ID, "accountId": "missing"}, true), 400)
|
||||
requireStatus(t, f.request("POST", "/api/v1/monitoring/check", map[string]any{"targetId": target.ID, "accountId": "missing"}, true), 400)
|
||||
requireStatus(t, f.request("POST", "/api/v1/monitoring/check", map[string]any{"targetId": target.ID, "accountId": "first", "attempts": 4}, true), 400)
|
||||
requireStatus(t, f.request("POST", "/api/v1/monitoring/settings", map[string]int{"intervalSeconds": 1}, true), 400)
|
||||
|
||||
@@ -0,0 +1,35 @@
|
||||
package api
|
||||
|
||||
import (
|
||||
"encoding/json"
|
||||
"linkport/internal/model"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRabbitManagementConfigurationRoundTrip(t *testing.T) {
|
||||
f := newFixture(t)
|
||||
f.login()
|
||||
w := f.request("POST", "/api/v1/targets", map[string]any{"name": "rabbit management", "type": "rabbitmq", "targetHost": "rabbit.internal", "targetPort": 5672, "managementPort": 15671, "managementTLS": true, "accounts": []any{map[string]any{"id": "upstream", "username": "target", "password": "secret"}}}, true)
|
||||
requireStatus(t, w, 201)
|
||||
var target struct {
|
||||
Config model.Target `json:"config"`
|
||||
}
|
||||
if err := json.Unmarshal(w.Body.Bytes(), &target); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if target.Config.ManagementPort != 15671 || !target.Config.ManagementTLS {
|
||||
t.Fatal("management target settings lost")
|
||||
}
|
||||
w = f.request("POST", "/api/v1/proxies", map[string]any{"name": "rabbit dual port", "type": "rabbitmq", "targetId": target.Config.ID, "targetAccountId": "upstream", "listenHost": "127.0.0.1", "listenPort": 5672, "managementListenPort": 15672, "gatewayAccounts": []any{map[string]any{"id": "reader", "username": "reader", "password": "fixture", "accessMode": "readonly"}}}, true)
|
||||
requireStatus(t, w, 201)
|
||||
view := readView(t, w)
|
||||
if view.Config.ManagementListenPort != 15672 || view.Config.GatewayAccounts[0].AccessMode != "readonly" {
|
||||
t.Fatal("gateway management/readonly settings lost")
|
||||
}
|
||||
saved, err := f.store.Get(view.Config.ID)
|
||||
if err != nil || saved.ManagementPort != 15671 || !saved.ManagementTLS {
|
||||
t.Fatalf("target settings not resolved: %+v %v", saved, err)
|
||||
}
|
||||
requireStatus(t, f.request("PUT", "/api/v1/proxies/"+view.Config.ID, map[string]any{"managementPort": 9999}, true), 400)
|
||||
requireStatus(t, f.request("PUT", "/api/v1/proxies/"+view.Config.ID, map[string]any{"managementListenPort": 5672}, true), 400)
|
||||
}
|
||||
@@ -384,7 +384,7 @@ func gatewayFields(patch map[string]any) error {
|
||||
if _, present := patch["connectionOptions"]; present {
|
||||
return errors.New("edit connectionOptions through target services")
|
||||
}
|
||||
for _, k := range []string{"cluster", "group", "targetHost", "targetPort", "targetUser", "targetPassword", "targetTLS", "targetCA", "targetCAAssetId", "targetCompression", "authSource", "outbound", "outboundId"} {
|
||||
for _, k := range []string{"managementPort", "managementTLS", "cluster", "group", "targetHost", "targetPort", "targetUser", "targetPassword", "targetTLS", "targetCA", "targetCAAssetId", "targetCompression", "authSource", "outbound", "outboundId"} {
|
||||
if _, ok := patch[k]; ok {
|
||||
return errors.New("edit upstream connection settings through target services")
|
||||
}
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"context"
|
||||
"errors"
|
||||
"linkport/internal/cluster"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/outbound"
|
||||
"math"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (m *Manager) DatabaseMonitoring(ctx context.Context, p model.Proxy) (model.DatabaseMonitoring, error) {
|
||||
h, ok := m.handlers[p.Type].(model.DatabaseMonitor)
|
||||
if !ok {
|
||||
return model.DatabaseMonitoring{}, errors.New("database monitoring is unsupported for this protocol")
|
||||
}
|
||||
ctx, cancel := context.WithTimeout(ctx, 10*time.Second)
|
||||
defer cancel()
|
||||
p.PreparedTarget, p.TargetDialer = nil, nil
|
||||
dial := outbound.Dialer(p)
|
||||
if p.Type == "rabbitmq" {
|
||||
p.TargetDialer = outbound.Dialer
|
||||
}
|
||||
if p.Type != "rabbitmq" {
|
||||
var err error
|
||||
p, _, err = cluster.Resolve(ctx, p, dial)
|
||||
if err != nil {
|
||||
return model.DatabaseMonitoring{}, errors.New("database monitoring connection failed")
|
||||
}
|
||||
}
|
||||
out, err := h.Monitor(ctx, p, dial)
|
||||
if err != nil {
|
||||
return out, errors.New("database monitoring unavailable; check connectivity and target permissions")
|
||||
}
|
||||
metrics := out.Metrics[:0]
|
||||
for _, metric := range out.Metrics {
|
||||
if !math.IsNaN(metric.Value) && !math.IsInf(metric.Value, 0) {
|
||||
metrics = append(metrics, metric)
|
||||
}
|
||||
}
|
||||
out.Metrics = metrics
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,21 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"linkport/internal/model"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestSlowRequestsThresholdAndBoundedHistory(t *testing.T) {
|
||||
m, p, _ := testManager(t)
|
||||
m.recordExecution(p, model.Execution{DurationMS: 999})
|
||||
if len(m.SlowSnapshot()) != 0 {
|
||||
t.Fatal("fast execution recorded")
|
||||
}
|
||||
for i := 0; i < 510; i++ {
|
||||
m.recordExecution(p, model.Execution{Protocol: "mysql", Operation: "query", DurationMS: 1000 + float64(i)})
|
||||
}
|
||||
got := m.SlowSnapshot()
|
||||
if len(got) != 500 || got[0].DurationMS != 1010 || got[0].TargetID != p.TargetID {
|
||||
t.Fatal("slow history bounds or identity lost")
|
||||
}
|
||||
}
|
||||
@@ -6,6 +6,9 @@ func (i *instance) stopLocked() {
|
||||
i.stopping = true
|
||||
i.cancel()
|
||||
i.listener.Close()
|
||||
if i.managementListener != nil {
|
||||
i.managementListener.Close()
|
||||
}
|
||||
for connection := range i.connections {
|
||||
connection.Close()
|
||||
}
|
||||
|
||||
@@ -43,6 +43,9 @@ func (m *Manager) StartFrom(load func() (model.Proxy, error), persist func(model
|
||||
}
|
||||
if err = persist(p); err != nil {
|
||||
prepared.listener.Close()
|
||||
if prepared.managementListener != nil {
|
||||
prepared.managementListener.Close()
|
||||
}
|
||||
return p, err
|
||||
}
|
||||
m.publishStartLocked(prepared)
|
||||
@@ -221,6 +224,14 @@ func (m *Manager) updateConfigs(changes []ConfigUpdate, compareCurrent bool, per
|
||||
closeReplacements()
|
||||
return errors.New("gateway protocol cannot change while running")
|
||||
}
|
||||
if config.ManagementListenPort != inst.config.ManagementListenPort || (inst.managementListener != nil && config.ListenHost != inst.config.ListenHost) {
|
||||
closeReplacements()
|
||||
return errors.New("stop the gateway before changing RabbitMQ management listen settings")
|
||||
}
|
||||
if config.ManagementListenPort != 0 && config.ManagementListenPort == config.ListenPort {
|
||||
closeReplacements()
|
||||
return errors.New("RabbitMQ management and AMQP listen ports must differ")
|
||||
}
|
||||
handler := m.handlers[config.Type]
|
||||
if handler == nil {
|
||||
closeReplacements()
|
||||
|
||||
@@ -16,6 +16,7 @@ import (
|
||||
)
|
||||
|
||||
type Status struct {
|
||||
ManagementBoundAddress string `json:"managementBoundAddress,omitempty"`
|
||||
PersistentTargets []PersistentTargetStatus `json:"persistentTargets"`
|
||||
BoundAddress string `json:"boundAddress,omitempty"`
|
||||
ClientDestinations []string `json:"clientDestinations"`
|
||||
@@ -26,6 +27,7 @@ type Status struct {
|
||||
Error string `json:"error,omitempty"`
|
||||
}
|
||||
type instance struct {
|
||||
managementListener net.Listener
|
||||
persistent *persistentTargets
|
||||
config model.Proxy // Immutable snapshot replaced under Manager.mu.
|
||||
ctx context.Context
|
||||
@@ -38,12 +40,14 @@ type instance struct {
|
||||
}
|
||||
|
||||
type preparedStart struct {
|
||||
managementListener net.Listener
|
||||
config model.Proxy
|
||||
listener net.Listener
|
||||
handler model.Handler
|
||||
}
|
||||
|
||||
type Manager struct {
|
||||
slowRequests []SlowRequest
|
||||
mu sync.Mutex
|
||||
instances map[string]*instance
|
||||
snapshots map[string]model.Proxy
|
||||
@@ -106,13 +110,30 @@ func (m *Manager) prepareStartLocked(p model.Proxy) (preparedStart, error) {
|
||||
metric.RecentErrorAt = time.Now().UTC()
|
||||
return preparedStart{}, errors.New(m.failures[p.ID])
|
||||
}
|
||||
return preparedStart{config: p, listener: l, handler: h}, nil
|
||||
var management net.Listener
|
||||
if p.Type == "rabbitmq" && p.ManagementListenPort != 0 {
|
||||
if m.handlers["http"] == nil {
|
||||
l.Close()
|
||||
return preparedStart{}, errors.New("HTTP management handler unavailable")
|
||||
}
|
||||
managementAddress, e := listenAddress(p.ListenHost, p.ManagementListenPort)
|
||||
if e == nil {
|
||||
management, e = net.Listen("tcp", managementAddress)
|
||||
}
|
||||
if e != nil {
|
||||
l.Close()
|
||||
m.failures[p.ID] = listenStartError(managementAddress, e).Error()
|
||||
return preparedStart{}, errors.New(m.failures[p.ID])
|
||||
}
|
||||
}
|
||||
return preparedStart{config: p, listener: l, managementListener: management, handler: h}, nil
|
||||
}
|
||||
|
||||
func (m *Manager) publishStartLocked(prepared preparedStart) {
|
||||
p, l, h := prepared.config, prepared.listener, prepared.handler
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
inst := &instance{config: p, ctx: ctx, listener: l, cancel: cancel, connections: map[net.Conn]struct{}{}}
|
||||
inst.managementListener = prepared.managementListener
|
||||
inst.persistent = newPersistentTargets(ctx, m.handlers)
|
||||
inst.persistent.Reconcile(p)
|
||||
m.instances[p.ID] = inst
|
||||
@@ -120,6 +141,10 @@ func (m *Manager) publishStartLocked(prepared preparedStart) {
|
||||
delete(m.failures, p.ID)
|
||||
inst.wg.Add(1)
|
||||
go m.accept(ctx, inst, l, h)
|
||||
if inst.managementListener != nil {
|
||||
inst.wg.Add(1)
|
||||
go m.accept(ctx, inst, inst.managementListener, m.handlers["http"])
|
||||
}
|
||||
}
|
||||
func (m *Manager) accept(ctx context.Context, i *instance, listener net.Listener, h model.Handler) {
|
||||
defer i.wg.Done()
|
||||
@@ -135,6 +160,11 @@ func (m *Manager) accept(ctx context.Context, i *instance, listener net.Listener
|
||||
p := i.config
|
||||
p.TargetDialer = i.persistent.Dialer
|
||||
p.PreparedTarget = i.persistent.Borrow
|
||||
if listener == i.managementListener {
|
||||
// Management HTTP needs its own port, including for SSH/Kubernetes tunnels.
|
||||
p.TargetDialer = outbound.Dialer
|
||||
p.PreparedTarget = nil
|
||||
}
|
||||
metric := m.metric(p.ID)
|
||||
metric.ConnectionAttempts++
|
||||
allowed := !i.stopping && Allowed(p.AllowCIDRs, source) && len(i.connections) < p.MaxConnections
|
||||
@@ -201,6 +231,9 @@ func (m *Manager) shutdown(grace time.Duration) {
|
||||
instances = append(instances, inst)
|
||||
inst.stopping = true
|
||||
inst.listener.Close()
|
||||
if inst.managementListener != nil {
|
||||
inst.managementListener.Close()
|
||||
}
|
||||
}
|
||||
m.mu.Unlock()
|
||||
var wg sync.WaitGroup
|
||||
@@ -223,7 +256,11 @@ func (m *Manager) Status(id string) Status {
|
||||
defer m.mu.Unlock()
|
||||
if i := m.instances[id]; i != nil {
|
||||
bound := i.listener.Addr().String()
|
||||
return Status{Running: true, Connections: len(i.connections), Rejected: i.rejected, BoundAddress: bound, ClientDestinations: clientDestinations(bound), ClientDestinationScope: clientDestinationScope(bound), PersistentTargets: i.persistent.Status()}
|
||||
management := ""
|
||||
if i.managementListener != nil {
|
||||
management = i.managementListener.Addr().String()
|
||||
}
|
||||
return Status{ManagementBoundAddress: management, Running: true, Connections: len(i.connections), Rejected: i.rejected, BoundAddress: bound, ClientDestinations: clientDestinations(bound), ClientDestinationScope: clientDestinationScope(bound), PersistentTargets: i.persistent.Status()}
|
||||
}
|
||||
return Status{Error: m.failures[id], ClientDestinations: []string{}}
|
||||
}
|
||||
|
||||
@@ -42,6 +42,32 @@ type HealthResult struct {
|
||||
Result model.TestResult `json:"result"`
|
||||
}
|
||||
|
||||
type SlowRequest struct {
|
||||
GatewayID string `json:"gatewayId"`
|
||||
TargetID string `json:"targetId"`
|
||||
AccountID string `json:"accountId"`
|
||||
model.Execution
|
||||
}
|
||||
|
||||
func (m *Manager) SlowSnapshot() []SlowRequest {
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
return append([]SlowRequest{}, m.slowRequests...)
|
||||
}
|
||||
|
||||
func (m *Manager) recordExecution(p model.Proxy, e model.Execution) {
|
||||
if e.DurationMS < 1000 {
|
||||
return
|
||||
}
|
||||
m.mu.Lock()
|
||||
defer m.mu.Unlock()
|
||||
m.slowRequests = append(m.slowRequests, SlowRequest{GatewayID: p.ID, TargetID: p.TargetID, AccountID: p.TargetAccountID, Execution: e})
|
||||
if len(m.slowRequests) > 500 {
|
||||
copy(m.slowRequests, m.slowRequests[len(m.slowRequests)-500:])
|
||||
m.slowRequests = m.slowRequests[:500]
|
||||
}
|
||||
}
|
||||
|
||||
func (m *Manager) metric(id string) *Metrics {
|
||||
v := m.metrics[id]
|
||||
if v == nil {
|
||||
|
||||
@@ -0,0 +1,60 @@
|
||||
package gateway
|
||||
|
||||
import (
|
||||
"errors"
|
||||
"linkport/internal/model"
|
||||
"net"
|
||||
"strconv"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestRabbitManagementListenerLifecycle(t *testing.T) {
|
||||
m, p, entered := testManager(t)
|
||||
m.handlers["rabbitmq"], m.handlers["http"] = heldHandler{entered}, heldHandler{entered}
|
||||
p.Type = "rabbitmq"
|
||||
p.MaxConnections = 2
|
||||
reserved, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
_, port, _ := net.SplitHostPort(reserved.Addr().String())
|
||||
p.ManagementListenPort, _ = strconv.Atoi(port)
|
||||
if err = m.Start(p); err == nil {
|
||||
t.Fatal("started with occupied management port")
|
||||
}
|
||||
reserved.Close()
|
||||
if _, err = m.StartFrom(func() (model.Proxy, error) { return p, nil }, func(model.Proxy) error { return errors.New("persist failed") }); err == nil {
|
||||
t.Fatal("expected rollback")
|
||||
}
|
||||
if err = m.Start(p); err != nil {
|
||||
t.Fatal("listener leaked after rollback", err)
|
||||
}
|
||||
status := m.Status(p.ID)
|
||||
next := p
|
||||
next.ManagementListenPort = 0
|
||||
persisted := false
|
||||
if err = m.UpdateConfig(next, func() error { persisted = true; return nil }); err == nil || persisted {
|
||||
t.Fatal("running management listener changed without stopping")
|
||||
}
|
||||
for _, address := range []string{status.BoundAddress, status.ManagementBoundAddress} {
|
||||
c, e := net.DialTimeout("tcp", address, time.Second)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
defer c.Close()
|
||||
select {
|
||||
case <-entered:
|
||||
case <-time.After(time.Second):
|
||||
t.Fatal("listener did not admit connection")
|
||||
}
|
||||
}
|
||||
m.Stop(p.ID)
|
||||
for _, address := range []string{status.BoundAddress, status.ManagementBoundAddress} {
|
||||
l, e := net.Listen("tcp", address)
|
||||
if e != nil {
|
||||
t.Fatal("listener leaked on stop", e)
|
||||
}
|
||||
l.Close()
|
||||
}
|
||||
}
|
||||
@@ -58,7 +58,8 @@ func (s *connectionSession) serve(handler model.Handler) {
|
||||
}
|
||||
|
||||
func (s *connectionSession) commandContext() context.Context {
|
||||
ctx := model.WithCommandAudit(s.ctx, func(command model.Command) error {
|
||||
ctx := model.WithExecutionObserver(s.ctx, func(e model.Execution) { s.manager.recordExecution(s.auditSnapshot(), e) })
|
||||
ctx = model.WithCommandAudit(ctx, func(command model.Command) error {
|
||||
return s.manager.store.EnqueueCommand(s.auditSnapshot(), s.connection.Source, command)
|
||||
})
|
||||
return model.WithCommandOutputAudit(ctx, func(output model.CommandOutput) error {
|
||||
|
||||
@@ -2,6 +2,51 @@ package model
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestRedisRabbitReadonlyModes(t *testing.T) {
|
||||
for _, protocol := range []string{"redis", "rabbitmq"} {
|
||||
p := Defaults()
|
||||
p.Type, p.Name, p.ListenPort, p.AccessMode = protocol, "readonly", 12345, "readonly"
|
||||
p.GatewayAccounts = []GatewayAccount{{ID: "client", Username: "client", Password: "fixture", AccessMode: "readonly"}}
|
||||
if err := p.ValidateGateway(); err != nil {
|
||||
t.Fatalf("%s: %v", protocol, err)
|
||||
}
|
||||
if p.GatewayAccounts[0].Apply(p).AccessMode != "readonly" {
|
||||
t.Fatal("account policy lost")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRabbitManagementConfiguration(t *testing.T) {
|
||||
p := Defaults()
|
||||
p.Type, p.Name, p.ListenPort = "rabbitmq", "management", 5672
|
||||
p.ManagementListenPort = 15672
|
||||
p.GatewayAccounts = []GatewayAccount{{ID: "client", Username: "client", Password: "fixture"}}
|
||||
if err := p.ValidateGateway(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
for _, port := range []int{-1, 65536, 5672} {
|
||||
q := p
|
||||
q.ManagementListenPort = port
|
||||
if q.ValidateGateway() == nil {
|
||||
t.Fatalf("invalid management listen port accepted: %d", port)
|
||||
}
|
||||
}
|
||||
target := TargetFromProxy(p)
|
||||
target.TargetHost, target.TargetPort, target.ManagementPort, target.ManagementTLS = "rabbit.internal", 5672, 15671, true
|
||||
target.Accounts = []TargetAccount{{ID: "upstream", Username: "target", Password: "fixture"}}
|
||||
if err := target.Validate(); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
resolved := target.Apply(p)
|
||||
if resolved.ManagementPort != 15671 || !resolved.ManagementTLS || resolved.ManagementListenPort != 15672 {
|
||||
t.Fatal("management configuration lost")
|
||||
}
|
||||
stored := resolved.GatewayOnly()
|
||||
if stored.ManagementPort != 0 || stored.ManagementTLS || stored.ManagementListenPort != 15672 {
|
||||
t.Fatal("target settings leaked into gateway storage")
|
||||
}
|
||||
}
|
||||
|
||||
func TestMySQLAccountAccessModes(t *testing.T) {
|
||||
for _, gateway := range []string{"", "readwrite", "readonly", "unrestricted"} {
|
||||
for _, account := range []string{"", "readwrite", "readonly", "unrestricted"} {
|
||||
|
||||
@@ -0,0 +1,69 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"context"
|
||||
"sync"
|
||||
"time"
|
||||
)
|
||||
|
||||
type Execution struct {
|
||||
Protocol string `json:"protocol"`
|
||||
Operation string `json:"operation"`
|
||||
DurationMS float64 `json:"durationMs"`
|
||||
At time.Time `json:"at"`
|
||||
}
|
||||
type executionKey struct{}
|
||||
|
||||
func WithExecutionObserver(ctx context.Context, observe func(Execution)) context.Context {
|
||||
return context.WithValue(ctx, executionKey{}, observe)
|
||||
}
|
||||
func ObserveExecution(ctx context.Context, protocol, operation string, started time.Time) {
|
||||
if observe, ok := ctx.Value(executionKey{}).(func(Execution)); ok {
|
||||
observe(Execution{Protocol: protocol, Operation: operation, DurationMS: float64(time.Since(started)) / float64(time.Millisecond), At: time.Now().UTC()})
|
||||
}
|
||||
}
|
||||
|
||||
// ExecutionQueue correlates ordered replies without retaining SQL, keys or payloads.
|
||||
// shortcut: stop measuring after 4096 outstanding requests; forwarding remains unaffected.
|
||||
type ExecutionQueue struct {
|
||||
mu sync.Mutex
|
||||
pending []executionStart
|
||||
disabled bool
|
||||
}
|
||||
type executionStart struct {
|
||||
operation string
|
||||
at time.Time
|
||||
}
|
||||
|
||||
func (q *ExecutionQueue) Start(operation string) { q.StartAt(operation, time.Now()) }
|
||||
func (q *ExecutionQueue) StartAt(operation string, at time.Time) {
|
||||
q.mu.Lock()
|
||||
defer q.mu.Unlock()
|
||||
if q.disabled {
|
||||
return
|
||||
}
|
||||
if len(q.pending) >= 4096 {
|
||||
q.disabled = true
|
||||
q.pending = nil
|
||||
return
|
||||
}
|
||||
q.pending = append(q.pending, executionStart{operation, at})
|
||||
}
|
||||
func (q *ExecutionQueue) Disable() {
|
||||
q.mu.Lock()
|
||||
defer q.mu.Unlock()
|
||||
q.disabled = true
|
||||
q.pending = nil
|
||||
}
|
||||
func (q *ExecutionQueue) Finish(ctx context.Context, protocol string) {
|
||||
q.mu.Lock()
|
||||
if len(q.pending) == 0 {
|
||||
q.mu.Unlock()
|
||||
return
|
||||
}
|
||||
v := q.pending[0]
|
||||
q.pending[0] = executionStart{}
|
||||
q.pending = q.pending[1:]
|
||||
q.mu.Unlock()
|
||||
ObserveExecution(ctx, protocol, v.operation, v.at)
|
||||
}
|
||||
@@ -0,0 +1,35 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"context"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestExecutionQueueOrderedBoundedAndDisabled(t *testing.T) {
|
||||
var got []Execution
|
||||
ctx := WithExecutionObserver(context.Background(), func(e Execution) { got = append(got, e) })
|
||||
var q ExecutionQueue
|
||||
q.StartAt("first", time.Now().Add(-2*time.Second))
|
||||
q.Start("second")
|
||||
q.Finish(ctx, "redis")
|
||||
q.Finish(ctx, "redis")
|
||||
q.Finish(ctx, "redis")
|
||||
if len(got) != 2 || got[0].Operation != "first" || got[0].DurationMS < 2000 || got[1].Operation != "second" {
|
||||
t.Fatalf("wrong order: %+v", got)
|
||||
}
|
||||
q.Start("discard")
|
||||
q.Disable()
|
||||
q.Finish(ctx, "redis")
|
||||
if len(got) != 2 {
|
||||
t.Fatal("disabled queue produced result")
|
||||
}
|
||||
var over ExecutionQueue
|
||||
for i := 0; i < 4097; i++ {
|
||||
over.Start("pending")
|
||||
}
|
||||
over.Finish(ctx, "redis")
|
||||
if len(got) != 2 {
|
||||
t.Fatal("overflow miscorrelated request")
|
||||
}
|
||||
}
|
||||
@@ -105,8 +105,8 @@ func (p Proxy) validateGatewayAccounts() error {
|
||||
if a.AccessMode == "unrestricted" && p.Type != "mysql" {
|
||||
return errors.New("unrestricted accessMode is supported only for MySQL")
|
||||
}
|
||||
if a.AccessMode == "readonly" && p.Type != "mysql" && p.Type != "mongodb" {
|
||||
return errors.New("readonly is currently supported only for MySQL and MongoDB")
|
||||
if a.AccessMode == "readonly" && p.Type != "mysql" && p.Type != "mongodb" && p.Type != "redis" && p.Type != "rabbitmq" {
|
||||
return errors.New("readonly is supported only for MySQL, MongoDB, Redis and RabbitMQ")
|
||||
}
|
||||
if IsSSH(p.Type) && a.AccessMode != "" {
|
||||
return errors.New("SSH has no accessMode")
|
||||
|
||||
@@ -0,0 +1,23 @@
|
||||
package model
|
||||
|
||||
import (
|
||||
"context"
|
||||
"time"
|
||||
)
|
||||
|
||||
type ServerMetric struct {
|
||||
Name string `json:"name"`
|
||||
Value float64 `json:"value"`
|
||||
Unit string `json:"unit,omitempty"`
|
||||
}
|
||||
|
||||
type DatabaseMonitoring struct {
|
||||
CheckedAt time.Time `json:"checkedAt"`
|
||||
Metrics []ServerMetric `json:"metrics"`
|
||||
Notices []string `json:"notices"`
|
||||
}
|
||||
|
||||
// DatabaseMonitor collects fixed read-only diagnostics without changing server settings.
|
||||
type DatabaseMonitor interface {
|
||||
Monitor(context.Context, Proxy, DialFunc) (DatabaseMonitoring, error)
|
||||
}
|
||||
@@ -4,6 +4,9 @@ package model
|
||||
// Only GatewayOnly is persisted in the gateway table. HTTP responses must use
|
||||
// Redacted rather than marshal a snapshot containing decrypted credentials.
|
||||
type Proxy struct {
|
||||
ManagementListenPort int `json:"managementListenPort,omitempty"`
|
||||
ManagementPort int `json:"managementPort,omitempty"`
|
||||
ManagementTLS bool `json:"managementTLS,omitempty"`
|
||||
ConnectionOptions ConnectionOptions `json:"connectionOptions,omitempty"`
|
||||
Color string `json:"color,omitempty"`
|
||||
Remark string `json:"remark,omitempty"`
|
||||
|
||||
@@ -9,6 +9,8 @@ import (
|
||||
|
||||
// Target owns upstream identity and routing. It can be shared by many gateways.
|
||||
type Target struct {
|
||||
ManagementPort int `json:"managementPort,omitempty"`
|
||||
ManagementTLS bool `json:"managementTLS,omitempty"`
|
||||
ConnectionOptions ConnectionOptions `json:"connectionOptions,omitempty"`
|
||||
Color string `json:"color,omitempty"`
|
||||
Remark string `json:"remark,omitempty"`
|
||||
@@ -39,9 +41,10 @@ type TargetSSH struct {
|
||||
}
|
||||
|
||||
func TargetFromProxy(p Proxy) Target {
|
||||
return Target{ConnectionOptions: p.ConnectionOptions, Color: p.Color, Remark: p.Remark, ShowRemark: p.ShowRemark, Database: p.Database, Project: p.Project, Environment: p.Environment, Cluster: p.Cluster, OutboundID: p.OutboundID, TargetCompression: p.TargetCompression, ID: p.TargetID, Name: p.Name, Group: p.Group, Type: p.Type, TargetHost: p.TargetHost, TargetPort: p.TargetPort, TargetTLS: p.TargetTLS, TargetCA: p.TargetCA, TargetCAAssetID: p.TargetCAAssetID, AuthSource: p.AuthSource, Outbound: p.Outbound, SSH: TargetSSH{TargetHostKey: p.SSH.TargetHostKey, IgnoreHostKey: p.SSH.IgnoreHostKey}}
|
||||
return Target{ManagementPort: p.ManagementPort, ManagementTLS: p.ManagementTLS, ConnectionOptions: p.ConnectionOptions, Color: p.Color, Remark: p.Remark, ShowRemark: p.ShowRemark, Database: p.Database, Project: p.Project, Environment: p.Environment, Cluster: p.Cluster, OutboundID: p.OutboundID, TargetCompression: p.TargetCompression, ID: p.TargetID, Name: p.Name, Group: p.Group, Type: p.Type, TargetHost: p.TargetHost, TargetPort: p.TargetPort, TargetTLS: p.TargetTLS, TargetCA: p.TargetCA, TargetCAAssetID: p.TargetCAAssetID, AuthSource: p.AuthSource, Outbound: p.Outbound, SSH: TargetSSH{TargetHostKey: p.SSH.TargetHostKey, IgnoreHostKey: p.SSH.IgnoreHostKey}}
|
||||
}
|
||||
func (t Target) Apply(p Proxy) Proxy {
|
||||
p.ManagementPort, p.ManagementTLS = t.ManagementPort, t.ManagementTLS
|
||||
p.ConnectionOptions = t.ConnectionOptions
|
||||
p.Database = t.Database
|
||||
p.Cluster = t.Cluster
|
||||
|
||||
@@ -15,6 +15,15 @@ func Defaults() Proxy {
|
||||
return Proxy{Project: DefaultProject, Environment: DefaultEnvironment, ListenHost: "0.0.0.0", MaxConnections: 100, TargetTLS: true, AccessMode: "readwrite", AuthSource: "admin", Outbound: Outbound{Mode: "direct", Auth: "none"}, SSH: SSHConfig{AuthMethod: "password"}}
|
||||
}
|
||||
func (p *Proxy) Validate() error {
|
||||
if p.ManagementListenPort < 0 || p.ManagementListenPort > 65535 || p.ManagementPort < 0 || p.ManagementPort > 65535 {
|
||||
return errors.New("RabbitMQ management ports must be between 0 and 65535")
|
||||
}
|
||||
if p.Type != "rabbitmq" && (p.ManagementListenPort != 0 || p.ManagementPort != 0 || p.ManagementTLS) {
|
||||
return errors.New("management ports are supported only for RabbitMQ")
|
||||
}
|
||||
if p.ManagementListenPort != 0 && p.ManagementListenPort == p.ListenPort {
|
||||
return errors.New("RabbitMQ management and AMQP listen ports must differ")
|
||||
}
|
||||
if err := p.ValidateTCPForward(); err != nil {
|
||||
return err
|
||||
}
|
||||
@@ -140,8 +149,8 @@ func (p *Proxy) Validate() error {
|
||||
if p.AccessMode == "unrestricted" && p.Type != "mysql" {
|
||||
return errors.New("unrestricted accessMode is supported only for MySQL")
|
||||
}
|
||||
if p.Type != "mysql" && p.Type != "mongodb" && p.AccessMode == "readonly" {
|
||||
return errors.New("readonly is currently supported only for MySQL and MongoDB")
|
||||
if p.Type != "mysql" && p.Type != "mongodb" && p.Type != "redis" && p.Type != "rabbitmq" && p.AccessMode == "readonly" {
|
||||
return errors.New("readonly is supported only for MySQL, MongoDB, Redis and RabbitMQ")
|
||||
}
|
||||
if p.GatewayPassword == "" && !unauthenticatedGateway {
|
||||
return errors.New("gateway password is required")
|
||||
|
||||
@@ -159,6 +159,28 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
return
|
||||
}
|
||||
selected := account.Apply(p)
|
||||
if p.Type == "rabbitmq" {
|
||||
selected.TargetPort = selected.ManagementPort
|
||||
if selected.TargetPort == 0 {
|
||||
selected.TargetPort = 15672
|
||||
}
|
||||
selected.TargetTLS = selected.ManagementTLS
|
||||
if selected.AccessMode == "readonly" && !rabbitReadonlyAllowed(r) {
|
||||
if event != nil {
|
||||
event(model.EventReadOnlyRejected, "rabbitmq_management_readonly")
|
||||
}
|
||||
if err := model.AuditCommand(r.Context(), r.Method, r.URL.EscapedPath(), "denied"); err != nil {
|
||||
http.Error(w, "audit unavailable", http.StatusServiceUnavailable)
|
||||
return
|
||||
}
|
||||
http.Error(w, "RabbitMQ gateway readonly policy: only information requests are allowed", http.StatusForbidden)
|
||||
return
|
||||
}
|
||||
}
|
||||
if err := selected.CheckTargetCredential(); err != nil {
|
||||
http.Error(w, "upstream credentials unavailable", http.StatusBadGateway)
|
||||
return
|
||||
}
|
||||
if identity == "" {
|
||||
var err error
|
||||
tr, err = clientTransport(selected, selected.DialForTarget(dial))
|
||||
@@ -183,6 +205,10 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
pr.Out.Header.Del("Proxy-Authorization")
|
||||
pr.Out.Header.Del("Forwarded")
|
||||
pr.Out.Header.Del("Authorization")
|
||||
if p.Type == "rabbitmq" {
|
||||
pr.Out.Header.Del("X-HTTP-Method-Override")
|
||||
pr.Out.Header.Del("X-Method-Override")
|
||||
}
|
||||
if selected.TargetUser != "" {
|
||||
pr.Out.SetBasicAuth(selected.TargetUser, selected.TargetPassword)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,44 @@
|
||||
package httpproxy
|
||||
|
||||
import (
|
||||
"net/http"
|
||||
"net/url"
|
||||
"strings"
|
||||
)
|
||||
|
||||
func rabbitReadonlyAllowed(r *http.Request) bool {
|
||||
if r.Method != http.MethodGet && r.Method != http.MethodHead {
|
||||
return false
|
||||
}
|
||||
parts := strings.Split(strings.TrimPrefix(r.URL.EscapedPath(), "/"), "/")
|
||||
for i, part := range parts {
|
||||
decoded, err := url.PathUnescape(part)
|
||||
if err != nil || decoded == "." || decoded == ".." || strings.ContainsAny(decoded, "\\\x00%") || (strings.Contains(decoded, "/") && !(i == 2 && decoded == "/")) {
|
||||
return false
|
||||
}
|
||||
}
|
||||
if parts[0] != "api" {
|
||||
switch parts[0] {
|
||||
case "", "index.html", "favicon.ico":
|
||||
return len(parts) == 1
|
||||
case "js", "css", "img", "fonts":
|
||||
return true
|
||||
}
|
||||
return false
|
||||
}
|
||||
if len(parts) < 2 {
|
||||
return false
|
||||
}
|
||||
// Only built-in information endpoints; plugin routes and message retrieval fail closed.
|
||||
switch parts[1] {
|
||||
case "overview", "whoami", "extensions", "auth":
|
||||
return len(parts) == 2
|
||||
case "nodes", "connections", "channels", "consumers", "vhosts", "users", "permissions", "topic-permissions", "policies", "operator-policies", "parameters", "global-parameters", "bindings":
|
||||
return true
|
||||
case "queues", "exchanges":
|
||||
return len(parts) <= 4 || (len(parts) >= 5 && parts[4] == "bindings")
|
||||
case "health":
|
||||
return len(parts) >= 3 && parts[2] == "checks"
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,87 @@
|
||||
package httpproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/json"
|
||||
"errors"
|
||||
"io"
|
||||
"linkport/internal/model"
|
||||
"net/http"
|
||||
"time"
|
||||
)
|
||||
|
||||
func MonitorRabbit(ctx context.Context, p model.Proxy, dial model.DialFunc) (model.DatabaseMonitoring, error) {
|
||||
out := model.DatabaseMonitoring{Metrics: []model.ServerMetric{}, Notices: []string{}}
|
||||
p.TargetPort = p.ManagementPort
|
||||
if p.TargetPort == 0 {
|
||||
p.TargetPort = 15672
|
||||
}
|
||||
p.TargetTLS = p.ManagementTLS
|
||||
if err := p.CheckTargetCredential(); err != nil {
|
||||
return out, err
|
||||
}
|
||||
tr, err := clientTransport(p, p.DialForTarget(dial))
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
defer tr.CloseIdleConnections()
|
||||
get := func(path string, dst any) error {
|
||||
u := targetURL(p)
|
||||
u.Path = path
|
||||
req, _ := http.NewRequestWithContext(ctx, "GET", u.String(), nil)
|
||||
req.SetBasicAuth(p.TargetUser, p.TargetPassword)
|
||||
resp, err := tr.RoundTrip(req)
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
defer resp.Body.Close()
|
||||
if resp.StatusCode != 200 {
|
||||
return errors.New("management API unavailable")
|
||||
}
|
||||
return json.NewDecoder(io.LimitReader(resp.Body, 4<<20)).Decode(dst)
|
||||
}
|
||||
var overview struct {
|
||||
QueueTotals map[string]float64 `json:"queue_totals"`
|
||||
ObjectTotals map[string]float64 `json:"object_totals"`
|
||||
}
|
||||
if err = get("/api/overview", &overview); err != nil {
|
||||
out.Notices = append(out.Notices, "RabbitMQ 管理概览不可用，请检查管理端口、management 插件及账号权限")
|
||||
} else {
|
||||
for _, key := range []string{"messages", "messages_ready", "messages_unacknowledged"} {
|
||||
if v, ok := overview.QueueTotals[key]; ok {
|
||||
out.Metrics = append(out.Metrics, model.ServerMetric{Name: key, Value: v, Unit: "messages"})
|
||||
}
|
||||
}
|
||||
for _, key := range []string{"connections", "channels", "consumers", "queues"} {
|
||||
if v, ok := overview.ObjectTotals[key]; ok {
|
||||
out.Metrics = append(out.Metrics, model.ServerMetric{Name: key, Value: v})
|
||||
}
|
||||
}
|
||||
}
|
||||
var nodes []map[string]any
|
||||
if err = get("/api/nodes", &nodes); err != nil {
|
||||
out.Notices = append(out.Notices, "RabbitMQ 节点压力不可用，需要 monitoring 或 administrator 管理权限")
|
||||
} else {
|
||||
for i, node := range nodes {
|
||||
if i >= 128 {
|
||||
break
|
||||
}
|
||||
name, _ := node["name"].(string)
|
||||
for _, key := range []string{"mem_used", "mem_limit", "disk_free", "disk_free_limit", "fd_used", "fd_total", "proc_used", "proc_total", "run_queue", "mem_alarm", "disk_free_alarm"} {
|
||||
v, ok := node[key].(float64)
|
||||
if b, yes := node[key].(bool); yes {
|
||||
ok = true
|
||||
if b {
|
||||
v = 1
|
||||
}
|
||||
}
|
||||
if ok {
|
||||
out.Metrics = append(out.Metrics, model.ServerMetric{Name: name + "." + key, Value: v})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out.Notices = append(out.Notices, "队列积压、节点内存/磁盘及告警为管理 API 当前可见值，不读取消息正文。")
|
||||
out.CheckedAt = time.Now().UTC()
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,41 @@
|
||||
package httpproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"linkport/internal/model"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRabbitMonitoringReadsOnlyInformation(t *testing.T) {
|
||||
seen := map[string]bool{}
|
||||
server := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
user, password, ok := r.BasicAuth()
|
||||
if !ok || user != "monitor" || password != "secret" || r.Method != "GET" {
|
||||
t.Error("invalid monitoring request")
|
||||
}
|
||||
seen[r.URL.Path] = true
|
||||
switch r.URL.Path {
|
||||
case "/api/overview":
|
||||
w.Write([]byte(`{"queue_totals":{"messages_ready":12}}`))
|
||||
case "/api/nodes":
|
||||
w.Write([]byte(`[{"name":"node","mem_used":128,"mem_alarm":true}]`))
|
||||
default:
|
||||
t.Error("unexpected endpoint", r.URL.Path)
|
||||
http.Error(w, "unexpected", 400)
|
||||
}
|
||||
}))
|
||||
defer server.Close()
|
||||
dial := func(ctx context.Context, _, _ string) (net.Conn, error) {
|
||||
return (&net.Dialer{}).DialContext(ctx, "tcp", server.Listener.Addr().String())
|
||||
}
|
||||
result, err := MonitorRabbit(context.Background(), model.Proxy{TargetHost: "localhost", TargetUser: "monitor", TargetPassword: "secret"}, dial)
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if len(seen) != 2 || len(result.Metrics) != 3 {
|
||||
t.Fatalf("unexpected diagnostics: %+v; requests %v", result, seen)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,93 @@
|
||||
package httpproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"io"
|
||||
"linkport/internal/model"
|
||||
"net"
|
||||
"net/http"
|
||||
"net/http/httptest"
|
||||
"strconv"
|
||||
"strings"
|
||||
"sync/atomic"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func TestRabbitManagementReadonlyPolicy(t *testing.T) {
|
||||
for _, path := range []string{"/", "/js/main.js", "/api/overview", "/api/queues/%2F/q", "/api/queues/%2F/q/bindings", "/api/nodes/node/memory", "/api/health/checks/alarms"} {
|
||||
if !rabbitReadonlyAllowed(httptest.NewRequest("GET", path, nil)) {
|
||||
t.Errorf("read rejected: %s", path)
|
||||
}
|
||||
}
|
||||
for _, path := range []string{"//api/aliveness-test/%2F", "/api/aliveness-test/%2F", "/api/queues/%2F/q/get", "/api/queues/%2F/q/contents", "/api/plugin/action", "/api/definitions", "/js/../api/aliveness-test", "/api/queues/%2e%2e%2faliveness-test", "/api/queues/%252e%252e"} {
|
||||
if rabbitReadonlyAllowed(httptest.NewRequest("GET", path, nil)) {
|
||||
t.Errorf("unsafe path allowed: %s", path)
|
||||
}
|
||||
}
|
||||
for _, method := range []string{"POST", "PUT", "PATCH", "DELETE", "CONNECT", "OPTIONS"} {
|
||||
if rabbitReadonlyAllowed(httptest.NewRequest(method, "/api/queues/%2F/q", nil)) {
|
||||
t.Error("unsafe method allowed", method)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRabbitManagementMappedCredentialsPortAndReadonly(t *testing.T) {
|
||||
var requests atomic.Int32
|
||||
up := httptest.NewServer(http.HandlerFunc(func(w http.ResponseWriter, r *http.Request) {
|
||||
requests.Add(1)
|
||||
u, p, ok := r.BasicAuth()
|
||||
if !ok || u != "target" || p != "target-secret" {
|
||||
t.Error("unmapped target credentials")
|
||||
}
|
||||
io.WriteString(w, "management fixture")
|
||||
}))
|
||||
defer up.Close()
|
||||
host, port, _ := net.SplitHostPort(strings.TrimPrefix(up.URL, "http://"))
|
||||
n, _ := strconv.Atoi(port)
|
||||
p := model.Proxy{Type: "rabbitmq", TargetHost: host, TargetPort: 1, ManagementPort: n, TargetAccountID: "upstream", TargetAccounts: []model.TargetAccount{{ID: "upstream", Username: "target", Password: "target-secret"}}, GatewayAccounts: []model.GatewayAccount{{ID: "client", Username: "client", Password: "client-secret", AccessMode: "readonly"}}}
|
||||
l, err := net.Listen("tcp", "127.0.0.1:0")
|
||||
if err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
defer l.Close()
|
||||
ctx, cancel := context.WithCancel(context.Background())
|
||||
defer cancel()
|
||||
done := make(chan error, 1)
|
||||
go func() {
|
||||
c, e := l.Accept()
|
||||
if e != nil {
|
||||
done <- e
|
||||
return
|
||||
}
|
||||
done <- New().Serve(ctx, c, p, (&net.Dialer{}).DialContext, nil, nil)
|
||||
}()
|
||||
tr := &http.Transport{}
|
||||
defer tr.CloseIdleConnections()
|
||||
client := &http.Client{Transport: tr, Timeout: 3 * time.Second}
|
||||
for _, check := range []struct {
|
||||
method, path string
|
||||
status int
|
||||
}{{"GET", "/api/overview", 200}, {"POST", "/api/queues/%2F/q/get", 403}, {"PUT", "/api/queues/%2F/q", 403}, {"DELETE", "/api/queues/%2F/q", 403}, {"GET", "/api/aliveness-test/%2F", 403}, {"GET", "/api/queues/%2F/q", 200}} {
|
||||
r, _ := http.NewRequest(check.method, "http://"+l.Addr().String()+check.path, nil)
|
||||
r.SetBasicAuth("client", "client-secret")
|
||||
resp, e := client.Do(r)
|
||||
if e != nil {
|
||||
t.Fatal(e)
|
||||
}
|
||||
io.Copy(io.Discard, resp.Body)
|
||||
resp.Body.Close()
|
||||
if resp.StatusCode != check.status {
|
||||
t.Fatalf("%s %s: %d", check.method, check.path, resp.StatusCode)
|
||||
}
|
||||
}
|
||||
if requests.Load() != 2 {
|
||||
t.Fatal("rejected request reached upstream", requests.Load())
|
||||
}
|
||||
cancel()
|
||||
select {
|
||||
case <-done:
|
||||
case <-time.After(3 * time.Second):
|
||||
t.Fatal("HTTP gateway did not stop")
|
||||
}
|
||||
}
|
||||
@@ -556,6 +556,7 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
if e := audit("allowed"); e != nil {
|
||||
return errors.New("command audit unavailable")
|
||||
}
|
||||
started := time.Now()
|
||||
if e := writeAll(u, m.raw); e != nil {
|
||||
return e
|
||||
}
|
||||
@@ -579,6 +580,7 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
if e := writeAll(c, response.raw); e != nil {
|
||||
return e
|
||||
}
|
||||
model.ObserveExecution(ctx, "mongodb", cmd, started)
|
||||
}
|
||||
}
|
||||
|
||||
|
||||
@@ -0,0 +1,45 @@
|
||||
package mongoproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"go.mongodb.org/mongo-driver/v2/bson"
|
||||
"linkport/internal/model"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (h *Handler) Monitor(ctx context.Context, p model.Proxy, dial model.DialFunc) (model.DatabaseMonitoring, error) {
|
||||
out := model.DatabaseMonitoring{Metrics: []model.ServerMetric{}, Notices: []string{}}
|
||||
u, err := connect(ctx, p, dial)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
defer u.Close()
|
||||
stop := context.AfterFunc(ctx, func() { u.Close() })
|
||||
defer stop()
|
||||
d, err := exchange(u, bson.D{{Key: "serverStatus", Value: int32(1)}, {Key: "$db", Value: "admin"}})
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if !ok(d) {
|
||||
out.Notices = append(out.Notices, "MongoDB serverStatus 不可用，请检查 clusterMonitor 权限")
|
||||
} else {
|
||||
for _, path := range []string{"connections.current", "connections.available", "connections.active", "mem.resident", "mem.virtual", "opcounters.query", "opcounters.insert", "opcounters.update", "opcounters.delete", "network.bytesIn", "network.bytesOut", "globalLock.currentQueue.total", "globalLock.activeClients.total"} {
|
||||
var value any = d
|
||||
for _, key := range strings.Split(path, ".") {
|
||||
doc, ok := value.(bson.D)
|
||||
if !ok {
|
||||
value = nil
|
||||
break
|
||||
}
|
||||
value = lookup(doc, key)
|
||||
}
|
||||
if value != nil {
|
||||
out.Metrics = append(out.Metrics, model.ServerMetric{Name: path, Value: float64(number(value))})
|
||||
}
|
||||
}
|
||||
}
|
||||
out.Notices = append(out.Notices, "mem 指标单位为 MiB；操作与流量为累计计数。仅当前连接节点，不启用 profiler、不读取 system.profile。")
|
||||
out.CheckedAt = time.Now().UTC()
|
||||
return out, nil
|
||||
}
|
||||
@@ -0,0 +1,42 @@
|
||||
package mysqlproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"linkport/internal/model"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (h *Handler) Monitor(ctx context.Context, p model.Proxy, dial model.DialFunc) (model.DatabaseMonitoring, error) {
|
||||
out := model.DatabaseMonitoring{Metrics: []model.ServerMetric{}, Notices: []string{}}
|
||||
c, err := open(ctx, p, dial, "")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
defer c.Close()
|
||||
stop := context.AfterFunc(ctx, func() { c.Close() })
|
||||
defer stop()
|
||||
for _, query := range []string{
|
||||
"SHOW GLOBAL STATUS WHERE Variable_name IN ('Threads_connected','Threads_running','Max_used_connections','Slow_queries','Questions','Uptime','Innodb_buffer_pool_reads','Innodb_buffer_pool_read_requests','Innodb_row_lock_current_waits','Bytes_received','Bytes_sent')",
|
||||
"SHOW GLOBAL VARIABLES WHERE Variable_name IN ('max_connections','innodb_buffer_pool_size')",
|
||||
} {
|
||||
r, e := c.Execute(query)
|
||||
if e != nil {
|
||||
out.Notices = append(out.Notices, "MySQL 全局状态或配置指标不可用，请检查目标账号权限")
|
||||
continue
|
||||
}
|
||||
for row := 0; row < r.RowNumber(); row++ {
|
||||
name, _ := r.GetString(row, 0)
|
||||
value, _ := r.GetString(row, 1)
|
||||
n, e := strconv.ParseFloat(value, 64)
|
||||
if e == nil {
|
||||
out.Metrics = append(out.Metrics, model.ServerMetric{Name: strings.Clone(name), Value: n})
|
||||
}
|
||||
}
|
||||
r.Close()
|
||||
}
|
||||
out.Notices = append(out.Notices, "仅采集服务端状态，不读取上游慢日志。CPU、宿主机负载不由这些数据库指标推算。")
|
||||
out.CheckedAt = time.Now().UTC()
|
||||
return out, nil
|
||||
}
|
||||
@@ -7,7 +7,7 @@ import (
|
||||
)
|
||||
|
||||
var errReadOnly = errors.New("request denied by read-only policy")
|
||||
var functions = wordSet("count sum avg min max abs ceil ceiling floor round coalesce ifnull nullif if concat concat_ws length char_length lower upper substring trim replace cast convert date year month day now current_timestamp database schema version user current_user hex unhex lpad")
|
||||
var functions = wordSet("count sum avg min max abs ceil ceiling floor round coalesce ifnull nullif if concat concat_ws length char_length lower upper substring substring_index left trim replace cast convert date year month day now current_timestamp from_unixtime unix_timestamp time_to_sec sec_to_time database schema version user current_user hex unhex lpad")
|
||||
|
||||
func wordSet(s string) map[string]bool {
|
||||
m := map[string]bool{}
|
||||
@@ -83,6 +83,8 @@ func checkReadOnlyStatement(query string) error {
|
||||
switch n.(type) {
|
||||
case *sqlparser.AndExpr, *sqlparser.OrExpr, *sqlparser.XorExpr, *sqlparser.NotExpr, *sqlparser.ComparisonExpr, *sqlparser.BetweenExpr, *sqlparser.IsExpr, *sqlparser.ExistsExpr, *sqlparser.Literal, *sqlparser.Argument, *sqlparser.NullVal, sqlparser.BoolVal, *sqlparser.ColName, sqlparser.ValTuple, *sqlparser.Subquery, sqlparser.ListArg, *sqlparser.BinaryExpr, *sqlparser.UnaryExpr, *sqlparser.IntroducerExpr, *sqlparser.CollateExpr, *sqlparser.CaseExpr, *sqlparser.CastExpr, *sqlparser.Default:
|
||||
case *sqlparser.Count, *sqlparser.CountStar, *sqlparser.Sum, *sqlparser.Avg, *sqlparser.Min, *sqlparser.Max, *sqlparser.ConvertExpr, *sqlparser.ConvertUsingExpr, *sqlparser.SubstrExpr:
|
||||
case *sqlparser.CharExpr:
|
||||
// CHAR is a dedicated AST node; the walk still validates every argument.
|
||||
case *sqlparser.JSONObjectExpr, *sqlparser.JSONArrayAgg:
|
||||
case *sqlparser.TrimFuncExpr, *sqlparser.CurTimeFuncExpr:
|
||||
s := strings.ToLower(sqlparser.String(n))
|
||||
|
||||
@@ -0,0 +1,72 @@
|
||||
package mysqlproxy
|
||||
|
||||
import "testing"
|
||||
|
||||
func TestReadOnlyMonitoringQueries(t *testing.T) {
|
||||
wave := `SELECT id, code, startPickUploadSts, startPickUploadRetryCount,
|
||||
pickingGuideUploadSts, pickingGuideUploadRetryCount, completeTaskUploadSts, completeTaskUploadRetryCount
|
||||
FROM shopee_wave WHERE created >= FROM_UNIXTIME(?) AND created < FROM_UNIXTIME(?)
|
||||
AND (startPickUploadSts = 950 OR pickingGuideUploadSts = 950 OR completeTaskUploadSts = 950)
|
||||
ORDER BY created DESC,id DESC LIMIT 200`
|
||||
allowed := []string{
|
||||
"SELECT UNIX_TIMESTAMP('2026-10-12 00:00:00'), TIME_TO_SEC('00:00:05'), SEC_TO_TIME(5), LEFT('abc', 2)",
|
||||
`EXPLAIN FORMAT=TRADITIONAL SELECT thread_id,UNIX_TIMESTAMP(start_time),CAST(TIME_TO_SEC(query_time) AS CHAR),
|
||||
db,LEFT(CONVERT(sql_text USING utf8mb4),2000),rows_examined,rows_sent
|
||||
FROM mysql.slow_log WHERE start_time>=FROM_UNIXTIME(?) AND start_time<FROM_UNIXTIME(?) AND db=DATABASE() AND query_time>=SEC_TO_TIME(?)
|
||||
ORDER BY start_time DESC LIMIT 200`,
|
||||
"SELECT FROM_UNIXTIME(0)",
|
||||
"SELECT FROM_UNIXTIME(0, '%Y-%m-%d')",
|
||||
"SELECT CHAR(39)",
|
||||
"SELECT CHAR(65, 66 USING utf8mb4)",
|
||||
"SELECT SUBSTRING_INDEX('a@b', '@', 1)",
|
||||
wave,
|
||||
"EXPLAIN " + wave,
|
||||
"EXPLAIN FORMAT=TRADITIONAL " + wave,
|
||||
`SELECT COUNT(*) FROM information_schema.USER_PRIVILEGES
|
||||
WHERE PRIVILEGE_TYPE IN ('INSERT','UPDATE','DELETE','CREATE','DROP','ALTER','SUPER','GRANT OPTION')
|
||||
AND GRANTEE=CONCAT(CHAR(39),SUBSTRING_INDEX(CURRENT_USER(),'@',1),CHAR(39),'@',CHAR(39),SUBSTRING_INDEX(CURRENT_USER(),'@',-1),CHAR(39))`,
|
||||
"SELECT @@version,@@slow_query_log,@@log_output,CAST(@@long_query_time AS CHAR)",
|
||||
`SELECT TABLE_NAME,COUNT(*) FROM information_schema.STATISTICS
|
||||
WHERE TABLE_SCHEMA=DATABASE() AND SEQ_IN_INDEX=1 AND
|
||||
((TABLE_NAME='shopee_wave' AND COLUMN_NAME='created') OR (TABLE_NAME='location_inventory' AND COLUMN_NAME='lastUpdated')) GROUP BY TABLE_NAME`,
|
||||
}
|
||||
for _, query := range allowed {
|
||||
t.Run(query, func(t *testing.T) {
|
||||
if err := CheckReadOnly(query); err != nil {
|
||||
t.Fatalf("read-only monitoring query rejected: %v", err)
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadOnlyMonitoringFunctionsStillCheckArguments(t *testing.T) {
|
||||
denied := []string{
|
||||
"SELECT UNIX_TIMESTAMP(SLEEP(1))",
|
||||
"SELECT TIME_TO_SEC(LOAD_FILE('/tmp/time'))",
|
||||
"SELECT SEC_TO_TIME(GET_LOCK('x', 1))",
|
||||
"SELECT LEFT('abc', (SELECT SLEEP(1)))",
|
||||
"SELECT app.unix_timestamp()",
|
||||
"SELECT app.time_to_sec('00:00:05')",
|
||||
"SELECT app.sec_to_time(5)",
|
||||
"SELECT app.left('abc', 2)",
|
||||
"SELECT FROM_UNIXTIME(SLEEP(1))",
|
||||
"SELECT FROM_UNIXTIME(0, LOAD_FILE('/tmp/format'))",
|
||||
"SELECT CHAR(SLEEP(1))",
|
||||
"SELECT CHAR(65, (SELECT SLEEP(1)) USING utf8mb4)",
|
||||
"SELECT CHAR(@x := 65)",
|
||||
"SELECT SUBSTRING_INDEX(LOAD_FILE('/tmp/file'), '@', 1)",
|
||||
"SELECT SUBSTRING_INDEX('a@b', '@', GET_LOCK('x', 1))",
|
||||
"SELECT app.from_unixtime(0)",
|
||||
"SELECT app.substring_index('a@b', '@', 1)",
|
||||
"EXPLAIN ANALYZE SELECT FROM_UNIXTIME(0)",
|
||||
"EXPLAIN FORMAT=TRADITIONAL SELECT CHAR(SLEEP(1))",
|
||||
"SELECT CHAR(65); DELETE FROM t",
|
||||
}
|
||||
for _, query := range denied {
|
||||
t.Run(query, func(t *testing.T) {
|
||||
if err := CheckReadOnly(query); err == nil {
|
||||
t.Fatal("unsafe query accepted")
|
||||
}
|
||||
})
|
||||
}
|
||||
}
|
||||
@@ -66,6 +66,9 @@ func (a *queryResultAudit) finish(complete bool) error {
|
||||
if !a.startedAt.IsZero() {
|
||||
duration := float64(time.Since(a.startedAt)) / float64(time.Millisecond)
|
||||
a.output.Result.DurationMS = &duration
|
||||
if complete {
|
||||
model.ObserveExecution(a.ctx, "mysql", "query", a.startedAt)
|
||||
}
|
||||
}
|
||||
return a.persist(nil)
|
||||
}
|
||||
|
||||
@@ -41,6 +41,8 @@ func TestQueryResultAuditRecordsObservedDuration(t *testing.T) {
|
||||
return nil
|
||||
})
|
||||
audit := &queryResultAudit{ctx: ctx}
|
||||
var executions []model.Execution
|
||||
audit.ctx = model.WithExecutionObserver(ctx, func(e model.Execution) { executions = append(executions, e) })
|
||||
command := model.Command{AuditID: "query", RecordedAt: time.Now().UTC()}
|
||||
if err := audit.start(command); err != nil {
|
||||
t.Fatal(err)
|
||||
@@ -50,6 +52,9 @@ func TestQueryResultAuditRecordsObservedDuration(t *testing.T) {
|
||||
t.Fatal(err)
|
||||
}
|
||||
result := captured[len(captured)-1].Result
|
||||
if len(executions) != 1 || executions[0].Protocol != "mysql" || executions[0].DurationMS < 3500 {
|
||||
t.Fatalf("missing completed gateway execution: %+v", executions)
|
||||
}
|
||||
if result.State != "completed" || result.DurationMS == nil || *result.DurationMS < 3500 {
|
||||
t.Fatal("completed duration was not recorded", result)
|
||||
}
|
||||
|
||||
@@ -0,0 +1,40 @@
|
||||
package pgproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"linkport/internal/model"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (h *Handler) Monitor(ctx context.Context, p model.Proxy, dial model.DialFunc) (model.DatabaseMonitoring, error) {
|
||||
out := model.DatabaseMonitoring{Metrics: []model.ServerMetric{}, Notices: []string{}}
|
||||
c, err := open(ctx, p, dial, p.AuthSource)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
defer c.Close(context.Background())
|
||||
for _, query := range []string{
|
||||
"SELECT numbackends, xact_commit, xact_rollback, blks_read, blks_hit, deadlocks, temp_bytes FROM pg_stat_database WHERE datname = current_database()",
|
||||
"SELECT count(*) FILTER (WHERE state = 'active') AS active_sessions, count(*) FILTER (WHERE wait_event_type = 'Lock') AS lock_waiting_sessions FROM pg_stat_activity",
|
||||
} {
|
||||
results, err := c.Exec(ctx, query).ReadAll()
|
||||
if err != nil {
|
||||
out.Notices = append(out.Notices, "PostgreSQL 统计视图不可用，请检查 pg_monitor/pg_read_all_stats 权限")
|
||||
continue
|
||||
}
|
||||
for _, result := range results {
|
||||
for _, row := range result.Rows {
|
||||
for i, value := range row {
|
||||
n, e := strconv.ParseFloat(string(value), 64)
|
||||
if e == nil {
|
||||
out.Metrics = append(out.Metrics, model.ServerMetric{Name: string(result.FieldDescriptions[i].Name), Value: n})
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
out.Notices = append(out.Notices, "数据库统计为累计值；活动会话受目标账号可见权限限制，不代表宿主机 CPU/内存。")
|
||||
out.CheckedAt = time.Now().UTC()
|
||||
return out, nil
|
||||
}
|
||||
@@ -208,6 +208,7 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
raw.SetDeadline(time.Time{})
|
||||
connection.Conn.SetDeadline(time.Time{})
|
||||
done := make(chan error, 1)
|
||||
var timing model.ExecutionQueue
|
||||
go func() {
|
||||
for {
|
||||
message, e := connection.Frontend.Receive()
|
||||
@@ -222,8 +223,12 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
raw.Close()
|
||||
return
|
||||
}
|
||||
if _, ok := message.(*pgproto3.ReadyForQuery); ok {
|
||||
timing.Finish(ctx, "pgsql")
|
||||
}
|
||||
}
|
||||
}()
|
||||
var extendedStarted time.Time
|
||||
for {
|
||||
message, err := backend.Receive()
|
||||
if err != nil {
|
||||
@@ -250,6 +255,20 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
if err = model.AuditCommand(ctx, kind, text, "allowed"); err != nil {
|
||||
return err
|
||||
}
|
||||
switch message.(type) {
|
||||
case *pgproto3.Query:
|
||||
timing.Start("query batch")
|
||||
case *pgproto3.Parse, *pgproto3.Bind, *pgproto3.Execute:
|
||||
if extendedStarted.IsZero() {
|
||||
extendedStarted = time.Now()
|
||||
}
|
||||
case *pgproto3.Sync:
|
||||
if extendedStarted.IsZero() {
|
||||
extendedStarted = time.Now()
|
||||
}
|
||||
timing.StartAt("extended query cycle", extendedStarted)
|
||||
extendedStarted = time.Time{}
|
||||
}
|
||||
connection.Frontend.Send(message)
|
||||
if err = connection.Frontend.Flush(); err != nil {
|
||||
return err
|
||||
|
||||
@@ -0,0 +1,61 @@
|
||||
package rabbitproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"fmt"
|
||||
"linkport/internal/model"
|
||||
"sync"
|
||||
)
|
||||
|
||||
// Only RPCs with a mandatory reply are timed; async publish/consumer traffic is not execution completion.
|
||||
type rpcTimings struct {
|
||||
mu sync.Mutex
|
||||
pending map[uint64]*model.ExecutionQueue
|
||||
}
|
||||
|
||||
func (t *rpcTimings) request(f frame) {
|
||||
if f.kind != 1 || len(f.body) < 4 {
|
||||
return
|
||||
}
|
||||
class, id := binary.BigEndian.Uint16(f.body), binary.BigEndian.Uint16(f.body[2:])
|
||||
if class == 20 && (id == 10 || id == 40) {
|
||||
} else if (class == 40 || class == 50) && id == 10 && len(f.body) >= 7 {
|
||||
_, rest, err := short(f.body[6:])
|
||||
if class == 40 && err == nil {
|
||||
_, rest, err = short(rest)
|
||||
}
|
||||
if err != nil || len(rest) == 0 || rest[0]&16 != 0 {
|
||||
return
|
||||
}
|
||||
} else {
|
||||
return
|
||||
}
|
||||
key := uint64(f.channel)<<32 | uint64(class)<<16 | uint64(id+1)
|
||||
t.mu.Lock()
|
||||
defer t.mu.Unlock()
|
||||
if t.pending == nil {
|
||||
t.pending = map[uint64]*model.ExecutionQueue{}
|
||||
}
|
||||
q := t.pending[key]
|
||||
if q == nil {
|
||||
if len(t.pending) >= 4096 {
|
||||
return
|
||||
}
|
||||
q = &model.ExecutionQueue{}
|
||||
t.pending[key] = q
|
||||
}
|
||||
q.Start(fmt.Sprintf("AMQP %d.%d", class, id))
|
||||
}
|
||||
func (t *rpcTimings) response(ctx context.Context, f frame) {
|
||||
if f.kind != 1 || len(f.body) < 4 {
|
||||
return
|
||||
}
|
||||
key := uint64(f.channel)<<32 | uint64(binary.BigEndian.Uint16(f.body))<<16 | uint64(binary.BigEndian.Uint16(f.body[2:]))
|
||||
t.mu.Lock()
|
||||
q := t.pending[key]
|
||||
t.mu.Unlock()
|
||||
if q != nil {
|
||||
q.Finish(ctx, "rabbitmq")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
package rabbitproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"linkport/internal/model"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestRPCTimingMatchesChannelAndExcludesNoWait(t *testing.T) {
|
||||
var observed []model.Execution
|
||||
ctx := model.WithExecutionObserver(context.Background(), func(e model.Execution) { observed = append(observed, e) })
|
||||
var timings rpcTimings
|
||||
timings.request(passiveDeclaration(50, 1))
|
||||
reply := frame{kind: 1, channel: 2, body: []byte{0, 50, 0, 11}}
|
||||
timings.response(ctx, reply)
|
||||
if len(observed) != 0 {
|
||||
t.Fatal("matched wrong channel")
|
||||
}
|
||||
reply.channel = 1
|
||||
timings.response(ctx, reply)
|
||||
timings.response(ctx, reply)
|
||||
timings.request(passiveDeclaration(50, 17))
|
||||
timings.response(ctx, reply)
|
||||
if len(observed) != 1 || observed[0].Protocol != "rabbitmq" {
|
||||
t.Fatalf("unexpected timings: %+v", observed)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,11 @@
|
||||
package rabbitproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/protocol/httpproxy"
|
||||
)
|
||||
|
||||
func (h *Handler) Monitor(ctx context.Context, p model.Proxy, dial model.DialFunc) (model.DatabaseMonitoring, error) {
|
||||
return httpproxy.MonitorRabbit(ctx, p, dial)
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package rabbitproxy
|
||||
|
||||
import "encoding/binary"
|
||||
|
||||
// Strict readonly permits passive inspection but never delivery or acknowledgement.
|
||||
func readonlyAllowed(f frame) bool {
|
||||
if f.kind == 8 {
|
||||
return f.channel == 0 && len(f.body) == 0
|
||||
}
|
||||
if f.kind != 1 || len(f.body) < 4 {
|
||||
return false
|
||||
}
|
||||
class, id := binary.BigEndian.Uint16(f.body), binary.BigEndian.Uint16(f.body[2:])
|
||||
switch class {
|
||||
case 10:
|
||||
return id == 50 || id == 51 // connection.close / close-ok
|
||||
case 20:
|
||||
return id == 10 || id == 20 || id == 21 || id == 40 || id == 41
|
||||
case 40, 50:
|
||||
if id != 10 || len(f.body) < 7 {
|
||||
return false
|
||||
}
|
||||
name, rest, err := short(f.body[6:])
|
||||
if err != nil || name == "" {
|
||||
return false
|
||||
}
|
||||
if class == 40 {
|
||||
_, rest, err = short(rest) // exchange type precedes flags
|
||||
}
|
||||
// Only passive inspection, never active declaration or auto-delete.
|
||||
if err != nil || len(rest) < 5 || rest[0]&1 == 0 || rest[0]&^byte(1|2|16) != 0 {
|
||||
return false
|
||||
}
|
||||
_, tail, err := long(rest[1:])
|
||||
return err == nil && len(tail) == 0
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,161 @@
|
||||
package rabbitproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"io"
|
||||
"linkport/internal/model"
|
||||
"net"
|
||||
"testing"
|
||||
"time"
|
||||
)
|
||||
|
||||
func passiveDeclaration(class uint16, flags byte) frame {
|
||||
body := binary.BigEndian.AppendUint16(nil, class)
|
||||
body = append(body, 0, 10, 0, 0, 1, 'q')
|
||||
if class == 40 {
|
||||
body = append(body, 6, 'd', 'i', 'r', 'e', 'c', 't')
|
||||
}
|
||||
body = append(body, flags, 0, 0, 0, 0)
|
||||
return frame{1, 1, body}
|
||||
}
|
||||
|
||||
func TestStrictReadonlyPolicy(t *testing.T) {
|
||||
for _, class := range []uint16{40, 50} {
|
||||
if !readonlyAllowed(passiveDeclaration(class, 1)) {
|
||||
t.Fatal("passive inspection rejected")
|
||||
}
|
||||
for _, flags := range []byte{0, 2, 4, 8, 5, 9, 255} {
|
||||
if readonlyAllowed(passiveDeclaration(class, flags)) {
|
||||
t.Fatalf("unsafe declaration accepted: %d %d", class, flags)
|
||||
}
|
||||
}
|
||||
f := passiveDeclaration(class, 1)
|
||||
for n := 0; n < len(f.body); n++ {
|
||||
if readonlyAllowed(frame{1, 1, f.body[:n]}) {
|
||||
t.Fatalf("truncated declaration allowed: %d", n)
|
||||
}
|
||||
}
|
||||
}
|
||||
for _, f := range []frame{{2, 1, []byte{}}, {3, 1, []byte("message")}, {8, 1, nil}, {1, 1, []byte{0, 99, 0, 1}}} {
|
||||
if readonlyAllowed(f) {
|
||||
t.Fatal("unknown/content frame accepted")
|
||||
}
|
||||
}
|
||||
for _, id := range []uint16{10, 20, 30, 40, 70, 80, 90, 100, 110, 120} {
|
||||
if readonlyAllowed(frame{1, 1, []byte{0, 60, byte(id >> 8), byte(id)}}) {
|
||||
t.Fatalf("basic method %d accepted", id)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestStrictReadonlyRejectsBeforeUpstream(t *testing.T) {
|
||||
for _, blocked := range []frame{
|
||||
{1, 1, []byte{0, 60, 0, 20}}, // consume
|
||||
{1, 1, []byte{0, 60, 0, 70}}, // get
|
||||
{1, 1, []byte{0, 60, 0, 80}}, // ack
|
||||
{1, 1, []byte{0, 60, 0, 40}}, // publish
|
||||
{1, 1, []byte{0, 50, 0, 40}}, // delete
|
||||
passiveDeclaration(50, 0),
|
||||
{3, 1, []byte("orphan content")},
|
||||
} {
|
||||
ctx, cancel := context.WithTimeout(context.Background(), 5*time.Second)
|
||||
seen := make(chan frame, 4)
|
||||
upDone := make(chan struct{})
|
||||
dial := func(context.Context, string, string) (net.Conn, error) {
|
||||
a, b := net.Pipe()
|
||||
go func() {
|
||||
defer close(upDone)
|
||||
defer b.Close()
|
||||
b.SetDeadline(time.Now().Add(5 * time.Second))
|
||||
if _, err := io.ReadFull(b, make([]byte, 8)); err != nil {
|
||||
return
|
||||
}
|
||||
g := addLong([]byte{0, 10, 0, 10, 0, 9}, nil)
|
||||
g = addLong(g, []byte("PLAIN"))
|
||||
g = addLong(g, []byte("en_US"))
|
||||
if write(b, frame{1, 0, g}) != nil {
|
||||
return
|
||||
}
|
||||
if _, err := read(b); err != nil {
|
||||
return
|
||||
}
|
||||
if write(b, frame{1, 0, []byte{0, 10, 0, 30, 0, 0, 0, 2, 0, 0, 0, 0}}) != nil {
|
||||
return
|
||||
}
|
||||
if _, err := read(b); err != nil {
|
||||
return
|
||||
}
|
||||
if _, err := read(b); err != nil {
|
||||
return
|
||||
}
|
||||
if write(b, frame{1, 0, []byte{0, 10, 0, 41, 0}}) != nil {
|
||||
return
|
||||
}
|
||||
for {
|
||||
f, err := read(b)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
seen <- f
|
||||
if method(f, 50, 10) {
|
||||
write(b, frame{1, 1, []byte{0, 50, 0, 11, 1, 'q', 0, 0, 0, 3, 0, 0, 0, 0}})
|
||||
}
|
||||
}
|
||||
}()
|
||||
return a, nil
|
||||
}
|
||||
client, raw := net.Pipe()
|
||||
client.SetDeadline(time.Now().Add(5 * time.Second))
|
||||
p := model.Proxy{Type: "rabbitmq", TargetHost: "fixture", TargetPort: 5672, GatewayAccounts: []model.GatewayAccount{{ID: "client", Username: "client", Password: "password", AccessMode: "readonly"}}}
|
||||
p.TargetAccountID = "upstream"
|
||||
p.TargetAccounts = []model.TargetAccount{{ID: "upstream", Username: "target", Password: "secret"}}
|
||||
done := make(chan error, 1)
|
||||
go func() { done <- New().Serve(ctx, raw, p, dial, nil, nil) }()
|
||||
if _, err := client.Write(protocolHeader); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err := read(client); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err := write(client, startOK("client", "password")); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
tune, err := read(client)
|
||||
if err != nil {
|
||||
t.Fatalf("handshake: %v; gateway: %v", err, <-done)
|
||||
}
|
||||
tune.body[3] = 31
|
||||
if err = write(client, tune); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = write(client, frame{1, 0, []byte{0, 10, 0, 40, 1, '/', 0, 0}}); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if _, err = read(client); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if err = write(client, passiveDeclaration(50, 1)); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
if f, err := read(client); err != nil || !method(f, 50, 11) {
|
||||
t.Fatalf("passive reply: %+v %v", f, err)
|
||||
}
|
||||
if err = write(client, blocked); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
f, err := read(client)
|
||||
if err != nil || !method(f, 10, 50) || binary.BigEndian.Uint16(f.body[4:]) != 403 {
|
||||
t.Fatalf("missing access refusal: %+v %v", f, err)
|
||||
}
|
||||
client.Close()
|
||||
if err = <-done; err == nil {
|
||||
t.Fatal("expected policy rejection")
|
||||
}
|
||||
<-upDone
|
||||
cancel()
|
||||
if len(seen) != 1 {
|
||||
t.Fatalf("forbidden frame reached target: %d frames", len(seen))
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -50,7 +50,7 @@ func read(c net.Conn) (frame, error) {
|
||||
}
|
||||
return frame{h[0], binary.BigEndian.Uint16(h[1:3]), b[:n]}, nil
|
||||
}
|
||||
func write(c net.Conn, f frame) error {
|
||||
func write(c io.Writer, f frame) error {
|
||||
b := []byte{f.kind, byte(f.channel >> 8), byte(f.channel)}
|
||||
b = binary.BigEndian.AppendUint32(b, uint32(len(f.body)))
|
||||
b = append(b, f.body...)
|
||||
@@ -312,6 +312,8 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
}
|
||||
raw.SetDeadline(time.Time{})
|
||||
up.SetDeadline(time.Time{})
|
||||
out := &transport.LockedWriter{Writer: c}
|
||||
var timing rpcTimings
|
||||
go func() {
|
||||
defer raw.Close()
|
||||
for {
|
||||
@@ -319,9 +321,10 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
if e != nil {
|
||||
return
|
||||
}
|
||||
if write(c, f) != nil {
|
||||
if write(out, f) != nil {
|
||||
return
|
||||
}
|
||||
timing.response(ctx, f)
|
||||
}
|
||||
}()
|
||||
for {
|
||||
@@ -329,6 +332,28 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
if err != nil {
|
||||
return err
|
||||
}
|
||||
if p.AccessMode == "readonly" && !readonlyAllowed(f) {
|
||||
if event != nil {
|
||||
event(model.EventReadOnlyRejected, "rabbitmq_readonly")
|
||||
}
|
||||
if err = model.AuditCommand(ctx, "readonly.reject", fmt.Sprintf("channel=%d", f.channel), "denied"); err != nil {
|
||||
return err
|
||||
}
|
||||
// Close the stream so queued publish content cannot become orphan frames.
|
||||
message := "gateway readonly policy: only passive inspection is allowed"
|
||||
body := binary.BigEndian.AppendUint16([]byte{0, 10, 0, 50}, 403)
|
||||
body = append(body, byte(len(message)))
|
||||
body = append(body, message...)
|
||||
if f.kind == 1 && len(f.body) >= 4 {
|
||||
body = append(body, f.body[:4]...)
|
||||
} else {
|
||||
body = append(body, 0, 0, 0, 0)
|
||||
}
|
||||
if err = write(out, frame{kind: 1, body: body}); err != nil {
|
||||
return err
|
||||
}
|
||||
return errors.New("AMQP gateway readonly policy rejected client frame")
|
||||
}
|
||||
if f.kind == 1 {
|
||||
if len(f.body) < 4 {
|
||||
return errors.New("invalid AMQP method")
|
||||
@@ -347,6 +372,7 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
return err
|
||||
}
|
||||
}
|
||||
timing.request(f)
|
||||
if err = write(up, f); err != nil {
|
||||
return err
|
||||
}
|
||||
|
||||
@@ -245,6 +245,7 @@ func serveCluster(ctx context.Context, c net.Conn, reader *bufio.Reader, raw net
|
||||
return err
|
||||
}
|
||||
}
|
||||
started := time.Now()
|
||||
name := strings.ToUpper(args[0])
|
||||
commandBytes := 16 * len(args)
|
||||
for _, arg := range args {
|
||||
@@ -321,6 +322,15 @@ func serveCluster(ctx context.Context, c net.Conn, reader *bufio.Reader, raw net
|
||||
}
|
||||
}
|
||||
initial = false
|
||||
if v.Raw == nil && r.p.AccessMode == "readonly" && !readonlyAllowed(args) {
|
||||
if event != nil {
|
||||
event(model.EventReadOnlyRejected, "redis_readonly")
|
||||
}
|
||||
if err := model.AuditCommand(ctx, strings.ToLower(name), name, "denied"); err != nil {
|
||||
return err
|
||||
}
|
||||
v = redisError(readonlyError)
|
||||
}
|
||||
if v.Raw != nil {
|
||||
if err := transport.Write(c, v.Raw); err != nil {
|
||||
return err
|
||||
@@ -472,6 +482,7 @@ func serveCluster(ctx context.Context, c net.Conn, reader *bufio.Reader, raw net
|
||||
if err := transport.Write(c, v.Raw); err != nil {
|
||||
return err
|
||||
}
|
||||
model.ObserveExecution(ctx, "redis", name, started)
|
||||
if name == "QUIT" {
|
||||
return nil
|
||||
}
|
||||
|
||||
@@ -0,0 +1,75 @@
|
||||
package redisproxy
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"errors"
|
||||
"io"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/protocol/transport"
|
||||
"strconv"
|
||||
"strings"
|
||||
)
|
||||
|
||||
// Stream response bodies without imposing the diagnostic parser's 32 MiB limit.
|
||||
func copyReply(w io.Writer, r *bufio.Reader, depth int) (byte, error) {
|
||||
if depth > 64 {
|
||||
return 0, errors.New("unmeasured nesting")
|
||||
}
|
||||
header, err := r.ReadSlice('\n')
|
||||
if len(header) > 0 {
|
||||
if e := transport.Write(w, header); e != nil {
|
||||
return 0, e
|
||||
}
|
||||
}
|
||||
if err != nil || len(header) < 3 {
|
||||
return 0, errors.New("unmeasured RESP header")
|
||||
}
|
||||
kind := header[0]
|
||||
switch kind {
|
||||
case '+', '-', ':', ',', '#', '_', '(':
|
||||
return kind, nil
|
||||
case '$', '!', '=':
|
||||
n, e := strconv.ParseInt(strings.TrimSuffix(strings.TrimSuffix(string(header[1:]), "\n"), "\r"), 10, 64)
|
||||
if e != nil || n < -1 || n > 1<<60 {
|
||||
return kind, errors.New("unmeasured bulk")
|
||||
}
|
||||
if n >= 0 {
|
||||
_, e = io.CopyN(w, r, n+2)
|
||||
}
|
||||
return kind, e
|
||||
case '*', '~', '>', '%', '|':
|
||||
n, e := strconv.ParseInt(strings.TrimSpace(string(header[1:])), 10, 64)
|
||||
if e != nil || n < -1 || n > 1<<30 {
|
||||
return kind, errors.New("unmeasured aggregate")
|
||||
}
|
||||
if kind == '%' || kind == '|' {
|
||||
n *= 2
|
||||
}
|
||||
for i := int64(0); i < n; i++ {
|
||||
if _, e = copyReply(w, r, depth+1); e != nil {
|
||||
return kind, e
|
||||
}
|
||||
}
|
||||
if kind == '|' {
|
||||
return copyReply(w, r, depth+1)
|
||||
}
|
||||
return kind, nil
|
||||
}
|
||||
return kind, errors.New("unmeasured reply")
|
||||
}
|
||||
|
||||
func relayReplies(ctx context.Context, w io.Writer, source io.Reader, q *model.ExecutionQueue) {
|
||||
r := bufio.NewReader(source)
|
||||
for {
|
||||
kind, err := copyReply(w, r, 0)
|
||||
if err != nil {
|
||||
q.Disable()
|
||||
io.Copy(w, r)
|
||||
return
|
||||
}
|
||||
if kind != '>' {
|
||||
q.Finish(ctx, "redis")
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,38 @@
|
||||
package redisproxy
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"linkport/internal/model"
|
||||
"strings"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestResponseTimingPreservesPipelinePushAndUnknownRESP(t *testing.T) {
|
||||
for _, wire := range []string{"+OK\r\n>2\r\n+notice\r\n+data\r\n$5\r\nvalue\r\n", "|1\r\n+k\r\n+v\r\n+OK\r\n*2\r\n:1\r\n$-1\r\n", "$?\r\n;3\r\nabc\r\n;0\r\n"} {
|
||||
var got []model.Execution
|
||||
ctx := model.WithExecutionObserver(context.Background(), func(e model.Execution) { got = append(got, e) })
|
||||
var q model.ExecutionQueue
|
||||
q.Start("first")
|
||||
q.Start("second")
|
||||
var out bytes.Buffer
|
||||
relayReplies(ctx, &out, strings.NewReader(wire), &q)
|
||||
if out.String() != wire {
|
||||
t.Fatalf("response altered: %q", out.String())
|
||||
}
|
||||
want := 2
|
||||
if strings.HasPrefix(wire, "$?") {
|
||||
want = 0
|
||||
}
|
||||
if len(got) != want {
|
||||
t.Fatalf("wrong correlation: %+v", got)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestRedisPressureMetricsDoNotExposeINFOSecrets(t *testing.T) {
|
||||
got := redisMetrics("connected_clients:7\r\nused_memory:1024\r\nmaster_auth:private\r\nused_cpu_sys:2.5\r\n")
|
||||
if len(got) != 3 || got[0].Value != 7 || got[2].Value != 2.5 {
|
||||
t.Fatalf("metrics: %+v", got)
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,62 @@
|
||||
package redisproxy
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"linkport/internal/cluster"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/protocol/transport"
|
||||
"strconv"
|
||||
"strings"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (h *Handler) Monitor(ctx context.Context, p model.Proxy, dial model.DialFunc) (model.DatabaseMonitoring, error) {
|
||||
out := model.DatabaseMonitoring{Metrics: []model.ServerMetric{}, Notices: []string{}}
|
||||
c, err := open(ctx, p, dial)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
defer c.Close()
|
||||
stop := context.AfterFunc(ctx, func() { c.Close() })
|
||||
defer stop()
|
||||
r := bufio.NewReader(c)
|
||||
command := func(args ...string) (cluster.RedisValue, error) {
|
||||
if err := transport.Write(c, encode(args)); err != nil {
|
||||
return cluster.RedisValue{}, err
|
||||
}
|
||||
return cluster.ReadRedis(r)
|
||||
}
|
||||
v, err := command("INFO")
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
if v.Kind == '-' {
|
||||
out.Notices = append(out.Notices, "Redis INFO 不可用，请检查目标账号权限")
|
||||
} else {
|
||||
out.Metrics = redisMetrics(v.Text)
|
||||
}
|
||||
out.Notices = append(out.Notices, "仅采集 INFO，不读取上游慢日志。CPU 为累计秒数；集群指标仅代表本次连接节点。")
|
||||
out.CheckedAt = time.Now().UTC()
|
||||
return out, nil
|
||||
}
|
||||
|
||||
func redisMetrics(info string) []model.ServerMetric {
|
||||
units := map[string]string{"connected_clients": "connections", "blocked_clients": "connections", "used_memory": "bytes", "maxmemory": "bytes", "used_memory_rss": "bytes", "mem_fragmentation_ratio": "ratio", "instantaneous_ops_per_sec": "ops/s", "instantaneous_input_kbps": "KiB/s", "instantaneous_output_kbps": "KiB/s", "rejected_connections": "count", "evicted_keys": "count", "keyspace_hits": "count", "keyspace_misses": "count", "used_cpu_sys": "seconds (total)", "used_cpu_user": "seconds (total)"}
|
||||
out := []model.ServerMetric{}
|
||||
for _, line := range strings.Split(info, "\n") {
|
||||
name, value, ok := strings.Cut(strings.TrimSpace(line), ":")
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
unit, ok := units[name]
|
||||
if !ok {
|
||||
continue
|
||||
}
|
||||
n, err := strconv.ParseFloat(value, 64)
|
||||
if err == nil {
|
||||
out = append(out, model.ServerMetric{Name: name, Value: n, Unit: unit})
|
||||
}
|
||||
}
|
||||
return out
|
||||
}
|
||||
@@ -0,0 +1,50 @@
|
||||
package redisproxy
|
||||
|
||||
import "strings"
|
||||
|
||||
const readonlyError = "NOPERM command is not allowed by the gateway readonly policy"
|
||||
|
||||
// Unknown commands, scripts and streaming modes fail closed.
|
||||
func readonlyAllowed(args []string) bool {
|
||||
if len(args) == 0 {
|
||||
return false
|
||||
}
|
||||
switch strings.ToUpper(args[0]) {
|
||||
case "AUTH", "HELLO", "PING", "ECHO", "QUIT", "SELECT",
|
||||
"GET", "MGET", "GETRANGE", "STRLEN", "GETBIT", "BITCOUNT", "BITPOS",
|
||||
"EXISTS", "TYPE", "TTL", "PTTL", "EXPIRETIME", "PEXPIRETIME", "DUMP", "KEYS", "SCAN", "RANDOMKEY",
|
||||
"HGET", "HMGET", "HGETALL", "HEXISTS", "HKEYS", "HVALS", "HLEN", "HSTRLEN", "HSCAN", "HRANDFIELD",
|
||||
"LINDEX", "LLEN", "LRANGE", "LPOS",
|
||||
"SCARD", "SISMEMBER", "SMISMEMBER", "SMEMBERS", "SRANDMEMBER", "SSCAN", "SDIFF", "SINTER", "SUNION", "SINTERCARD",
|
||||
"ZCARD", "ZCOUNT", "ZLEXCOUNT", "ZSCORE", "ZMSCORE", "ZRANK", "ZREVRANK", "ZRANGE", "ZREVRANGE", "ZRANGEBYSCORE", "ZREVRANGEBYSCORE", "ZRANGEBYLEX", "ZREVRANGEBYLEX", "ZSCAN", "ZRANDMEMBER", "ZDIFF", "ZINTER", "ZUNION", "ZINTERCARD",
|
||||
"GEODIST", "GEOHASH", "GEOPOS", "GEOSEARCH",
|
||||
"XLEN", "XRANGE", "XREVRANGE", "XREAD", "XPENDING",
|
||||
"INFO", "DBSIZE", "TIME", "LASTSAVE", "ROLE":
|
||||
return true
|
||||
case "CLIENT":
|
||||
return subcommand(args, "ID", "INFO", "GETNAME", "SETNAME", "SETINFO")
|
||||
case "CLUSTER":
|
||||
return subcommand(args, "INFO", "NODES", "SLOTS", "SHARDS", "MYID", "KEYSLOT", "COUNTKEYSINSLOT", "GETKEYSINSLOT", "REPLICAS", "SLAVES")
|
||||
case "COMMAND":
|
||||
return len(args) == 1 || subcommand(args, "COUNT", "DOCS", "INFO", "LIST", "GETKEYS", "GETKEYSANDFLAGS")
|
||||
case "OBJECT":
|
||||
return subcommand(args, "ENCODING", "FREQ", "IDLETIME", "REFCOUNT")
|
||||
case "MEMORY":
|
||||
return subcommand(args, "USAGE", "STATS", "DOCTOR", "MALLOC-STATS")
|
||||
case "XINFO":
|
||||
return subcommand(args, "STREAM", "GROUPS", "CONSUMERS")
|
||||
}
|
||||
return false
|
||||
}
|
||||
|
||||
func subcommand(args []string, allowed ...string) bool {
|
||||
if len(args) < 2 {
|
||||
return false
|
||||
}
|
||||
for _, name := range allowed {
|
||||
if strings.EqualFold(args[1], name) {
|
||||
return true
|
||||
}
|
||||
}
|
||||
return false
|
||||
}
|
||||
@@ -0,0 +1,103 @@
|
||||
package redisproxy
|
||||
|
||||
import (
|
||||
"bufio"
|
||||
"context"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/protocol/transport"
|
||||
"net"
|
||||
"strings"
|
||||
"sync"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestReadonlyPolicy(t *testing.T) {
|
||||
for _, command := range []string{"GET key", "mget a b", "HGET hash field", "SCAN 0", "XRANGE s - +", "XINFO STREAM s", "CLIENT SETINFO LIB-NAME test", "CLUSTER SLOTS", "HELLO 3", "COMMAND"} {
|
||||
if !readonlyAllowed(strings.Fields(command)) {
|
||||
t.Errorf("read rejected: %s", command)
|
||||
}
|
||||
}
|
||||
for _, command := range []string{"", "SET k v", "GETEX k EX 1", "GETDEL k", "DEL k", "FLUSHALL", "EVAL return 1 0", "EVAL_RO return 1 0", "FCALL x 0", "MULTI", "EXEC", "SORT k STORE out", "GEORADIUS k 0 0 1 km STORE out", "BITFIELD k SET i8 0 1", "XREADGROUP GROUP g c STREAMS s >", "XACK s g 1", "PFCOUNT h", "CONFIG SET dir /tmp", "CLIENT REPLY OFF", "CLIENT TRACKING ON", "CLIENT KILL ID 1", "CLUSTER FAILOVER", "MODULE LOAD x", "unknown.command", "SUBSCRIBE news", "RESET"} {
|
||||
if readonlyAllowed(strings.Fields(command)) {
|
||||
t.Errorf("unsafe command allowed: %s", command)
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
func TestReadonlyPipelineNeverForwardsWrites(t *testing.T) {
|
||||
for _, clustered := range []bool{false, true} {
|
||||
for _, authenticated := range []bool{false, true} {
|
||||
p := fixture
|
||||
p.AccessMode = "readonly"
|
||||
p.GatewayAccounts = nil
|
||||
if authenticated {
|
||||
p.GatewayAccounts = []model.GatewayAccount{{ID: "client", Username: "gateway", Password: "gateway-secret", AccessMode: "readonly"}}
|
||||
}
|
||||
if clustered {
|
||||
p.Cluster = model.ClusterConfig{Mode: "redis-cluster"}
|
||||
}
|
||||
var mu sync.Mutex
|
||||
var seen []string
|
||||
var wg sync.WaitGroup
|
||||
dial := func(context.Context, string, string) (net.Conn, error) {
|
||||
c, s := net.Pipe()
|
||||
wg.Add(1)
|
||||
go func() {
|
||||
defer wg.Done()
|
||||
defer s.Close()
|
||||
r := bufio.NewReader(s)
|
||||
for {
|
||||
a, err := request(r)
|
||||
if err != nil {
|
||||
return
|
||||
}
|
||||
mu.Lock()
|
||||
seen = append(seen, a[0])
|
||||
mu.Unlock()
|
||||
reply := "+OK\r\n"
|
||||
switch a[0] {
|
||||
case "GET":
|
||||
reply = "$5\r\nvalue\r\n"
|
||||
case "CLUSTER":
|
||||
reply = "*0\r\n"
|
||||
case "COMMAND":
|
||||
reply = "*1\r\n$3\r\nkey\r\n"
|
||||
case "INFO":
|
||||
reply = redisBulk("redis_mode:cluster\r\ncluster_enabled:1\r\nrole:master\r\n")
|
||||
case "PING":
|
||||
reply = "+PONG\r\n"
|
||||
}
|
||||
if transport.Write(s, []byte(reply)) != nil {
|
||||
return
|
||||
}
|
||||
}
|
||||
}()
|
||||
return c, nil
|
||||
}
|
||||
c, done := gateway(t, context.Background(), p, dial)
|
||||
if authenticated {
|
||||
writeCommand(t, c, "AUTH", "gateway", "gateway-secret")
|
||||
readBytes(t, c, "+OK\r\n")
|
||||
}
|
||||
var pipeline []byte
|
||||
for _, a := range [][]string{{"GET", "key"}, {"SET", "key", "bad"}, {"GET", "key"}, {"MULTI"}, {"EVAL", "return 1", "0"}, {"PING"}} {
|
||||
pipeline = append(pipeline, encode(a)...)
|
||||
}
|
||||
if err := transport.Write(c, pipeline); err != nil {
|
||||
t.Fatal(err)
|
||||
}
|
||||
denied := "-" + readonlyError + "\r\n"
|
||||
readBytes(t, c, "$5\r\nvalue\r\n"+denied+"$5\r\nvalue\r\n"+denied+denied+"+PONG\r\n")
|
||||
c.Close()
|
||||
waitDone(t, done)
|
||||
wg.Wait()
|
||||
mu.Lock()
|
||||
for _, name := range seen {
|
||||
if name == "SET" || name == "MULTI" || name == "EVAL" {
|
||||
t.Errorf("write reached upstream: %s", name)
|
||||
}
|
||||
}
|
||||
mu.Unlock()
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -279,6 +279,8 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
reader := bufio.NewReader(c)
|
||||
out := &transport.LockedWriter{Writer: c}
|
||||
var upstream net.Conn
|
||||
var upstreamReader *bufio.Reader
|
||||
var timing model.ExecutionQueue
|
||||
base := p
|
||||
defer func() {
|
||||
if upstream != nil {
|
||||
@@ -302,6 +304,9 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
}
|
||||
if gatewayAuthRequired && valid {
|
||||
selected := account.Apply(base)
|
||||
if upstream != nil && (selected.AccessMode == "readonly") != (p.AccessMode == "readonly") {
|
||||
return errors.New("Redis access mode change requires reconnect")
|
||||
}
|
||||
// An established stream cannot move servers without losing in-flight
|
||||
// replies, transactions and subscriptions. Require a new client session.
|
||||
if upstream != nil && selected.TargetID != p.TargetID {
|
||||
@@ -337,7 +342,11 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
}
|
||||
upstream.SetDeadline(time.Time{})
|
||||
raw.SetDeadline(time.Time{})
|
||||
go func() { io.Copy(out, upstream); raw.Close() }()
|
||||
if p.AccessMode == "readonly" {
|
||||
upstreamReader = bufio.NewReader(upstream)
|
||||
} else {
|
||||
go func() { relayReplies(ctx, out, upstream, &timing); raw.Close() }()
|
||||
}
|
||||
if name == "AUTH" {
|
||||
if err = transport.Write(out, []byte("+OK\r\n")); err != nil {
|
||||
return err
|
||||
@@ -345,6 +354,18 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
continue
|
||||
}
|
||||
}
|
||||
if p.AccessMode == "readonly" && !readonlyAllowed(args) {
|
||||
if event != nil {
|
||||
event(model.EventReadOnlyRejected, "redis_readonly")
|
||||
}
|
||||
if err = model.AuditCommand(ctx, strings.ToLower(name), name, "denied"); err != nil {
|
||||
return err
|
||||
}
|
||||
if err = transport.Write(out, redisError(readonlyError).Raw); err != nil {
|
||||
return err
|
||||
}
|
||||
continue
|
||||
}
|
||||
if name == "AUTH" {
|
||||
args = []string{"PING"}
|
||||
if targetAuth(p) {
|
||||
@@ -378,8 +399,30 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
return err
|
||||
}
|
||||
}
|
||||
switch name {
|
||||
case "SUBSCRIBE", "PSUBSCRIBE", "SSUBSCRIBE", "MONITOR":
|
||||
timing.Disable()
|
||||
case "CLIENT":
|
||||
if subcommand(args, "REPLY", "TRACKING") {
|
||||
timing.Disable()
|
||||
}
|
||||
}
|
||||
timing.Start(name)
|
||||
if err = transport.Write(upstream, encode(args)); err != nil {
|
||||
return err
|
||||
}
|
||||
if upstreamReader != nil {
|
||||
// Complete each reply before a local denial so pipelines stay ordered.
|
||||
stopRead := context.AfterFunc(ctx, func() { upstream.Close() })
|
||||
reply, readErr := cluster.ReadRedis(upstreamReader)
|
||||
stopRead()
|
||||
if readErr != nil {
|
||||
return readErr
|
||||
}
|
||||
if err = transport.Write(out, reply.Raw); err != nil {
|
||||
return err
|
||||
}
|
||||
timing.Finish(ctx, "redis")
|
||||
}
|
||||
}
|
||||
}
|
||||
|
||||
@@ -0,0 +1,38 @@
|
||||
package sqlserverproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"io"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/protocol/transport"
|
||||
)
|
||||
|
||||
func relayMeasured(ctx context.Context, w io.Writer, r io.Reader, q *model.ExecutionQueue) error {
|
||||
login := true
|
||||
for {
|
||||
var header [8]byte
|
||||
if _, err := io.ReadFull(r, header[:]); err != nil {
|
||||
return err
|
||||
}
|
||||
if err := transport.Write(w, header[:]); err != nil {
|
||||
return err
|
||||
}
|
||||
n := int64(binary.BigEndian.Uint16(header[2:4])) - 8
|
||||
if n < 0 {
|
||||
q.Disable()
|
||||
_, err := io.Copy(w, r)
|
||||
return err
|
||||
}
|
||||
if _, err := io.CopyN(w, r, n); err != nil {
|
||||
return err
|
||||
}
|
||||
if header[1]&1 != 0 {
|
||||
if login {
|
||||
login = false
|
||||
} else {
|
||||
q.Finish(ctx, "sqlserver")
|
||||
}
|
||||
}
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,27 @@
|
||||
package sqlserverproxy
|
||||
|
||||
import (
|
||||
"bytes"
|
||||
"context"
|
||||
"encoding/binary"
|
||||
"linkport/internal/model"
|
||||
"testing"
|
||||
)
|
||||
|
||||
func TestTDSTimingSkipsLoginAndWaitsForEndOfMessage(t *testing.T) {
|
||||
var wire []byte
|
||||
for _, flags := range []byte{1, 0, 1} {
|
||||
header := []byte{4, flags, 0, 9, 0, 0, 1, 0}
|
||||
binary.BigEndian.PutUint16(header[2:], 9)
|
||||
wire = append(wire, append(header, 'x')...)
|
||||
}
|
||||
var q model.ExecutionQueue
|
||||
q.Start("sql_batch")
|
||||
var got []model.Execution
|
||||
ctx := model.WithExecutionObserver(context.Background(), func(e model.Execution) { got = append(got, e) })
|
||||
var out bytes.Buffer
|
||||
relayMeasured(ctx, &out, bytes.NewReader(wire), &q)
|
||||
if !bytes.Equal(out.Bytes(), wire) || len(got) != 1 || got[0].Operation != "sql_batch" {
|
||||
t.Fatal("wrong response timing or modified TDS")
|
||||
}
|
||||
}
|
||||
@@ -0,0 +1,52 @@
|
||||
package sqlserverproxy
|
||||
|
||||
import (
|
||||
"context"
|
||||
"database/sql"
|
||||
mssql "github.com/microsoft/go-mssqldb"
|
||||
"github.com/microsoft/go-mssqldb/msdsn"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/protocol/transport"
|
||||
"net"
|
||||
"strconv"
|
||||
"time"
|
||||
)
|
||||
|
||||
func (h *Handler) Monitor(ctx context.Context, p model.Proxy, dial model.DialFunc) (model.DatabaseMonitoring, error) {
|
||||
out := model.DatabaseMonitoring{Metrics: []model.ServerMetric{}, Notices: []string{}}
|
||||
if err := p.CheckTargetCredential(); err != nil {
|
||||
return out, err
|
||||
}
|
||||
tls, err := transport.TLSConfig(p)
|
||||
if err != nil {
|
||||
return out, err
|
||||
}
|
||||
encryption := msdsn.Encryption(msdsn.EncryptionDisabled)
|
||||
if p.TargetTLS {
|
||||
encryption = msdsn.EncryptionRequired
|
||||
}
|
||||
c := mssql.NewConnectorConfig(msdsn.Config{Host: p.TargetHost, Port: uint64(p.TargetPort), User: p.TargetUser, Password: p.TargetPassword, Database: p.AuthSource, Encryption: encryption, TLSConfig: tls, Protocols: []string{"tcp"}, DisableRetry: true})
|
||||
c.Dialer = fixedDialer{dial, net.JoinHostPort(p.TargetHost, strconv.Itoa(p.TargetPort))}
|
||||
db := sql.OpenDB(c)
|
||||
defer db.Close()
|
||||
if err = db.PingContext(ctx); err != nil {
|
||||
return out, err
|
||||
}
|
||||
for _, query := range []string{
|
||||
"SELECT 'user_sessions', COUNT_BIG(*) FROM sys.dm_exec_sessions WHERE is_user_process = 1",
|
||||
"SELECT 'active_requests', COUNT_BIG(*) FROM sys.dm_exec_requests WHERE session_id > 50",
|
||||
"SELECT 'runnable_tasks', SUM(CONVERT(bigint,runnable_tasks_count)) FROM sys.dm_os_schedulers WHERE status = 'VISIBLE ONLINE'",
|
||||
"SELECT 'physical_memory_in_use_kb', physical_memory_in_use_kb FROM sys.dm_os_process_memory",
|
||||
} {
|
||||
var name string
|
||||
var value float64
|
||||
if err = db.QueryRowContext(ctx, query).Scan(&name, &value); err != nil {
|
||||
out.Notices = append(out.Notices, "SQL Server 部分 DMV 指标不可用，需要 VIEW SERVER STATE 或 VIEW SERVER PERFORMANCE STATE")
|
||||
continue
|
||||
}
|
||||
out.Metrics = append(out.Metrics, model.ServerMetric{Name: name, Value: value})
|
||||
}
|
||||
out.Notices = append(out.Notices, "会话、请求和调度队列表示数据库压力，不等同于宿主机 CPU 百分比；未读取 Query Store 或上游慢日志。")
|
||||
out.CheckedAt = time.Now().UTC()
|
||||
return out, nil
|
||||
}
|
||||
@@ -11,7 +11,6 @@ import (
|
||||
"errors"
|
||||
"github.com/microsoft/go-mssqldb"
|
||||
"github.com/microsoft/go-mssqldb/msdsn"
|
||||
"io"
|
||||
"linkport/internal/cluster"
|
||||
"linkport/internal/model"
|
||||
"linkport/internal/outbound"
|
||||
@@ -185,7 +184,8 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
raw.SetDeadline(time.Time{})
|
||||
up.SetDeadline(time.Time{})
|
||||
done := make(chan error, 1)
|
||||
go func() { _, e := io.Copy(down, target); raw.Close(); done <- e }()
|
||||
var timing model.ExecutionQueue
|
||||
go func() { e := relayMeasured(ctx, down, target, &timing); raw.Close(); done <- e }()
|
||||
for {
|
||||
var flags byte
|
||||
kind, payload, e := readWithFlags(down, &flags)
|
||||
@@ -215,6 +215,13 @@ func (h *Handler) Serve(ctx context.Context, raw net.Conn, p model.Proxy, dial m
|
||||
if err != nil {
|
||||
break
|
||||
}
|
||||
if kind == 1 {
|
||||
timing.Start("sql_batch")
|
||||
} else if kind == 3 {
|
||||
timing.Start("rpc")
|
||||
} else {
|
||||
timing.Disable()
|
||||
}
|
||||
if err = writeWithFlags(target, kind, payload, flags); err != nil {
|
||||
break
|
||||
}
|
||||
|
||||
+2
-2
@@ -1,4 +1,4 @@
|
||||
param([string]$Version, [switch]$SkipTests, [switch]$RebuildWeb, [string]$OutputPath = 'bin/linkport.exe')
|
||||
param([string]$Version, [switch]$SkipTests, [switch]$RebuildWeb, [string]$OutputPath = 'bin/linkport.exe', [switch]$Development)
|
||||
$ErrorActionPreference = 'Stop'
|
||||
. (Join-Path $PSScriptRoot 'build-lock.ps1')
|
||||
$buildLock = Enter-LinkPortBuildLock
|
||||
@@ -11,7 +11,7 @@ function Write-BuildPhase([string]$Name) {
|
||||
$phaseTimer.Restart()
|
||||
}
|
||||
. (Join-Path $PSScriptRoot 'version.ps1')
|
||||
$buildInfo = Get-linkportBuildInfo -Version $Version
|
||||
$buildInfo = Get-linkportBuildInfo -Version $Version -Development:$Development
|
||||
Set-Location -LiteralPath (Split-Path $PSScriptRoot -Parent)
|
||||
$env:GOTOOLCHAIN = 'go1.27.1'
|
||||
$env:GOPROXY = 'https://goproxy.cn'
|
||||
|
||||
@@ -9,7 +9,7 @@ if ($sourceMode) {
|
||||
$binary = Join-Path $projectRoot 'bin\linkport.exe'
|
||||
$candidate = Join-Path $projectRoot 'bin\dev-build\linkport.exe'
|
||||
Write-Host 'Preparing latest linkport (cached frontend and Go compilation)...'
|
||||
$buildArguments = @('-NoProfile', '-ExecutionPolicy', 'Bypass', '-File', $buildScript, '-OutputPath', $candidate)
|
||||
$buildArguments = @('-NoProfile', '-ExecutionPolicy', 'Bypass', '-File', $buildScript, '-OutputPath', $candidate, '-Development')
|
||||
if (!$FullCheck) { $buildArguments += '-SkipTests' }
|
||||
if ($RebuildWeb) { $buildArguments += '-RebuildWeb' }
|
||||
& powershell.exe @buildArguments
|
||||
|
||||
@@ -11,6 +11,14 @@ foreach ($name in @('LINKPORT_DATA', 'LINKPORT_START_TEST_DATA_PATH', 'LINKPORT_
|
||||
}
|
||||
function Assert-StartTest([bool]$Condition, [string]$Message) { if (!$Condition) { throw $Message } }
|
||||
try {
|
||||
. (Join-Path $PSScriptRoot 'version.ps1')
|
||||
$beforeBuildTime = [DateTime]::UtcNow.AddSeconds(-1)
|
||||
$developmentInfo = Get-linkportBuildInfo -Development
|
||||
$releaseInfo = Get-linkportBuildInfo
|
||||
Assert-StartTest ($developmentInfo.BuildTime -eq 'unknown') 'Development timestamp invalidates the Go executable cache'
|
||||
Assert-StartTest ($developmentInfo.Version -eq $releaseInfo.Version -and $developmentInfo.Commit -eq $releaseInfo.Commit) 'Development build lost version or commit metadata'
|
||||
$releaseTime = [DateTimeOffset]::Parse($releaseInfo.BuildTime).UtcDateTime
|
||||
Assert-StartTest ($releaseTime -ge $beforeBuildTime -and $releaseTime -le [DateTime]::UtcNow) 'Release build lost its actual build timestamp'
|
||||
New-Item -ItemType Directory -Path (Join-Path $fixture 'scripts'), (Join-Path $fixture 'data') -Force | Out-Null
|
||||
Copy-Item -LiteralPath (Join-Path $repo 'start-dev.cmd') -Destination $fixture
|
||||
Copy-Item -LiteralPath (Join-Path $PSScriptRoot 'start-dev.ps1') -Destination (Join-Path $fixture 'scripts')
|
||||
@@ -26,9 +34,9 @@ try {
|
||||
$nextLock.Dispose()
|
||||
[IO.File]::WriteAllText((Join-Path $fixture 'data/sentinel'), 'fixture data must stay unchanged')
|
||||
$fakeBuild = @'
|
||||
param([switch]$SkipTests, [switch]$RebuildWeb, [string]$OutputPath)
|
||||
param([switch]$SkipTests, [switch]$RebuildWeb, [string]$OutputPath, [switch]$Development)
|
||||
$root = Split-Path $PSScriptRoot -Parent
|
||||
@{skipTests=[bool]$SkipTests;rebuildWeb=[bool]$RebuildWeb;output=$OutputPath;workingDirectory=(Get-Location).Path} | ConvertTo-Json | Set-Content (Join-Path $root 'arguments.json')
|
||||
@{skipTests=[bool]$SkipTests;rebuildWeb=[bool]$RebuildWeb;development=[bool]$Development;output=$OutputPath;workingDirectory=(Get-Location).Path} | ConvertTo-Json | Set-Content (Join-Path $root 'arguments.json')
|
||||
if (Test-Path (Join-Path $root 'fail-build')) { exit 9 }
|
||||
New-Item -ItemType Directory -Force (Split-Path $OutputPath -Parent) | Out-Null
|
||||
if (Test-Path -LiteralPath (Join-Path $root 'fixture.exe')) {
|
||||
@@ -44,7 +52,7 @@ exit 0
|
||||
& $launcher -BuildOnly
|
||||
Assert-StartTest ($LASTEXITCODE -eq 0) 'Default build-only failed'
|
||||
$arguments = Get-Content -Raw (Join-Path $fixture 'arguments.json') | ConvertFrom-Json
|
||||
Assert-StartTest ($arguments.skipTests -and !$arguments.rebuildWeb) 'Fast flags not forwarded'
|
||||
Assert-StartTest ($arguments.skipTests -and !$arguments.rebuildWeb -and $arguments.development) 'Fast flags not forwarded'
|
||||
Assert-StartTest ($arguments.workingDirectory -eq $fixture) 'CMD did not use the entry directory'
|
||||
Assert-StartTest ((Get-Location).Path -eq $callerDirectory) 'CMD changed the caller directory'
|
||||
Assert-StartTest ($arguments.output -eq (Join-Path $fixture 'bin/dev-build/linkport.exe')) 'Build output not isolated'
|
||||
@@ -53,7 +61,7 @@ exit 0
|
||||
& $launcher -BuildOnly -FullCheck -RebuildWeb
|
||||
Assert-StartTest ($LASTEXITCODE -eq 0) 'Full build-only failed'
|
||||
$arguments = Get-Content -Raw (Join-Path $fixture 'arguments.json') | ConvertFrom-Json
|
||||
Assert-StartTest (!$arguments.skipTests -and $arguments.rebuildWeb) 'Full-check/force flags not forwarded'
|
||||
Assert-StartTest (!$arguments.skipTests -and $arguments.rebuildWeb -and $arguments.development) 'Full-check/force flags not forwarded'
|
||||
& powershell.exe -NoProfile -ExecutionPolicy Bypass -File (Join-Path $fixture 'scripts/start-dev.ps1') -BuildOnly
|
||||
Assert-StartTest ($LASTEXITCODE -eq 0) 'Direct PowerShell build-only failed'
|
||||
$arguments = Get-Content -Raw (Join-Path $fixture 'arguments.json') | ConvertFrom-Json
|
||||
|
||||
+3
-2
@@ -1,5 +1,5 @@
|
||||
function Get-linkportBuildInfo {
|
||||
param([string]$Version)
|
||||
param([string]$Version, [switch]$Development)
|
||||
$repoRoot = Split-Path $PSScriptRoot -Parent
|
||||
if (!$Version) { $Version = (Get-Content -LiteralPath (Join-Path $repoRoot 'VERSION') -Raw).Trim() }
|
||||
if ($Version -notmatch '^(0|[1-9]\d*)\.(0|[1-9]\d*)\.(0|[1-9]\d*)(?:-[0-9A-Za-z.-]+)?(?:\+[0-9A-Za-z.-]+)?$') { throw 'Invalid version' }
|
||||
@@ -13,7 +13,8 @@ function Get-linkportBuildInfo {
|
||||
}
|
||||
}
|
||||
$commit = if ($dirty) { "$revision-dirty" } else { $revision }
|
||||
$buildTime = [DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ')
|
||||
# A changing timestamp forces the linker to run even when all sources are cached.
|
||||
$buildTime = if ($Development) { 'unknown' } else { [DateTime]::UtcNow.ToString('yyyy-MM-ddTHH:mm:ssZ') }
|
||||
return [PSCustomObject]@{
|
||||
Version = $Version; Commit = $commit; Revision = $revision; Dirty = $dirty; BuildTime = $buildTime
|
||||
LDFlags = "-s -w -X linkport/internal/buildinfo.Version=$Version -X linkport/internal/buildinfo.Commit=$commit -X linkport/internal/buildinfo.BuildTime=$buildTime"
|
||||
|
||||
@@ -1,5 +1,6 @@
|
||||
export function accessModeOptions(protocol: string): { value: string; label: string }[] {
|
||||
const options = [{ value: 'readonly', label: '只读' }, { value: 'readwrite', label: '读写' }];
|
||||
const options = [{ value: 'readwrite', label: '读写' }];
|
||||
if (['mysql', 'mongodb', 'redis', 'rabbitmq'].includes(protocol)) options.unshift({ value: 'readonly', label: '只读' });
|
||||
if (protocol === 'mysql') options.push({ value: 'unrestricted', label: '全放行' });
|
||||
return options;
|
||||
}
|
||||
|
||||
@@ -10,6 +10,7 @@ export function createConnectionDraft(
|
||||
return {
|
||||
name: '', color: '', remark: '', showRemark: false, connectionOptions: { persistent: true },
|
||||
...scope,
|
||||
managementListenPort: 0, managementPort: type === 'rabbitmq' ? 15672 : 0, managementTLS: false,
|
||||
type, targetId: '', tcpForward: null as any, targetAccountId: '',
|
||||
accounts: targetAccountRequired(type) ? [{ id: accountID(), remark: '', username: '', password: '', authMethod: 'password', privateKey: '', passphrase: '' }] : [],
|
||||
group: '', outboundId: '', targetCompression: 'none', listenHost: '0.0.0.0',
|
||||
@@ -48,8 +49,8 @@ export function editConnectionDraft(config: any, target: boolean, defaults: any,
|
||||
return draft;
|
||||
}
|
||||
|
||||
const targetFields = ['name', 'project', 'environment', 'group', 'type', 'targetHost', 'targetPort', 'targetTLS', 'targetCA', 'targetCAAssetId', 'targetCompression', 'authSource', 'database', 'outboundId', 'outbound', 'ssh', 'accounts', 'cluster', 'connectionOptions', 'color', 'remark', 'showRemark'];
|
||||
const gatewayFields = ['name', 'project', 'environment', 'type', 'targetId', 'targetAccountId', 'tcpForward', 'listenHost', 'listenPort', 'allowCIDRs', 'gatewayAccounts', 'accessMode', 'maxConnections', 'ssh', 'color', 'remark', 'showRemark'];
|
||||
const targetFields = ['name', 'project', 'environment', 'group', 'type', 'targetHost', 'targetPort', 'managementPort', 'managementTLS', 'targetTLS', 'targetCA', 'targetCAAssetId', 'targetCompression', 'authSource', 'database', 'outboundId', 'outbound', 'ssh', 'accounts', 'cluster', 'connectionOptions', 'color', 'remark', 'showRemark'];
|
||||
const gatewayFields = ['name', 'project', 'environment', 'type', 'targetId', 'targetAccountId', 'tcpForward', 'listenHost', 'listenPort', 'managementListenPort', 'allowCIDRs', 'gatewayAccounts', 'accessMode', 'maxConnections', 'ssh', 'color', 'remark', 'showRemark'];
|
||||
const gatewayAccountFields = ['id', 'username', 'password', 'disabled', 'accessMode', 'allowCIDRs', 'authorizedKeys', 'authorizedKeysAssetId', 'targetId', 'targetAccountId', 'expiresAt', 'rotationReminderDays'];
|
||||
|
||||
/** Builds the existing API contract, including secret omission on updates. */
|
||||
|
||||
@@ -7,6 +7,7 @@ import { isNetworkProxy } from './networkProxy';
|
||||
import { proxyText } from './proxyLabels';
|
||||
import { singBoxValidationMessage, validateSingBoxOutbound } from './singBox';
|
||||
import { accountRemark, accountText } from './accountLabels';
|
||||
import { accessModeOptions } from './accessMode';
|
||||
export type FieldIssue = { path: string; label: string; message: string; tab: string; accountId?: string };
|
||||
type Options = { assets?: any[]; outbounds?: any[]; secretStatus?: Record<string, boolean>; editing?: boolean; targets?: any[] };
|
||||
const bytes = (v: string) => new TextEncoder().encode(v || '').length;
|
||||
@@ -69,6 +70,7 @@ export function validateTarget(draft: any, options: Options = {}): FieldIssue[]
|
||||
}
|
||||
if ((!draft.targetHost && !resourceAddress) || (draft.targetHost && (draft.targetHost.length > 253 || /[\s/\\\x00]/.test(draft.targetHost)))) add('targetHost','目标主机','请输入有效域名或 IP 地址，不包含协议前缀');
|
||||
if (!port(draft.targetPort)) add('targetPort','目标端口','端口必须为 1–65535');
|
||||
if (draft.managementPort != null && draft.managementPort !== 0 && (draft.type !== 'rabbitmq' || !port(draft.managementPort))) add('managementPort','管理目标端口','RabbitMQ 管理端口必须为 0–65535');
|
||||
if (draft.cluster?.mode && draft.cluster.mode !== 'single') {
|
||||
const seen = new Set([`${draft.targetHost}:${draft.targetPort}`]);
|
||||
for (const n of draft.cluster.nodes || []) {
|
||||
@@ -134,6 +136,7 @@ export function validateGateway(draft: any, options: Options = {}): FieldIssue[]
|
||||
if (!draft.name?.trim() || bytes(draft.name) > 100) add('name','网关名称','名称需要包含 1–100 个 UTF-8 字节','gateway');
|
||||
if (!validIP(draft.listenHost || '')) add('listenHost','监听 IP','请输入有效的 IPv4 或 IPv6 地址','gateway');
|
||||
if (!port(draft.listenPort)) add('listenPort','监听端口','端口必须为 1–65535','gateway');
|
||||
if (draft.managementListenPort != null && draft.managementListenPort !== 0 && (draft.type !== 'rabbitmq' || !port(draft.managementListenPort) || draft.managementListenPort === draft.listenPort)) add('managementListenPort','管理监听端口','管理端口必须为 1–65535 且不同于 AMQP 端口，0 表示关闭','gateway');
|
||||
if (!(Number.isInteger(draft.maxConnections) && draft.maxConnections >= 1 && draft.maxConnections <= 10000)) add('maxConnections','最大客户端连接数','连接数必须为 1–10000 的整数','gateway');
|
||||
if ((draft.allowCIDRs || []).some((v: string) => !validCIDR(v))) add('allowCIDRs','网关来源白名单','请输入有效 IP 或 CIDR，多项用逗号分隔','gateway');
|
||||
if (!draft.gatewayAccounts?.length && !['tcp', 'redis', 'mongodb'].includes(draft.type)) add('gatewayAccounts','网关账号','至少添加一个网关账号','accounts');
|
||||
@@ -149,6 +152,7 @@ export function validateGateway(draft: any, options: Options = {}): FieldIssue[]
|
||||
users.add(usernameIdentity);
|
||||
const key = ['ssh','sftp'].includes(draft.type) && (a.authorizedKeysAssetId || a.authorizedKeys?.length);
|
||||
if (!a.password && !saved && !key) add(`${prefix}.password`,'密码','请填写网关密码，SSH 也可选择客户端公钥资产','accounts');
|
||||
if (!['ssh','sftp','tcp'].includes(draft.type) && a.accessMode && !accessModeOptions(draft.type).some(mode => mode.value === a.accessMode)) add(`${prefix}.accessMode`,'账号访问模式','此协议不支持所选访问模式，请重新选择','accounts');
|
||||
if ((a.allowCIDRs || []).some((v: string) => !validCIDR(v))) add(`${prefix}.allowCIDRs`,'账号来源白名单','请输入有效 IP 或 CIDR','accounts');
|
||||
const target = a.targetId ? options.targets?.find(row => row.config.id === a.targetId)?.config : undefined;
|
||||
if (a.targetId && (!target || target.type !== draft.type)) add(`${prefix}.targetId`,'独立目标服务','请选择同协议的有效目标服务','accounts');
|
||||
@@ -239,6 +243,8 @@ export function translateAPIError(error: unknown, locale = 'zh'): string {
|
||||
[/target and gateway usernames are required/i,'请填写目标和网关用户名'],
|
||||
[/target protocol does not support unauthenticated accounts/i,'此目标协议不支持无认证账号'],
|
||||
[/gateway (account )?password is required/i,'请填写网关密码'],
|
||||
[/stop the gateway before changing RabbitMQ management listen settings/i,'请先停止网关，再修改 RabbitMQ 管理监听地址或端口'],
|
||||
[/readonly is (?:currently )?supported only for MySQL(?: and MongoDB|, MongoDB, Redis and RabbitMQ)/i,'网关只读模式仅支持 MySQL、MongoDB、Redis 和 RabbitMQ，请将当前协议的账号访问模式设为“读写”'],
|
||||
[/invalid (account )?source IP\/CIDR/i,'来源白名单包含无效的 IP 或 CIDR'],
|
||||
[/invalid or duplicate gateway username/i,'网关用户名为空、重复或格式无效'],
|
||||
[/invalid or duplicate (target|gateway) account id/i,'账号 ID 为空、重复或格式无效'],
|
||||
|
||||
@@ -14,7 +14,7 @@ export function persistentTargetSupport(draft:any, outbounds:any[] = []) {
|
||||
export function targetDraftPayload(draft:any) {
|
||||
if (draft.proxySelectionMode && !draft.outboundId) throw new Error(proxyText('proxyRequired'));
|
||||
const source = JSON.parse(JSON.stringify(draft));
|
||||
const fields = ['name','project','environment','group','type','targetHost','targetPort','targetTLS','targetCA','targetCAAssetId','targetCompression','authSource','database','outboundId','cluster','connectionOptions','color','remark','showRemark'];
|
||||
const fields = ['name','project','environment','group','type','targetHost','targetPort','managementPort','managementTLS','targetTLS','targetCA','targetCAAssetId','targetCompression','authSource','database','outboundId','cluster','connectionOptions','color','remark','showRemark'];
|
||||
const result:any = Object.fromEntries(fields.filter(key => key in source).map(key => [key, source[key]]));
|
||||
if (!source.outboundId && source.outbound?.mode === 'k8s') {
|
||||
// Send only routing metadata and an asset reference, never Kubeconfig content.
|
||||
|
||||
@@ -192,6 +192,7 @@ export default function ConnectionEditor(p: Props) {
|
||||
});
|
||||
let response: any;
|
||||
if (!target && !savedID && batch > 1) {
|
||||
if (payload.managementListenPort) throw Error("启用 RabbitMQ 管理端口时请逐个创建网关，分别配置两个监听端口");
|
||||
if (payload.listenPort + batch - 1 > 65535)
|
||||
throw Error("批量监听端口超出范围");
|
||||
const outcomes = [];
|
||||
@@ -460,12 +461,16 @@ export default function ConnectionEditor(p: Props) {
|
||||
</>)}
|
||||
{field("目标主机", "targetHost")}
|
||||
{field("目标端口", "targetPort", "number")}
|
||||
{draft.type === "rabbitmq" && <>
|
||||
{field("管理目标端口（0 使用 15672）", "managementPort", "number")}
|
||||
<Toggle label="管理接口启用 HTTPS" value={draft.managementTLS} onChange={v => set("managementTLS", v)}/>
|
||||
</>}
|
||||
{field("分组", "group")}
|
||||
{["mysql", "redis"].includes(draft.type) && field(draft.type === "redis" ? "数据库编号" : "默认数据库", "database")}
|
||||
{["mongodb", "pgsql", "sqlserver", "rabbitmq"].includes(draft.type) && field(draft.type === "rabbitmq" ? "Virtual Host" : "认证数据库", "authSource")}
|
||||
{!["ssh", "sftp", "tcp"].includes(draft.type) && (<>
|
||||
<Toggle label="启用 TLS" value={draft.targetTLS} onChange={v => set("targetTLS", v)}/>
|
||||
{draft.targetTLS && (<>
|
||||
{(draft.targetTLS || draft.type === "rabbitmq" && draft.managementTLS) && (<>
|
||||
{choice("CA 证书资产", "targetCAAssetId", assetOptions("tls_certificate"))}
|
||||
<Lines label="自定义 CA 证书（PEM）" value={draft.targetCA || ""} onChange={v => set("targetCA", v)}/>
|
||||
</>)}
|
||||
@@ -523,6 +528,10 @@ export default function ConnectionEditor(p: Props) {
|
||||
</>) : (<>
|
||||
{field("监听地址", "listenHost")}
|
||||
{field("监听端口", "listenPort", "number")}
|
||||
{draft.type === "rabbitmq" && <>
|
||||
{field("管理监听端口（0 关闭）", "managementListenPort", "number")}
|
||||
<small>AMQP 与 HTTP 管理入口共用账号和只读权限；修改管理监听设置前请停止网关。</small>
|
||||
</>}
|
||||
{field("最大连接数", "maxConnections", "number")}
|
||||
<Field label="网关来源白名单" value={(draft.allowCIDRs || []).join(", ")} onChange={v => set("allowCIDRs", v.split(/[\s,，]+/).filter(Boolean))}/>
|
||||
{!draft.gatewayAccounts.length && draft.type === "mongodb" && choice("免认证访问模式", "accessMode", accessModeOptions(draft.type))}
|
||||
@@ -630,6 +639,7 @@ export default function ConnectionEditor(p: Props) {
|
||||
{mapping(`gatewayAccounts.${accountIndex}.`)}
|
||||
<Toggle label="禁用账号" value={current.disabled} onChange={v => set(accountPath("disabled"), v)}/>
|
||||
{!["ssh", "sftp", "tcp"].includes(draft.type) && choice("账号访问模式", accountPath("accessMode"), accessModeOptions(draft.type))}
|
||||
{draft.type === "rabbitmq" && current.accessMode === "readonly" && <p>严格只读：仅检查已有队列和交换机，禁止消费、确认、发布及修改配置。</p>}
|
||||
<Field label="账号来源白名单" value={(current.allowCIDRs || []).join(", ")} onChange={v => set(accountPath("allowCIDRs"), v.split(/[\s,，]+/).filter(Boolean))}/>
|
||||
{["ssh", "sftp"].includes(draft.type) && choice("客户端公钥资产", accountPath("authorizedKeysAssetId"), assetOptions("ssh_public_key"))}
|
||||
{["ssh", "sftp"].includes(draft.type) && (<Lines label="授权公钥（每行一条）" value={(current.authorizedKeys || []).join("\n")} onChange={v => set(accountPath("authorizedKeys"), v.split("\n").filter(Boolean))}/>)}
|
||||
|
||||
@@ -0,0 +1,62 @@
|
||||
import { Alert, Button, Card, Descriptions, Switch, Table, Typography } from 'antd';
|
||||
import { useEffect, useRef, useState } from 'react';
|
||||
import { Pick, type API } from './pages/ui';
|
||||
|
||||
export default function DatabaseMonitoring({ api, targets, slowRequests }: { api: API; targets: any[]; slowRequests: any[] }) {
|
||||
const [targetID, setTargetID] = useState(''), [accountID, setAccountID] = useState('');
|
||||
const [result, setResult] = useState<any>(), [error, setError] = useState(''), [busy, setBusy] = useState(false), [auto, setAuto] = useState(false);
|
||||
const generation = useRef(0), running = useRef(false);
|
||||
const databases = targets.filter(row => ['mysql', 'pgsql', 'sqlserver', 'mongodb', 'redis', 'rabbitmq'].includes(row.config.type));
|
||||
const selected = databases.find(row => row.config.id === targetID)?.config;
|
||||
const valid = !!selected && (!selected.accounts?.length || selected.accounts.some((a: any) => a.id === accountID));
|
||||
const clear = () => { generation.current++; setResult(undefined); setError(''); };
|
||||
async function collect() {
|
||||
if (!valid || running.current) return;
|
||||
const current = generation.current;
|
||||
running.current = true; setBusy(true);
|
||||
try {
|
||||
const next = await api('monitoring/database', 'POST', { targetId: targetID, accountId: accountID, attempts: 1 });
|
||||
if (current === generation.current) { setResult(next); setError(''); }
|
||||
} catch (e) { if (current === generation.current) setError(e instanceof Error ? e.message : String(e)); }
|
||||
finally { running.current = false; setBusy(false); }
|
||||
}
|
||||
useEffect(() => { clear(); }, [targetID, accountID, selected?.id]);
|
||||
useEffect(() => () => { generation.current++; }, []);
|
||||
useEffect(() => {
|
||||
if (!auto || !valid) return;
|
||||
const timer = window.setInterval(() => { if (document.visibilityState === 'visible') void collect(); }, 15000);
|
||||
return () => window.clearInterval(timer);
|
||||
}, [auto, targetID, accountID, valid, api]);
|
||||
const names = new Map(targets.map(row => [row.config.id, row.config.name]));
|
||||
return <Card title="数据库健康与压力" className="lp-stack">
|
||||
<div className="lp-stack">
|
||||
<div className="lp-form-grid">
|
||||
<Pick label="监控数据库" value={targetID} options={[{ value: '', label: '选择数据库' }, ...databases.map(row => ({ value: row.config.id, label: `${row.config.name} · ${row.config.type}` }))]} onChange={v => { clear(); setTargetID(v); setAccountID(''); }}/>
|
||||
<Pick label="监控账号" value={accountID} options={selected?.accounts?.length ? [{ value: '', label: '选择账号' }, ...selected.accounts.map((a: any) => ({ value: a.id, label: a.remark || a.username || a.id }))] : [{ value: '', label: '无认证' }]} onChange={v => { clear(); setAccountID(v); }}/>
|
||||
</div>
|
||||
<div className="lp-page-actions">
|
||||
<Button disabled={!valid || busy} loading={busy} onClick={() => void collect()}>采集健康与压力</Button>
|
||||
<label><Switch checked={auto} onChange={setAuto} aria-label="自动采集数据库压力"/> 每 15 秒采集（仅当前页面可见时）</label>
|
||||
</div>
|
||||
{error && <Alert type="error" title={error}/>}
|
||||
{result && <>
|
||||
<Descriptions items={[
|
||||
{ key: 'health', label: '连接健康', children: result.health.healthy ? '正常' : `异常 · ${result.health.stage || '未知'}` },
|
||||
{ key: 'latency', label: '健康探测耗时', children: `${Number(result.health.latencyMs).toFixed(1)} ms` },
|
||||
{ key: 'at', label: '采集时间', children: result.diagnostics.checkedAt || result.health.checkedAt },
|
||||
]}/>
|
||||
{(result.diagnostics.notices || []).map((notice: string, i: number) => <Alert key={i} type="info" title={notice}/>)}
|
||||
<Table size="small" rowKey="name" dataSource={result.diagnostics.metrics || []} pagination={{ pageSize: 12 }} columns={[
|
||||
{ title: '服务端指标', dataIndex: 'name' }, { title: '值', dataIndex: 'value' }, { title: '单位', dataIndex: 'unit' },
|
||||
]} locale={{ emptyText: '暂无可用压力指标；请查看采集说明或目标账号权限。' }}/>
|
||||
</>}
|
||||
<Typography.Title level={4}>网关慢请求（≥ 1000 ms）</Typography.Title>
|
||||
<Typography.Paragraph type="secondary">仅记录经过 LinkPort 且已观察到完成响应的请求，保留本次进程最近 500 条；耗时包含网络、排队和响应传输。PostgreSQL 按查询周期计时，RabbitMQ 仅计量有明确回复的通道及声明请求；订阅、异步发布和无回复请求不计量。不读取上游慢日志。</Typography.Paragraph>
|
||||
<Table size="small" rowKey={(_, i) => String(i)} dataSource={[...slowRequests].reverse().filter(row => !targetID || row.targetId === targetID)} pagination={{ pageSize: 10 }} scroll={{ x: true }} columns={[
|
||||
{ title: '时间', dataIndex: 'at' }, { title: '目标', dataIndex: 'targetId', render: value => names.get(value) || value },
|
||||
{ title: '协议', dataIndex: 'protocol' }, { title: '操作', dataIndex: 'operation' },
|
||||
{ title: '耗时 ms', dataIndex: 'durationMs', render: value => Number(value).toFixed(1) },
|
||||
]}/>
|
||||
</div>
|
||||
</Card>;
|
||||
}
|
||||
@@ -2,6 +2,7 @@ import { Button, Card, Typography } from "antd";
|
||||
import { useEffect, useState } from "react";
|
||||
import { Check, DetailPage, Field, Json, Pick, Table, useAction, type API, } from "./pages/ui";
|
||||
import { integerField, requiredField, urlField, validateFields } from "./FormValidation";
|
||||
import DatabaseMonitoring from './DatabaseMonitoring';
|
||||
export default function Operations({ api, targets, scope, }: {
|
||||
api: API;
|
||||
targets: any[];
|
||||
@@ -47,6 +48,7 @@ export default function Operations({ api, targets, scope, }: {
|
||||
const selected = targets.find((r) => r.config.id === target)?.config;
|
||||
return a.form(<section className="lp-stack">
|
||||
{a.feedback}
|
||||
<DatabaseMonitoring api={api} targets={targets} slowRequests={monitor.slowRequests || []}/>
|
||||
<Card className="lp-stack">
|
||||
<div className="lp-stack">
|
||||
<div className="lp-page-actions">
|
||||
|
||||
@@ -118,6 +118,7 @@ export function RuntimeDetails({ row, targets }: {
|
||||
</Typography.Paragraph>
|
||||
<Typography.Paragraph>入口地址：{runtime.boundAddress || "尚未监听"}</Typography.Paragraph>
|
||||
<Typography.Paragraph>客户端目的地址：{runtime.clientDestinations?.join("、") || "—"}</Typography.Paragraph>
|
||||
{runtime.managementBoundAddress && <Typography.Paragraph>RabbitMQ HTTP 管理监听：{runtime.managementBoundAddress}</Typography.Paragraph>}
|
||||
<div className="lp-grid">
|
||||
{[
|
||||
["接收流量", `${Number(m.rxBytes || 0).toLocaleString()} 字节`],
|
||||
|
||||
@@ -2,11 +2,14 @@ import { test } from 'node:test';
|
||||
import assert from 'node:assert/strict';
|
||||
import { readFileSync } from 'node:fs';
|
||||
import ts from 'typescript';
|
||||
import { loadTypeScript } from './load-typescript.mjs';
|
||||
|
||||
const module = { exports: {} };
|
||||
const source = readFileSync(new URL('../src/domain/accessMode.ts', import.meta.url), 'utf8');
|
||||
new Function('module', 'exports', ts.transpileModule(source, { compilerOptions: { module: ts.ModuleKind.CommonJS } }).outputText)(module, module.exports);
|
||||
const { accessModeOptions, accessModeLabel, gatewayAccessModeLabel } = module.exports;
|
||||
const { validateGateway, translateAPIError } = loadTypeScript(new URL('../src/domain/formValidation.ts', import.meta.url));
|
||||
const { createConnectionDraft } = loadTypeScript(new URL('../src/domain/connectionDraft.ts', import.meta.url));
|
||||
|
||||
test('MySQL mode labels distinguish safe and ordinary writes; other protocols keep existing labels', () => {
|
||||
assert.equal(accessModeLabel('mysql'), '读写');
|
||||
@@ -31,3 +34,31 @@ test('mode choices expose three MySQL policies without adding unrestricted to Mo
|
||||
assert.deepEqual(accessModeOptions('mysql'), [{ value:'readonly', label:'只读' }, { value:'readwrite', label:'读写' }, { value:'unrestricted', label:'全放行' }]);
|
||||
assert.equal(accessModeOptions('mongodb').length, 2);
|
||||
});
|
||||
|
||||
test('protocols without a readonly policy offer only readwrite', () => {
|
||||
for (const protocol of ['pgsql', 'sqlserver', 'http', 'https']) {
|
||||
assert.deepEqual(accessModeOptions(protocol), [{ value: 'readwrite', label: '读写' }], protocol);
|
||||
}
|
||||
});
|
||||
|
||||
test('unsupported account modes are reported at the account field without changing permissions', () => {
|
||||
for (const protocol of ['mysql', 'mongodb', 'redis', 'pgsql', 'sqlserver', 'rabbitmq', 'http', 'https']) {
|
||||
const draft = {
|
||||
...createConnectionDraft(protocol), name: 'Access mode regression',
|
||||
gatewayAccounts: [{ id: 'client', username: 'client', password: 'fixture-password', accessMode: 'readonly' }],
|
||||
};
|
||||
const modeIssues = () => validateGateway(draft).filter(issue => issue.path.endsWith('.accessMode'));
|
||||
assert.equal(modeIssues().length, ['mysql', 'mongodb', 'redis', 'rabbitmq'].includes(protocol) ? 0 : 1, protocol);
|
||||
if (modeIssues().length) assert.equal(modeIssues()[0].path, 'gatewayAccounts.client.accessMode');
|
||||
assert.equal(draft.gatewayAccounts[0].accessMode, 'readonly');
|
||||
draft.gatewayAccounts[0].accessMode = 'readwrite';
|
||||
assert.equal(modeIssues().length, 0, protocol);
|
||||
draft.gatewayAccounts[0].accessMode = 'unrestricted';
|
||||
assert.equal(modeIssues().length, protocol === 'mysql' ? 0 : 1, protocol);
|
||||
}
|
||||
});
|
||||
|
||||
test('readonly capability errors explain the supported protocols in Chinese', () => {
|
||||
assert.match(translateAPIError(new Error('readonly is currently supported only for MySQL and MongoDB')), /只读.*MySQL.*MongoDB.*读写/);
|
||||
assert.match(translateAPIError(new Error('readonly is supported only for MySQL, MongoDB, Redis and RabbitMQ')), /Redis.*RabbitMQ/);
|
||||
});
|
||||
|
||||
@@ -7,6 +7,21 @@ const options = target => ({ target, editing: true });
|
||||
const create = type => createConnectionDraft(type, { project: 'project', environment: 'test' });
|
||||
const protocols = ['tcp','mysql','mongodb','ssh','sftp','http','https','sqlserver','redis','rabbitmq','pgsql'];
|
||||
|
||||
test('RabbitMQ management ports stay with their owning target and gateway', () => {
|
||||
const draft = create('rabbitmq');
|
||||
draft.managementPort = 15671;
|
||||
draft.managementTLS = true;
|
||||
draft.managementListenPort = 15672;
|
||||
const target = connectionSavePayload(draft, options(true));
|
||||
const gateway = connectionSavePayload(draft, options(false));
|
||||
assert.equal(target.managementPort, 15671);
|
||||
assert.equal(target.managementTLS, true);
|
||||
assert.equal(target.managementListenPort, undefined);
|
||||
assert.equal(gateway.managementListenPort, 15672);
|
||||
assert.equal(gateway.managementPort, undefined);
|
||||
assert.equal(gateway.managementTLS, undefined);
|
||||
});
|
||||
|
||||
test('account modes do not inherit gateway permissions', () => {
|
||||
const draft = editConnectionDraft({ type: 'mysql', accessMode: 'readonly', gatewayAccounts: [{id:'reader'}, {id:'writer',accessMode:'readwrite'}] }, false, create('mysql'));
|
||||
assert.equal(draft.gatewayAccounts[0].accessMode, undefined);
|
||||
|
||||
@@ -0,0 +1,26 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
test('数据库监控展示压力、网关慢请求及权限提示', async ({ page }) => {
|
||||
await page.route('**/api/v1/targets*', route => route.request().method() === 'GET'
|
||||
? route.fulfill({ json: [{ config: { id: 'monitor-db', name: 'Monitoring Redis', type: 'redis', accounts: [] } }] }) : route.continue());
|
||||
await page.route('**/api/v1/monitoring?*', route => route.fulfill({ json: {
|
||||
gateways: [], health: [], intervalSeconds: 0,
|
||||
slowRequests: [{ targetId: 'monitor-db', protocol: 'redis', operation: 'GET', durationMs: 1250, at: '2026-10-15T00:00:00Z' }],
|
||||
} }));
|
||||
await page.route('**/api/v1/monitoring/database', async route => {
|
||||
expect(route.request().postDataJSON()).toMatchObject({ targetId: 'monitor-db', accountId: '' });
|
||||
await route.fulfill({ json: { health: { healthy: true, latencyMs: 2 }, diagnostics: {
|
||||
checkedAt: '2026-10-15T00:00:00Z', metrics: [{ name: 'used_memory', value: 4096, unit: 'bytes' }], notices: ['部分指标需要额外权限'],
|
||||
} } });
|
||||
});
|
||||
await page.goto('/');
|
||||
await page.getByRole('navigation').getByRole('button', { name: '监控运维', exact: true }).click();
|
||||
const collect = page.getByRole('button', { name: '采集健康与压力', exact: true });
|
||||
await expect(collect).toBeDisabled();
|
||||
await page.getByRole('combobox', { name: '监控数据库', exact: true }).click();
|
||||
await page.getByRole('option', { name: 'Monitoring Redis · redis', exact: true }).click();
|
||||
await collect.click();
|
||||
await expect(page.getByText('used_memory', { exact: true })).toBeVisible();
|
||||
await expect(page.getByText('部分指标需要额外权限', { exact: true })).toBeVisible();
|
||||
await expect(page.getByText('1250.0', { exact: true })).toBeVisible();
|
||||
});
|
||||
@@ -0,0 +1,38 @@
|
||||
import { expect, test } from '@playwright/test';
|
||||
|
||||
for (const protocol of ['Redis', 'RabbitMQ']) test(`${protocol} 网关支持只读，并通过真实 POST 保存账号`, async ({ page }) => {
|
||||
let gatewayID = '';
|
||||
await page.goto('/');
|
||||
const session = await page.request.get('/api/v1/session');
|
||||
const headers = { 'X-CSRF-Token': (await session.json()).csrfToken };
|
||||
try {
|
||||
await page.getByRole('navigation').getByRole('button', { name: '网关服务', exact: true }).click();
|
||||
await page.getByRole('button', { name: '新建网关', exact: true }).click();
|
||||
const editor = page.getByRole('region', { name: '新建网关', exact: true });
|
||||
await editor.getByRole('combobox', { name: '协议类型', exact: true }).click();
|
||||
await page.getByRole('option', { name: protocol, exact: true }).click();
|
||||
await editor.getByLabel('网关名称', { exact: true }).fill('Redis access mode regression');
|
||||
if (protocol === 'RabbitMQ') await editor.getByLabel('管理监听端口（0 关闭）', { exact: true }).fill('15672');
|
||||
await editor.getByRole('tab', { name: '账号', exact: true }).click();
|
||||
if (protocol === 'Redis') await editor.getByRole('button', { name: '添加账号', exact: true }).click();
|
||||
await editor.getByLabel('用户名', { exact: true }).fill('redis-client');
|
||||
await editor.getByLabel('密码', { exact: true }).fill('isolated-redis-client-password');
|
||||
const mode = editor.getByRole('combobox', { name: '账号访问模式', exact: true });
|
||||
await mode.click();
|
||||
await expect(page.getByRole('option', { name: '只读', exact: true })).toHaveCount(1);
|
||||
await expect(page.getByRole('option', { name: '全放行', exact: true })).toHaveCount(0);
|
||||
await page.getByRole('option', { name: '只读', exact: true }).click();
|
||||
const saved = page.waitForResponse(response => response.url().endsWith('/api/v1/proxies') && response.request().method() === 'POST');
|
||||
await editor.getByRole('button', { name: '保存配置', exact: true }).click();
|
||||
const response = await saved;
|
||||
expect(response.status()).toBe(201);
|
||||
const stored = await response.json();
|
||||
gatewayID = stored.config.id;
|
||||
if (protocol === 'RabbitMQ') expect(stored.config.managementListenPort).toBe(15672);
|
||||
expect(response.request().postDataJSON().gatewayAccounts[0].accessMode).toBe('readonly');
|
||||
expect(stored.config.gatewayAccounts[0].accessMode).toBe('readonly');
|
||||
await expect(editor).not.toBeVisible();
|
||||
} finally {
|
||||
if (gatewayID) expect((await page.request.delete(`/api/v1/proxies/${gatewayID}`, { headers })).ok()).toBeTruthy();
|
||||
}
|
||||
});
|
||||
Reference in New Issue
Block a user