Commit Graph
858 Commits
Author SHA1 Message Date
shaw ef0ca5bdf5 style: 修正 dotStrippedEmailExpr 注释以满足 gofmt 文档注释规则 2026-07-25 19:51:19 +08:00
shaw bc3acd6e28 fix(auth): 收紧注册别名查重(根点绕过 / 误拒 / 无界扫描 / 并发竞态)
对 #4814 的审计跟进修复:

- 域名尾随点绕过:user@gmail.com. 的域名不在 gmail 家族名单内,点号折叠与
  googlemail 归一被整体跳过,别名刷号原样可复现。归一化入口统一去掉 FQDN 根点。
- 误拒合法用户:剥 "+后缀" 缺空串守卫,+alice@ 与 +bob@ 都折叠成 @domain,
  该域后续 "+x@" 注册会永久 EMAIL_EXISTS 且无自助恢复。改为仅当 "+" 不在首位时剥离。
- 无界不可索引全表扫描:原实现按 LOWER(email) LIKE '%@domain' 把整域邮箱读进内存,
  且挂在公开未鉴权的 send-verify-code 上。改为按去点邮箱
  REPLACE(LOWER(TRIM(email)), '.', '') 做等值 + "local+%@domain" 前缀探针并带 LIMIT,
  新增同表达式的部分索引(migrations/190)。TRIM 口径与既有精确匹配一致,
  历史带首尾空白的行同样命中;LIKE 元字符转义,% 与 _ 不会扩大匹配面。
- 并发竞态:注册改走 CreateWithEmailAliasGuard,在邮箱唯一性锁上追加收件箱身份锁并在
  锁内复查,避免同一收件箱的多个别名变体同时通过服务层前置查重。管理员建号仍走
  Create,不受别名限制。
- 能力断言静默 fail-open:别名查重方法上提到 UserRepository 端口(编译期强制),
  移除可选接口类型断言与静默降级分支。
- OAuth 邮箱注册的两条建号路径(同样发放注册赠额)纳入同一查重口径;邮箱换绑/绑定
  不纳入,否则用户把邮箱改成自己收件箱的别名会被误拒。
2026-07-25 19:40:53 +08:00
yianandClaude Fable 5 b6f9277515 fix(auth): normalize email aliases in registration dedup
Registration duplicate checks compared the email verbatim (lowercase +
trim only), so a single inbox could spawn unlimited accounts via provider
alias features: plus addressing (user+tag@gmail.com) and the Gmail dot
trick (u.s.e.r@gmail.com) both deliver to the same mailbox but were seen
as distinct, verifiable addresses. This lets abusers bulk-register to farm
signup grants while the domain whitelist and email verification pass.

Add NormalizeEmailForAliasDedup to collapse these variants to a single
"inbox identity" and use it (via existsByEmailOrAlias) on the three local
email-registration paths: register, send-verify-code, and its async
variant. The exact ExistsByEmail check runs first; only on a miss do we
scan the candidate domains (gmail.com/googlemail.com are mutual aliases)
and compare normalized forms. Lookup errors fail closed, matching the
existing check, so the path cannot be bypassed by inducing errors.

Scoped to registration only — email storage, display, login, and delivery
are unchanged. OAuth-bound emails (already provider-verified) are out of
scope.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-07-25 19:19:15 +08:00
shaw 1763db3a2d fix(ollama): 修复 PG<=16 上 due 判定失效并恢复抓取下限
#4850 的用量刷新调度有三个问题,本次一并修复。

jsonpath .datetime() 直到 PostgreSQL 17 才接受 ISO-8601 的 Z 标识符,
而服务写入的快照时间戳全部是 UTC(即 Z 形式)。在 PG 14/15/16 上
ollamaCloudUsageParseRFC3339SQL 因此把 fetched_at / last_attempt_at /
next_refresh_at 全部解析成 NULL,due 判定整条退化为 fail-open 分支:
每轮 20 个刷新额度被 id 最小的非 due 组占满,id 更大的组永远不会刷新——
正是该 PR 声称修复的饥饿场景。项目文档声明支持 PG 14/15/16,而集成测试
harness 固定使用 postgres:18.1,因此现有测试无法发现。

修法是在送入 jsonpath 前把结尾的 Z 改写为 +00:00。仍保留 jsonpath 而不
直接 ::timestamptz,因为通过形状正则但日历非法的值(如 2026-02-30)需要
fail-open 成 NULL 而非中断整条查询。已在真实 PG 14/15/16/17/18 上验证
五个版本行为一致,且非法日历与垃圾输入仍正确 fail-open。

成功路径不再查阅 next_refresh_at,而 nextOllamaCloudUsageDelay 的
15 分钟下限正作用于该字段,导致同组对 ollama.com 的抓取下限从 15 分钟
降到一个 runner 周期。请求间隔略大于 debounce 的交互式流量(典型 Claude
Code 用法)会把单组 24 小时抓取次数从 24-96 抬高到数百次。改为对成功路径
显式施加 fetched_at + OllamaCloudUsageMinFetchInterval 的下限,Go 与 SQL
两侧同步;空闲账号不再轮询这一主要收益不受影响。

debounce_minutes 与 interval_minutes 此前各自独立校验,因此
debounce >= interval 是合法组合;此时 min(lastUsed+debounce,
fetchedAt+maxWait) 中的 debounce 项恒为死项,管理员配置被静默忽略。
改为在写入时拒绝该组合。

另修复 refreshAccount 中 ListOllamaCloudUsageGroupAccounts 的错误被
静默吞掉(违反 CLAUDE.md 禁止忽略错误):失败时回退到更窄的活动信号会
改变 due 语义,现在记录日志。

测试:
- 恢复被删除的 7/8/9 位小数秒解析覆盖,并改写为可判别形式(断言"不应
  返回",解析失败会落入 fail-open 而被捕获)。已验证该测试在 PG 15 上
  无此修复时失败、有修复时通过。
- 新增 min fetch interval 下限与 debounce/interval 交叉校验的单元测试。
- integration harness 新增 SUB2API_TEST_POSTGRES_IMAGE 覆盖,使套件可
  针对最低受支持版本运行。
2026-07-25 18:34:37 +08:00
alfadb b403f88f51 fix(ollama): 避免刷新候选饥饿
ListDue 在 LIMIT 前用与 service 纯函数一致的 debounce/max-wait/backoff
规则筛真正 due 组,防止有活动但未到期的组占满每轮 20 名额。
2026-07-25 15:37:06 +08:00
alfadb 0f72b7dca7 feat(ollama): 按模型请求刷新云端用量
将 Ollama Cloud settings HTML 自动抓取从固定间隔轮询改为请求驱动的
trailing debounce + max-wait:无新请求不再抓取,连续请求最晚在
max-wait 强制刷新;失败退避仍优先于活动 due。新增 debounce_minutes
(默认 1),interval_minutes 保留为最长等待兼容字段。
2026-07-25 15:37:05 +08:00
shaw 5374ce2a0f Merge remote-tracking branch 'origin/main' into feature/openai-live-gateway 2026-07-25 15:16:05 +08:00
song db6fbdbf29 fix(openai): satisfy Live CI checks 2026-07-25 12:51:20 +08:00
song 988d4b577e feat(openai): add macOS Live attestation 2026-07-25 12:50:46 +08:00
song e6eb23eaac feat(openai): add Live gateway support 2026-07-25 12:50:46 +08:00
Edison42 1c0cb24c7e feat(usage): persist client session identifiers 2026-07-24 01:22:34 +08:00
shaw 2faa0891e4 fix(ollama): 审计日志不落会话明文并收紧凭证清理守卫
- PUT /admin/accounts/:id/ollama-cloud-usage/session 加入审计整体不入库
  路由,并把裸键 session 纳入键级脱敏兜底,防止浏览器会话 Cookie 明文
  留存 audit_logs.request_body
- UpdateCredentials 的 Ollama 清理分支加顶层 credentials DISTINCT 守卫,
  凭证未变化的持久化不再误清 openai 探测快照或重写 NULL extra
2026-07-23 16:43:18 +08:00
alfadb 5ac4a9fac2 feat(ollama): 支持 Cloud 官方用量自动刷新 2026-07-23 15:50:44 +08:00
Wesley Liddick fbc88edf90 Merge pull request #4750 from heathermhuang/codex/fix-simple-default-grok-image
fix(simple-mode): enable images for auto-created Grok default
2026-07-23 11:19:20 +08:00
Heatherm Huang a008b63c16 Add composite group route registry 2026-07-23 09:20:18 +08:00
Heatherm Huang c8d1e2e16f Harden composite group product surfaces 2026-07-23 09:19:25 +08:00
Heatherm Huang 33d694b89b fix(simple-mode): enable images for auto Grok default 2026-07-23 00:08:23 +08:00
Wesley Liddick ebfaf2496b Merge pull request #4674 from AdrianZhaoDev/main
feat(groups): add OpenAI reasoning policy
2026-07-22 09:37:06 +08:00
Wesley Liddick 7905d02ce5 Merge pull request #4679 from superman2003/fix/scheduler-last-used-side-key
fix(scheduler): isolate LastUsedAt cache writes
2026-07-22 09:35:42 +08:00
Wesley Liddick 134a2c0bd3 Merge pull request #4670 from wucm667/fix/issue-4669-scheduler-quota-metadata
fix: preserve quota state in scheduler metadata
2026-07-22 09:35:29 +08:00
superman2003 81e7397770 fix(scheduler): isolate LastUsedAt cache writes 2026-07-21 13:27:23 +08:00
zhaozewu 6af622c340 feat(groups): add OpenAI reasoning policy
Persist reasoning ceilings and exact mappings for OpenAI groups, enforce them across HTTP and WebSocket forwarding, and invalidate cached auth snapshots.
2026-07-21 11:02:43 +08:00
Wesley Liddick b8b72e1b18 Merge pull request #4666 from TTopoo/redis-username-support
fix(config): support Redis ACL username
2026-07-21 10:43:54 +08:00
Wesley Liddick 04a61ee6ad Merge pull request #4667 from jianjianai/new/perf-scheduler-snapshot-id-members-upstream
优化快照发布临时分配
2026-07-21 10:43:37 +08:00
wucm667 e51e61008a [verified] fix: preserve scheduler quota metadata 2026-07-21 10:41:11 +08:00
jjaw af3b6bbd68 优化快照发布临时分配 2026-07-21 03:07:01 +08:00
Jingru Shi 49200d4747 fix(config): support Redis ACL username 2026-07-21 01:31:46 +08:00
milesians de094fb63e fix: use dual-stack fallback for proxy probe 2026-07-20 22:33:08 +08:00
Wesley Liddick 9ccc9077cc Merge pull request #4581 from wucm667/feat/issue-4375-github-release-token
feat: support GitHub token for update checks
2026-07-20 10:26:15 +08:00
Wesley Liddick bfabfe60c8 Merge pull request #4593 from StarryKira/fix/image-storage-env-unreachable
fix: 异步生图开关配了却不生效(环境变量被静默丢弃 + 迁移到后台开关)
2026-07-20 09:20:15 +08:00
harukaandClaude Opus 4.8 b08cab91a9 feat(image-storage): 异步生图对象存储改为后台配置,保存即生效
此前开启异步生图必须改服务器上的 config.yaml 并重启容器(#4542),且若想
复用已配置的备份 S3,还得把同一套凭证再填一遍(#4458)。

- 新增 ImageStorageSettingService:配置存 settings 表,SecretAccessKey 经
  SecretEncryptor 加密落库、读回脱敏、留空表示沿用旧值,与备份 S3 配置同一套做法。
- reuse_backup_s3(默认开)直接借用 backup_s3_config 的端点与密钥,只用自己的
  bucket/prefix 区分对象,因此备份走 backups/、图片走 images/,且密钥不会在库里存两份。
- ImageTaskService 的启用状态改由 ImageStorageResolver 在运行时解析并缓存,
  保存设置后 Invalidate 使下次请求重建客户端——不再需要重启。
- repository 侧由提供实例改为提供工厂,客户端才可能在运行期重建。
- 轮询接口的门控从 enabled() 放宽为 Pollable():关掉开关只拒绝新提交,
  已受理的任务仍可取回结果,不再被中途吞掉。
- config.yaml 的 image_storage 保留为回落,后台从未保存过时沿用,
  升级前已用配置文件开启的部署不受影响。
- 管理端 GET/PUT/POST /admin/backups/image-storage,PUT 与备份 S3 配置一样要求
  step-up 2FA:改写存储目标同样能把生成内容导向外部账号。

注:go generate ./cmd/server 在当前 upstream 基线上即失败(securityaudit.
PromptAdminService 缺 provider),故 wire_gen.go 为手工同步。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01VHreE5pzCkSYz7J45fmd2Y
2026-07-19 00:56:21 -07:00
wucm667 510ee451bd feat: support GitHub token for update checks 2026-07-19 11:02:56 +08:00
Wesley Liddick a3f2b8fd87 Merge pull request #4540 from heathermhuang/codex/investigate-grok-oauth-test-4525
fix(grok): retry CLI chat permission denial
2026-07-18 20:50:07 +08:00
Wesley Liddick a62b821b5f Merge pull request #4478 from yardbirds0/codex/fix-upstream-billing-probe-refresh
fix: 完善上游 Sub2API 倍率探测刷新、展示与账号配置
2026-07-18 20:49:14 +08:00
Wesley Liddick 774ff5d8c8 Merge pull request #4515 from BenjaminAaron196/feat/filter-noise-rejected-requests
(fix) 过滤入口拒绝日志并强化鉴权安全边界
2026-07-18 20:46:50 +08:00
Heatherm Huang 2946281a43 fix(grok): retry CLI chat permission denial 2026-07-18 14:32:56 +08:00
benjamin 71afe24710 fix: 修复CI检查问题 2026-07-18 00:42:16 +08:00
benjamin b92bbf0299 fix: 过滤入口拒绝日志并强化鉴权边界 2026-07-18 00:11:18 +08:00
wucm667 2594950993 fix: classify transient account exhaustion as 503 instead of 404 2026-07-17 22:10:18 +08:00
Tian Lee d2585097f3 fix: 完善上游 Sub2API 计费倍率探测与账号展示 2026-07-17 20:59:29 +08:00
Heatherm Huang eaf0691752 fix(grok): preserve media eligibility in scheduler cache 2026-07-17 15:45:19 +08:00
shaw c1fd1cb44e Merge origin/main into codex/fix-grok-regressions-4412-4421
Reconcile the OAuth media route with the manual endpoint-switch redesign
(7f5d067af): media leaves for api.x.ai only when text traffic resolves to
the CLI gateway host; manually selected official/regional/custom endpoints
keep serving media as-is.
2026-07-16 19:57:48 +08:00
Wesley Liddick 7e13b6d039 Merge pull request #4425 from AdrianZhaoDev/agent/admin-users-batch-limits
feat(admin): batch update user concurrency and RPM
2026-07-16 19:33:32 +08:00
yan9651688 9fc006546c Make repeated group setup safer
Admins often recreate groups with the same pricing, routing, and account membership. A server-side duplicate creates an inactive copy for review, preserves eligible account priorities, and recovers ambiguous retries without creating extra groups.

Constraint: Group has no neutral JSON metadata field for durable operation recovery
Constraint: Model routing references account IDs, so copied configuration requires matching bindings
Rejected: Rebuild from the list response | it omits configuration and account priority details
Rejected: Store operation identity in business configuration | it would pollute real group settings
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep duplicated groups inactive until an administrator reviews the copied configuration
Tested: Go unit and full tests, go vet, integration-tag compile, frontend Vitest, lint, typecheck, production build, and Playwright duplicate flow
Not-tested: PostgreSQL container integration locally because Docker is unavailable; CI will execute the database-backed suite
2026-07-16 18:18:28 +08:00
Heatherm Huang 115116e8bf fix(grok): repair OAuth routing regressions 2026-07-16 16:52:25 +08:00
yan9651688 e2e375d694 Make repeated channel-monitor setup safer
Admins often recreate monitors with the same endpoint, model, and request settings. A server-side duplicate keeps the stored API key out of the browser, creates a disabled copy for review, and uses stable operation identity to recover ambiguous retries without creating extra rows.

Constraint: Stored monitor API keys must never be returned to the browser
Constraint: Applying a request template must preserve internal duplicate recovery metadata
Rejected: Rebuild the monitor from list data | list responses only contain a masked API key
Rejected: Copy runtime state and history | a duplicate should start as an unverified configuration
Confidence: high
Scope-risk: moderate
Reversibility: clean
Directive: Keep duplicated monitors disabled until an administrator reviews and enables them
Tested: Go unit tests for repository, service, and admin handler; integration-tag compile; go vet; golangci-lint v2.9; frontend Vitest, ESLint, typecheck, production build; Playwright duplicate flow
Not-tested: PostgreSQL container integration locally because Docker is unavailable; CI will execute the database-backed suite
2026-07-16 16:30:19 +08:00
zhaozewu 7947619cc3 feat(admin): batch update user limits 2026-07-16 15:37:35 +08:00
shaw 590efe29a5 Merge remote-tracking branch 'origin/main' into agent/fix-4326-async-image-object-storage
# Conflicts:
#	backend/cmd/server/wire_gen.go
2026-07-16 15:32:38 +08:00
shaw 2de6ccb071 fix(security): 补齐审计日志脱敏缺口 + step-up 下载/取消体验修复
审计发现修复:

高危——审计日志请求体脱敏不全(audit_logs 沦为明文凭证聚合点):
- 键名归一化比对(小写+去分隔符),覆盖 privateKey/apiv3key 等无分隔符与 camelCase 写法
- 程序化并入 SensitiveCredentialKeys 与 providerSensitiveConfigFields 两份权威敏感表,防清单漂移
- 补齐 proxy_key(内嵌代理密码)、custom_key(自设 API Key 明文)精确键
- Codex session 导入路由 body 整体由粘贴的 auth JSON 构成,键级脱敏无法覆盖,整体不入库
- 新增守卫测试:两份权威表的每个键必须被审计脱敏命中;provider_key 等渠道标识保留以便追责

中危——step-up 前端体验:
- 备份下载改同页 anchor 导航(预签名 URL 后端强制 attachment disposition),
  避免 step-up 弹窗 await 耗尽瞬态激活后 window.open 被浏览器拦截
- useStepUp.run 用户取消时抛 StepUpCancelledError sentinel,
  三个调用点静默处理,不再把取消误报为红色错误 toast

低危:
- 修正审计中间件挂载位置的陈旧注释(实际挂在认证之后)
- 审计 body 捕获改 LimitReader 按 256KB 上限截断读取,超出部分拼接回填,
  避免大体积导入请求被完整复制进内存两次
- TOTP step-up 弹窗验证成功后即时清空验证码输入
2026-07-16 14:38:16 +08:00
shaw 0ddd58aaf9 feat(security): 操作审计日志 + 会话IP/UA绑定 + 敏感操作 step-up 2FA
应对管理员访问凭证失守导致的数据外泄风险,新增三层防护:

审计日志(admin-only 可见,用户不可见)
- 新增 append-only audit_logs 表(migration 180)+ 异步批量写入 + 保留期清理
- 审计中间件挂在 admin/user/auth/admin-payment 组认证之后:记录所有变更类
  请求 + 白名单敏感读取(账号/代理导出、备份下载、admin/user API key 读取)
- 请求头凭证首尾掩码;请求体 JSON 递归脱敏(api_key/password 等擦除,base_url
  保留以便追责);非 JSON body 不入库
- 无单条删除;全量清空需现场 TOTP 校验、拒绝 admin API key、未启用 2FA 不允许,
  清空后同步写入留痕记录

会话 IP/UA 绑定(默认开启,可在系统设置关闭)
- JWT 携带 session id + IP/UA 指纹哈希;IP 或 UA 任一变化即撤销会话家族并要求
  重新登录;旧 token 无指纹时放行以平滑升级

敏感操作 step-up 2FA(sudo 窗口 15 分钟)
- 账号/代理导出、DB 备份创建/下载、S3 目标修改要求近期 TOTP 二次验证;admin API
  key 一律拒绝;前端 useStepUp 组合式 + TotpStepUpDialog 弹码后自动重试
- API key 查看按需求暂不加强管控

前端:新增 /admin/audit-logs 操作日志页面(筛选/详情/2FA 清空)、侧边栏入口、
step-up 弹窗接入导出与备份流程、安全设置项(绑定开关 + 日志保留天数)、zh/en i18n
2026-07-16 13:47:50 +08:00