Commit Graph
692 Commits
Author SHA1 Message Date
Haruka e236bff1ed fix: aggregate user model stats by requested model 2026-07-01 19:01:30 +08:00
Wesley Liddick 0a9146c3d1 Merge pull request #3586 from deqiying/codex/fix-subscription-revoke-soft-delete
修复订阅撤销操作实际上是软删除的bug
2026-07-01 15:38:45 +08:00
shaw fd004bdd88 fix(account-repo): Clone query before Count to prevent state pollution
ListWithFilters reused the Ent query builder across Count() and All(),
so interceptor-appended predicates (SoftDeleteMixin's deleted_at IS NULL)
accumulated on the shared builder — the same pattern already fixed in
group_repo/promo_code_repo/user_repo by commit 2588fa6a (P1-03 audit).

The integration test now asserts pagination.Total matches len(items) on a
single page, guarding the total-vs-items invariant reported in #3601.

Fixes #3601
2026-07-01 14:35:49 +08:00
DaydreamCodingandClaude Sonnet 5 bdf7ead157 feat(spark-shadow): OpenAI Spark 链接型影子账号
背景:gpt-5.3-codex-spark 使用独立于 codex 全局(5h/7d)的配额窗口(数据源是
/wham/usage 响应体的 codex_bengalfox,而非 codex 全局用的 x-codex-* 响应头),且
只能挂在已完成 OAuth 授权的 OpenAI 账号下复用其登录态,不能作为独立账号单独接入。
为此新增“链接型影子账号”(spark shadow account):影子账号本身不持有任何凭据,
通过 parent_account_id 指向母账号,凭据/token/代理透传自母账号并共享母账号的刷新
周期,仅在配额维度(quota_dimension=spark)和用量窗口上与母账号完全独立调度、互不
连坐。

实现:
- 数据模型:migration 154(+154a)给 accounts 表加 parent_account_id /
  quota_dimension 列 + 4 条约束(维度合法 / parent⟺非 global 维度一致 / 禁自指 /
  FK)+ 2 个 CONCURRENTLY 索引(母账号索引 + 每母账号至多一个影子的唯一索引)。
- 创建:POST /api/v1/admin/accounts/:id/shadow(CreateShadow)—— 一母一影(唯一
  索引兜底并发竞态),继承母账号 proxy/分组/并发/优先级(显式传参可覆盖),默认
  model_mapping 恒等映射到 spark(拒绝非 spark 模型),母账号必须是真实的 OpenAI
  OAuth 账号(非影子)。
- 凭据透传:resolveCredentialAccount 把影子解析回母账号,GetAccessToken / 请求头
  / WS 三条路径统一走此函数;影子自身 Credentials 恒为空(仅允许写 model_mapping),
  凭据写入的汇聚点 persistAccountCredentials 对影子早返 no-op,防止误写。
- 调度:parentHealthyForShadow 只看母账号是否仍是 OpenAI OAuth + 凭据/传输是否
  可用(active、token 未过期、未处于 401/刷新失败/传输故障导致的临时不可调度冷却),
  刻意不看母账号的 global 限流窗口——两条 429 道互不连坐。
- 用量:影子的 codex_5h/7d 走 OpenAIQuotaService.QueryUsage(/wham/usage 的
  codex_bengalfox),与母账号走的 WSv2 探测(/responses 头)完全独立的数据源、
  刷新节流与 staleness 判定。
- 备份:ExportData 显式排除影子账号(影子不持凭据,通用凭据型导入强制
  credentials 非空、无法表达父子链接),按 skipped_shadows 计数提示前端。
- 前端:账号操作菜单新增“创建 Spark 影子”入口,影子行展示回填的母账号信息
  (邮箱 / plan / 隐私模式 / 订阅到期 / chatgpt_account_id),批量操作自动跳过
  影子账号。

说明:migrations 目录用完整文件名(而非纯数字前缀)标识迁移,故本次新增的
154_account_spark_shadow.sql / 154a_..._notx.sql 与已有的
154_add_ops_system_logs_api_key_id.sql 按序号共存,与目录里 145/151 已有的
先例一致。

测试:新增约 20 个测试文件,覆盖 handler(CreateShadow 校验 / 母账号信息回填)、
repository(影子 round-trip / 一母一影唯一索引 / 迁移 schema)、service(凭据
透传三路径 / 调度母健康门 / 用量窗口来源与刷新节流 / CRS 母账号不变量 / 各类
早返与 fail-closed 场景)及前端组件(账号列表 / 操作菜单 / 用量重置)。

验证(镜像 CI;golangci-lint 首次全量分析耗时过长被跳过,其余全部实测):
- gofmt -l:干净
- go build ./... / go vet ./...:通过
- go test ./... -count=1:全绿(全部包 ok,含 internal/service、
  internal/repository、migrations)
- go test -tags integration ./internal/repository/... ./internal/service/...
  (真实 Postgres,testcontainers):全绿,含迁移幂等性
  (TestMigrationsRunner_IsIdempotent_AndSchemaIsUpToDate)与影子相关全部用例
- pnpm lint:check / pnpm typecheck / pnpm build(真实 vite 构建)/
  pnpm vitest run:全绿(124 文件 760 用例)

Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
2026-07-01 12:21:45 +08:00
deqiying 03727ac363 fix(subscription): 修复订阅撤销软删除失效
总: 增加明确的订阅撤销接口,修复撤销后的缓存失效和管理端 revoked 展示。

分: 同步失效订阅 L1 与 billing cache,补跨实例失效通知、soft-delete-aware 列表查询、revoked_at DTO 字段和回归测试。
2026-07-01 00:37:51 +08:00
PMExtra cafc95c3e2 feat: align user usage analytics with admin 2026-06-30 15:31:28 +08:00
DaydreamCodingandClaude Opus 4.8 185f9c9920 fix(auth-signup): 平台配额快照脱离注册事务 + grok 补入 CHECK 约束
自助注册(含钉钉/OAuth)报 500→404 的根因:grok 自 2026-06 进入默认平台配额
(default_platform_quotas / auth_source_*),但 user_platform_quotas 的 CHECK
约束(迁移 142)仅允许 anthropic/openai/gemini/antigravity。注册时
snapshotPlatformQuotaDefaults 写 grok 行违反约束 → 整个注册事务被 Postgres 标记
aborted → consumePendingOAuthBrowserSessionTx 撞 "transaction aborted" → 500 →
clearCookies → 用户重试拿到 404(PENDING_AUTH_SESSION_NOT_FOUND)。
影响面:所有新自助注册(不限钉钉)。

修复(两层):
- 事务隔离(fix①):snapshotPlatformQuotaDefaults 用 ent.WithoutTx 剥离调用方事务,
  在基础连接 autocommit 执行。best-effort 快照失败永不毒化注册主事务,从根上消除
  "事务内 fail-open 形同虚设"陷阱——今后任何平台/约束漂移都不会再连累注册。
- 迁移 157:把 grok 加入 user_platform_quotas.platform 的 CHECK 约束,与代码平台
  列表(domain/constants.go PlatformGrok)对齐(DROP IF EXISTS + ADD,可重入)。

新增 ent.WithoutTx(ctx) helper(手写文件,不动生成代码)。

测试:
- 单测 TestSnapshotPlatformQuotaDefaults_DetachesCallerTransaction(RED→GREEN):
  快照即便在事务 ctx 中也必须用脱离事务的 ctx 调 repo。
- 集成测试 TestUserPlatformQuotaRepository_BulkInsertInitial_GrokAllowed:
  迁移 157 后 grok 可写入(真实 postgres 容器验证)。

验证:go build ./... / go vet -tags unit ./... / 全量单测(-tags unit,45 包) /
平台配额+迁移集成测试 全绿。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 13:22:27 +08:00
Wesley Liddick e70e36e4d2 Merge pull request #3484 from bwliangc/feat/risk-control-matched-keyword
feat(risk-control): record matched keyword in keyword-block logs
2026-06-30 10:40:30 +08:00
Bestony bad87ff533 feat(ops): add api key filter to system logs 2026-06-27 14:35:19 +08:00
bwlcandClaude Opus 4.8 815bc6c9b5 feat(risk-control): record matched keyword in keyword-block logs
The risk control center's moderation log records had no field for the
keyword that triggered a keyword block, so the admin UI couldn't show
which keyword was hit (only the application slog logged it).

- migration 156: add matched_keyword column to content_moderation_logs
- ContentModerationLog gains MatchedKeyword; set it on keyword block
- repo CreateLog/ListLogs persist and read the column
- frontend: show "命中关键词" inline in the log table and detail modal
- i18n: add matchedKeyword (zh/en)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:43:05 +08:00
Heatherm Huang e7a4f3f465 test: fix grok oauth validation suite 2026-06-26 10:36:09 +08:00
Heatherm Huang b2e2c7e69c fix: harden grok oauth gateway paths 2026-06-26 10:36:09 +08:00
Heatherm Huang b3a07aeae7 fix: align grok oauth exchange with xai 2026-06-26 10:36:09 +08:00
Heatherm Huang 39be1ec97f feat: add grok subscription support 2026-06-26 10:36:09 +08:00
Wesley Liddick 1721b2b2d1 Merge pull request #3404 from wucm667/fix/balance-preflight-prevent-overdraft
fix(billing): 余额扣减增加充足性校验,防止余额透支为负
2026-06-26 09:45:46 +08:00
feitianbubu 4567f6582b test(admin/usage): update sqlmock rows for cache breakdown columns 2026-06-24 23:59:22 +08:00
feitianbubu 063454ae96 fix(admin/usage): populate cache creation/read token breakdown in stats 2026-06-24 23:43:03 +08:00
wucm667 9f5b57fc96 fix(billing): 防止余额计费持续透支 2026-06-22 10:30:27 +08:00
Wesley Liddick 2e0ff1cfd5 Merge pull request #3258 from bwliangc/feat/channel-monitor-jitter
feat(渠道监控): 检测间隔支持正负随机抖动配置
2026-06-16 16:58:23 +08:00
Wesley Liddick 16765bde69 Merge pull request #3230 from DaydreamCoding/feat/openai-cyber-policy-passthrough
feat(openai): cyber_policy 硬阻断全链路透传、审计与计费
2026-06-16 16:55:51 +08:00
shaw f069c9ae00 fix(outbox-dedup): buildSchedulerGroupPayload typed-nil broke dedup_key consistency
#3255 introduced payload-aware dedup_key (sha256 over event_type, account_id,
group_id, payload_json). enqueueSchedulerOutbox checks "if payload != nil"
before json.Marshal — but Go interface containing a typed-nil map (returned
by buildSchedulerGroupPayload(empty)) is NOT == nil at the interface level.

So an ungrouped account's account_changed event went through this path:
  payload := buildSchedulerGroupPayload(account.GroupIDs)  // typed-nil map
  enqueueSchedulerOutbox(..., payload)                     // interface != nil
    → json.Marshal(typedNilMap) = "null"
    → dedup_key hash = sha256(... + "null")

While other call sites pass literal nil:
  enqueueSchedulerOutbox(..., nil)                         // interface == nil
    → payloadJSON stays empty
    → dedup_key hash = sha256(... + "")

The two dedup_keys differ for what should be the same logical event,
silently degrading dedup effectiveness in bursts on ungrouped accounts.

Fix: change buildSchedulerGroupPayload return type from map[string]any to
any so empty input returns true untyped-nil. All call sites pass the
result straight to enqueueSchedulerOutbox(payload any) — no inspection,
no breakage.

Adds regression test TestEnqueueSchedulerOutbox_UngroupedAccountDedupesWithLiteralNilPayload
asserting (1) typed-nil regression doesn't sneak back, (2) dedup_key for
empty-groups payload matches the literal-nil-payload key.
2026-06-16 14:21:09 +08:00
shaw acaffe29ec fix(account-repo): refresh candidates SQL excluded healthy accounts; fix CI build
Post-merge audit of #3272 found two regressions:

1. ListOAuthRefreshCandidates used "AND NOT (a AND b)" which, under PG
   3-valued logic, evaluates to NULL when both temp_unschedulable_until
   and temp_unschedulable_reason are NULL — i.e., the common healthy
   account state. Such rows were silently excluded from the background
   token refresh worker, so their OAuth access tokens would never get
   refreshed and eventually start returning 401.

   Verified empirically against PostgreSQL: only 3 of 5 test rows
   matched before the fix; after switching to "(a AND b) IS NOT TRUE"
   the expected 4 rows match.

2. The new ListOAuthRefreshCandidates method on AccountRepository was
   not implemented on stubAccountRepo in api_contract_test.go (build
   tag "unit"), breaking "make test-unit" which CI runs in
   .github/workflows/backend-ci.yml.

Tests:
- Added IS NOT TRUE and "AND NOT (" assertions to the SQL-shape unit
  test so the predicate can't regress to the broken form again.
- "go test -tags=unit ./internal/..." now passes cleanly.
2026-06-16 14:08:50 +08:00
shaw 31dc8913ac chore(outbox-cleanup): add 10s grace to defend against id-vs-commit race
PG sequences advance outside transactions, so a slow committer can hold
an id that gets surpassed by the watermark before its commit becomes
visible. Without the grace period, cleanup would delete such rows before
the snapshot poller ever sees them. 10s is a comfortable upper bound
on realistic enqueueSchedulerOutbox commit latency.
2026-06-16 11:57:32 +08:00
jjaw cb14935e9a fix: cleanup consumed scheduler outbox rows 2026-06-16 11:56:40 +08:00
shaw 1fdbe52f96 chore(migrations): renumber scheduler outbox dedup migrations 151/152 -> 152/153
#3232 (already merged) occupies 151. Renumber #3255's dedup migrations to
avoid collision and keep linear ordering. Updated runner constant + tests.
2026-06-16 11:50:25 +08:00
jjaw b3ec6288ad fix: release scheduler outbox dedup on claim 2026-06-16 11:48:13 +08:00
jjaw 60cf89ae26 fix: recover scheduler outbox invalid dedup index 2026-06-16 11:48:13 +08:00
jjaw 3ef70b045d fix: safely coalesce scheduler outbox events 2026-06-16 11:48:12 +08:00
jjaw 34e66ec0a5 fix: outbox scheduler snapshot coalesce 2026-06-16 11:48:12 +08:00
jjaw 74199b6a6e fix: reduce token refresh retry amplification 2026-06-16 11:44:39 +08:00
jjaw 8b698ff4c1 fix account list parameter limit 2026-06-16 11:39:15 +08:00
Wesley Liddick 8ddc138094 Merge pull request #3232 from jianjianai/fix1/account-expiry-autopause-index
fix:account expiry autopause index / 账号过期自动禁用索引优化
2026-06-16 11:12:57 +08:00
bwlcandClaude Fable 5 c70c6a2659 feat(渠道监控): 检测间隔支持正负随机抖动配置
新增 jitter_seconds 配置:每轮调度在 interval 基础上 ± [0, jitter]
均匀随机偏移触发,避免多个监控以固定节奏同步请求上游。

- ent schema 新增 jitter_seconds 字段(默认 0),附迁移 151
- 校验:jitter >= 0 且 interval - jitter >= 15s(创建/更新均校验)
- runner 由固定 ticker 改为每轮重新随机化的 timer,0 抖动时行为不变
- 前端监控表单新增「随机抖动 (± 秒)」输入框,上限随间隔联动

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 22:09:53 +08:00
wucm667 c1c28ac7bb fix(gateway): 解压 zstd 上游响应体 2026-06-12 15:06:01 +08:00
DaydreamCodingandClaude Opus 4.8 b62b573f7f feat(openai): cyber_policy 硬阻断全链路透传、审计与计费
上游对单次请求下发 error.code=cyber_policy 硬阻断时,网关在所有端点
(/v1/responses、/v1/chat/completions、/v1/messages、WebSocket)及流式/
非流式路径下,将该结果原样透传给客户端,绝不 failover、换号或同步拦截;
命中后异步完成审计与计费:

- 风控中心记录 cyber_policy 留痕并发送通知邮件,落库先于发信,SMTP 阻塞
  不影响留痕
- ops 错误请求记录,状态码对齐客户端实际接收(流式 200 / 非流式 400)
- 用量明细标记 request_type=cyber,按上游真实 token 计费,HTTP 与
  WebSocket 计费口径统一,零 token 命中不误扣
- 会话级自动屏蔽(管理员开关,默认关):命中的会话在可配 TTL 内本地拦截
  不再发往上游,仅屏蔽该会话不影响同 Key 其他会话
- 封号计数排除开关:可选让 cyber 命中不计入自动封号,命中当次不判定且
  历史行在违规计数中一并排除

WebSocket 多轮连接下 cyber 标记按 turn 生命周期管理,逐轮独立检测与记录;
透传的错误响应不被兜底逻辑追加内容污染。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 01:47:01 +08:00
jjaw e4c255a77a fix:account expiry autopause index 2026-06-12 01:38:09 +08:00
DaydreamCodingandClaude Opus 4.8 329414ea4f feat(admin): /admin/users 新增按用户 API Key 所在分组过滤
- 支持专用/公开/订阅/已禁用四类分组:按 api_keys.group_id 精确匹配,
  排除软删除 key(EXISTS + DeletedAtIsNil);已禁用分组单独成区,
  覆盖 key 仍挂在禁用分组上的用户
- 后端:UserListFilters.APIKeyGroupID;handler 解析 api_key_group_id;
  repo HasAPIKeysWith 谓词;GetAllGroupsIncludingInactive 新接口
  (/admin/groups/all?include_inactive=true)
- 前端:下拉按类型分区单选;UsersView 独立 allGroupsForApiKeyFilter
  loader;分区标题用负数哨兵值修复 Vue :key 重复问题
- 测试:repo 集成 5/5(含软删除排除、多 key 去重、叠加 status 过滤)、
  handler 单测 5/5、前端 vitest 6/6

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 22:54:01 +08:00
Wesley Liddick 870cefddf5 Merge pull request #3097 from NeckBozia/fix/5h-resets-at-stale
fix(usage): sync 5h ResetsAt to SessionWindowEnd and zero expired window
2026-06-08 09:08:23 +08:00
Wesley Liddick a2bf6ad77a Merge pull request #3107 from DaydreamCoding/feat/proxy-failover-resilience
feat: 代理生命周期与故障健壮性增强
2026-06-08 08:59:15 +08:00
DaydreamCoding af19d44327 feat(proxies): 代理有效期与失败回退
- schema/迁移: 代理有效期、提醒天数、失败回退配置 + 账号 fallback 来源字段
- service/repo/DTO/handler: CRUD 透传新字段 + 校验
- fallback 目标解析纯函数(链式解析 + 环检测 + 兜底)
- SweepExpiredProxies 到期改投账号 + outbox 失效
- ProxyExpiryService 后台到期扫描任务 + wire 注册
- 账号侧手动回切原代理 + fallback 来源徽章/按钮
- 前端: 创建/编辑表单、列表到期徽章、类型/API/i18n
- ops 告警: proxy_expired_count / proxy_expiring_soon_count 指标
- 导入导出携带有效期/回退字段(备用按 name 映射)
- 补全测试 stub + 集成测试 + review 问题修复
2026-06-08 00:01:30 +08:00
CoolCoolTomato 1a86c6ce12 fix: enforce exclusive group access for api keys 2026-06-07 20:46:03 +08:00
NeckBozia 16bc876935 fix(usage): sync 5h ResetsAt to SessionWindowEnd and zero expired window
active poll 拿到新 5h ResetsAt 时只回写了 Extra.session_window_utilization,
没回写 SessionWindowEnd column;estimateSetupTokenUsage 读这个 column 作为 5h
窗口结束时间,导致被动采样模式下 dashboard 显示 utilization > 0 但 reset
时间渲染为「现在」。

- syncActiveToPassive 增加 UpdateSessionWindowEnd 回写
- estimateSetupTokenUsage 抄齐 Codex 分支的过期归零 guard,避免 active
  poll 没回写时 UI 渲染矛盾
- UsageProgressBar 区分「待刷新」/「现在」语义;i18n key 中英补齐
2026-06-07 15:23:45 +08:00
feitianbubu 029b6d61a2 feat(usage): 聚合统计拆分缓存创建与命中 token 2026-06-06 22:32:37 +08:00
Wesley Liddick 56a6f53848 Merge pull request #2999 from Pluviobyte/codex/sub2api-scheduler-snapshot-sync
fix: sync scheduler snapshots after account state updates
2026-06-06 14:27:31 +08:00
Wesley Liddick 5824359e97 Merge pull request #3062 from visa2/fix/multi-instance-bg-job-leader-lock
fix(service): gate periodic background jobs with a leader lock for multi-instance
2026-06-06 13:59:42 +08:00
Wesley Liddick b76f9524ba Merge pull request #3040 from bwliangc/feat/ops-ttft-sample-weighting
fix(ops): 运维监控首 token 延迟(TTFT)按流式样本数加权
2026-06-06 13:48:51 +08:00
Wesley Liddick 6f18f74030 Merge pull request #2970 from ghostg00/compat/redis-replicate-commands
compat(redis): make TIME-using Lua scripts work on Redis 3.2-4.x
2026-06-06 11:23:54 +08:00
shaw bba86f97d8 fix(repo): make userRepo.Delete reuse caller's transaction
userRepository.Delete used the r.client.Tx(ctx)+ErrTxStarted pattern,
which only reuses a transaction when r.client is itself a tx-bound
client. With the singleton base client (how Wire injects repos),
r.client.Tx() always starts a NEW independent transaction and ignores
any tx carried in the context.

As a result, AdminService.DeleteUser's transaction (added in #3047)
only covered the API-key deletions; the user soft-delete escaped into a
separate, immediately-committed transaction. If the outer commit failed
after the user tx had committed, the user was deleted while its API keys
stayed active, reintroducing the unrecoverable state of #3021.

Add a leading dbent.TxFromContext check (mirroring DeleteWithAudit) so
Delete joins an externally-provided transaction and lets the caller
commit/rollback. Extract the deletion body into deleteUser(ctx, exec,
id). All three original paths are preserved; callers passing a plain ctx
are unaffected.

Add user_repo_delete_atomicity_integration_test.go: builds repos with
the base client and a manual outer tx, asserting that delete-keys and
delete-user roll back together on abort and commit together on success.
2026-06-06 10:28:28 +08:00
visa2andClaude Opus 4.8 362f9e77bf fix(service): gate periodic background jobs with a leader lock for multi-instance
Three periodic background jobs ran on every instance with no cross-instance
coordination, multiplying their cost (and side effects) by the replica count:

- DashboardAggregationService.runScheduledAggregation: N× heavy GROUP BY
  aggregation queries every minute plus watermark write races.
- PaymentOrderExpiryService.runOnce: N× upstream payment-provider reconcile/
  expiry API calls per pending order.
- SubscriptionExpiryService.sendExpiryReminders: N× full active-subscription
  scans every minute and potential duplicate reminder emails.

Add a LeaderLockCache abstraction so only one instance runs each job per cycle:

- The interface lives in the service layer; the Redis-backed implementation
  (SetNX + compare-and-delete release) lives in the repository layer, so the
  service package keeps its depguard "must not import redis" boundary intact.
- tryAcquireSingletonLeaderLock prefers the cache and falls back to a Postgres
  advisory lock when Redis errors, mirroring the Ops background services. When
  neither backend is configured the job runs ungated, preserving single-instance
  and test behavior (no self-lockout: the lock is released every cycle).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 22:47:24 +08:00
方程 705fe7d880 fix(admin): delete user api keys with user 2026-06-05 16:23:56 +08:00