Commit Graph
687 Commits
Author SHA1 Message Date
PMExtra cafc95c3e2 feat: align user usage analytics with admin 2026-06-30 15:31:28 +08:00
DaydreamCodingandClaude Opus 4.8 185f9c9920 fix(auth-signup): 平台配额快照脱离注册事务 + grok 补入 CHECK 约束
自助注册(含钉钉/OAuth)报 500→404 的根因:grok 自 2026-06 进入默认平台配额
(default_platform_quotas / auth_source_*),但 user_platform_quotas 的 CHECK
约束(迁移 142)仅允许 anthropic/openai/gemini/antigravity。注册时
snapshotPlatformQuotaDefaults 写 grok 行违反约束 → 整个注册事务被 Postgres 标记
aborted → consumePendingOAuthBrowserSessionTx 撞 "transaction aborted" → 500 →
clearCookies → 用户重试拿到 404(PENDING_AUTH_SESSION_NOT_FOUND)。
影响面:所有新自助注册(不限钉钉)。

修复(两层):
- 事务隔离(fix①):snapshotPlatformQuotaDefaults 用 ent.WithoutTx 剥离调用方事务,
  在基础连接 autocommit 执行。best-effort 快照失败永不毒化注册主事务,从根上消除
  "事务内 fail-open 形同虚设"陷阱——今后任何平台/约束漂移都不会再连累注册。
- 迁移 157:把 grok 加入 user_platform_quotas.platform 的 CHECK 约束,与代码平台
  列表(domain/constants.go PlatformGrok)对齐(DROP IF EXISTS + ADD,可重入)。

新增 ent.WithoutTx(ctx) helper(手写文件,不动生成代码)。

测试:
- 单测 TestSnapshotPlatformQuotaDefaults_DetachesCallerTransaction(RED→GREEN):
  快照即便在事务 ctx 中也必须用脱离事务的 ctx 调 repo。
- 集成测试 TestUserPlatformQuotaRepository_BulkInsertInitial_GrokAllowed:
  迁移 157 后 grok 可写入(真实 postgres 容器验证)。

验证:go build ./... / go vet -tags unit ./... / 全量单测(-tags unit,45 包) /
平台配额+迁移集成测试 全绿。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-30 13:22:27 +08:00
Wesley Liddick e70e36e4d2 Merge pull request #3484 from bwliangc/feat/risk-control-matched-keyword
feat(risk-control): record matched keyword in keyword-block logs
2026-06-30 10:40:30 +08:00
Bestony bad87ff533 feat(ops): add api key filter to system logs 2026-06-27 14:35:19 +08:00
bwlcandClaude Opus 4.8 815bc6c9b5 feat(risk-control): record matched keyword in keyword-block logs
The risk control center's moderation log records had no field for the
keyword that triggered a keyword block, so the admin UI couldn't show
which keyword was hit (only the application slog logged it).

- migration 156: add matched_keyword column to content_moderation_logs
- ContentModerationLog gains MatchedKeyword; set it on keyword block
- repo CreateLog/ListLogs persist and read the column
- frontend: show "命中关键词" inline in the log table and detail modal
- i18n: add matchedKeyword (zh/en)

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-26 10:43:05 +08:00
Heatherm Huang e7a4f3f465 test: fix grok oauth validation suite 2026-06-26 10:36:09 +08:00
Heatherm Huang b2e2c7e69c fix: harden grok oauth gateway paths 2026-06-26 10:36:09 +08:00
Heatherm Huang b3a07aeae7 fix: align grok oauth exchange with xai 2026-06-26 10:36:09 +08:00
Heatherm Huang 39be1ec97f feat: add grok subscription support 2026-06-26 10:36:09 +08:00
Wesley Liddick 1721b2b2d1 Merge pull request #3404 from wucm667/fix/balance-preflight-prevent-overdraft
fix(billing): 余额扣减增加充足性校验,防止余额透支为负
2026-06-26 09:45:46 +08:00
feitianbubu 4567f6582b test(admin/usage): update sqlmock rows for cache breakdown columns 2026-06-24 23:59:22 +08:00
feitianbubu 063454ae96 fix(admin/usage): populate cache creation/read token breakdown in stats 2026-06-24 23:43:03 +08:00
wucm667 9f5b57fc96 fix(billing): 防止余额计费持续透支 2026-06-22 10:30:27 +08:00
Wesley Liddick 2e0ff1cfd5 Merge pull request #3258 from bwliangc/feat/channel-monitor-jitter
feat(渠道监控): 检测间隔支持正负随机抖动配置
2026-06-16 16:58:23 +08:00
Wesley Liddick 16765bde69 Merge pull request #3230 from DaydreamCoding/feat/openai-cyber-policy-passthrough
feat(openai): cyber_policy 硬阻断全链路透传、审计与计费
2026-06-16 16:55:51 +08:00
shaw f069c9ae00 fix(outbox-dedup): buildSchedulerGroupPayload typed-nil broke dedup_key consistency
#3255 introduced payload-aware dedup_key (sha256 over event_type, account_id,
group_id, payload_json). enqueueSchedulerOutbox checks "if payload != nil"
before json.Marshal — but Go interface containing a typed-nil map (returned
by buildSchedulerGroupPayload(empty)) is NOT == nil at the interface level.

So an ungrouped account's account_changed event went through this path:
  payload := buildSchedulerGroupPayload(account.GroupIDs)  // typed-nil map
  enqueueSchedulerOutbox(..., payload)                     // interface != nil
    → json.Marshal(typedNilMap) = "null"
    → dedup_key hash = sha256(... + "null")

While other call sites pass literal nil:
  enqueueSchedulerOutbox(..., nil)                         // interface == nil
    → payloadJSON stays empty
    → dedup_key hash = sha256(... + "")

The two dedup_keys differ for what should be the same logical event,
silently degrading dedup effectiveness in bursts on ungrouped accounts.

Fix: change buildSchedulerGroupPayload return type from map[string]any to
any so empty input returns true untyped-nil. All call sites pass the
result straight to enqueueSchedulerOutbox(payload any) — no inspection,
no breakage.

Adds regression test TestEnqueueSchedulerOutbox_UngroupedAccountDedupesWithLiteralNilPayload
asserting (1) typed-nil regression doesn't sneak back, (2) dedup_key for
empty-groups payload matches the literal-nil-payload key.
2026-06-16 14:21:09 +08:00
shaw acaffe29ec fix(account-repo): refresh candidates SQL excluded healthy accounts; fix CI build
Post-merge audit of #3272 found two regressions:

1. ListOAuthRefreshCandidates used "AND NOT (a AND b)" which, under PG
   3-valued logic, evaluates to NULL when both temp_unschedulable_until
   and temp_unschedulable_reason are NULL — i.e., the common healthy
   account state. Such rows were silently excluded from the background
   token refresh worker, so their OAuth access tokens would never get
   refreshed and eventually start returning 401.

   Verified empirically against PostgreSQL: only 3 of 5 test rows
   matched before the fix; after switching to "(a AND b) IS NOT TRUE"
   the expected 4 rows match.

2. The new ListOAuthRefreshCandidates method on AccountRepository was
   not implemented on stubAccountRepo in api_contract_test.go (build
   tag "unit"), breaking "make test-unit" which CI runs in
   .github/workflows/backend-ci.yml.

Tests:
- Added IS NOT TRUE and "AND NOT (" assertions to the SQL-shape unit
  test so the predicate can't regress to the broken form again.
- "go test -tags=unit ./internal/..." now passes cleanly.
2026-06-16 14:08:50 +08:00
shaw 31dc8913ac chore(outbox-cleanup): add 10s grace to defend against id-vs-commit race
PG sequences advance outside transactions, so a slow committer can hold
an id that gets surpassed by the watermark before its commit becomes
visible. Without the grace period, cleanup would delete such rows before
the snapshot poller ever sees them. 10s is a comfortable upper bound
on realistic enqueueSchedulerOutbox commit latency.
2026-06-16 11:57:32 +08:00
jjaw cb14935e9a fix: cleanup consumed scheduler outbox rows 2026-06-16 11:56:40 +08:00
shaw 1fdbe52f96 chore(migrations): renumber scheduler outbox dedup migrations 151/152 -> 152/153
#3232 (already merged) occupies 151. Renumber #3255's dedup migrations to
avoid collision and keep linear ordering. Updated runner constant + tests.
2026-06-16 11:50:25 +08:00
jjaw b3ec6288ad fix: release scheduler outbox dedup on claim 2026-06-16 11:48:13 +08:00
jjaw 60cf89ae26 fix: recover scheduler outbox invalid dedup index 2026-06-16 11:48:13 +08:00
jjaw 3ef70b045d fix: safely coalesce scheduler outbox events 2026-06-16 11:48:12 +08:00
jjaw 34e66ec0a5 fix: outbox scheduler snapshot coalesce 2026-06-16 11:48:12 +08:00
jjaw 74199b6a6e fix: reduce token refresh retry amplification 2026-06-16 11:44:39 +08:00
jjaw 8b698ff4c1 fix account list parameter limit 2026-06-16 11:39:15 +08:00
Wesley Liddick 8ddc138094 Merge pull request #3232 from jianjianai/fix1/account-expiry-autopause-index
fix:account expiry autopause index / 账号过期自动禁用索引优化
2026-06-16 11:12:57 +08:00
bwlcandClaude Fable 5 c70c6a2659 feat(渠道监控): 检测间隔支持正负随机抖动配置
新增 jitter_seconds 配置:每轮调度在 interval 基础上 ± [0, jitter]
均匀随机偏移触发,避免多个监控以固定节奏同步请求上游。

- ent schema 新增 jitter_seconds 字段(默认 0),附迁移 151
- 校验:jitter >= 0 且 interval - jitter >= 15s(创建/更新均校验)
- runner 由固定 ticker 改为每轮重新随机化的 timer,0 抖动时行为不变
- 前端监控表单新增「随机抖动 (± 秒)」输入框,上限随间隔联动

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
2026-06-12 22:09:53 +08:00
wucm667 c1c28ac7bb fix(gateway): 解压 zstd 上游响应体 2026-06-12 15:06:01 +08:00
DaydreamCodingandClaude Opus 4.8 b62b573f7f feat(openai): cyber_policy 硬阻断全链路透传、审计与计费
上游对单次请求下发 error.code=cyber_policy 硬阻断时,网关在所有端点
(/v1/responses、/v1/chat/completions、/v1/messages、WebSocket)及流式/
非流式路径下,将该结果原样透传给客户端,绝不 failover、换号或同步拦截;
命中后异步完成审计与计费:

- 风控中心记录 cyber_policy 留痕并发送通知邮件,落库先于发信,SMTP 阻塞
  不影响留痕
- ops 错误请求记录,状态码对齐客户端实际接收(流式 200 / 非流式 400)
- 用量明细标记 request_type=cyber,按上游真实 token 计费,HTTP 与
  WebSocket 计费口径统一,零 token 命中不误扣
- 会话级自动屏蔽(管理员开关,默认关):命中的会话在可配 TTL 内本地拦截
  不再发往上游,仅屏蔽该会话不影响同 Key 其他会话
- 封号计数排除开关:可选让 cyber 命中不计入自动封号,命中当次不判定且
  历史行在违规计数中一并排除

WebSocket 多轮连接下 cyber 标记按 turn 生命周期管理,逐轮独立检测与记录;
透传的错误响应不被兜底逻辑追加内容污染。

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-12 01:47:01 +08:00
jjaw e4c255a77a fix:account expiry autopause index 2026-06-12 01:38:09 +08:00
DaydreamCodingandClaude Opus 4.8 329414ea4f feat(admin): /admin/users 新增按用户 API Key 所在分组过滤
- 支持专用/公开/订阅/已禁用四类分组:按 api_keys.group_id 精确匹配,
  排除软删除 key(EXISTS + DeletedAtIsNil);已禁用分组单独成区,
  覆盖 key 仍挂在禁用分组上的用户
- 后端:UserListFilters.APIKeyGroupID;handler 解析 api_key_group_id;
  repo HasAPIKeysWith 谓词;GetAllGroupsIncludingInactive 新接口
  (/admin/groups/all?include_inactive=true)
- 前端:下拉按类型分区单选;UsersView 独立 allGroupsForApiKeyFilter
  loader;分区标题用负数哨兵值修复 Vue :key 重复问题
- 测试:repo 集成 5/5(含软删除排除、多 key 去重、叠加 status 过滤)、
  handler 单测 5/5、前端 vitest 6/6

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-08 22:54:01 +08:00
Wesley Liddick 870cefddf5 Merge pull request #3097 from NeckBozia/fix/5h-resets-at-stale
fix(usage): sync 5h ResetsAt to SessionWindowEnd and zero expired window
2026-06-08 09:08:23 +08:00
Wesley Liddick a2bf6ad77a Merge pull request #3107 from DaydreamCoding/feat/proxy-failover-resilience
feat: 代理生命周期与故障健壮性增强
2026-06-08 08:59:15 +08:00
DaydreamCoding af19d44327 feat(proxies): 代理有效期与失败回退
- schema/迁移: 代理有效期、提醒天数、失败回退配置 + 账号 fallback 来源字段
- service/repo/DTO/handler: CRUD 透传新字段 + 校验
- fallback 目标解析纯函数(链式解析 + 环检测 + 兜底)
- SweepExpiredProxies 到期改投账号 + outbox 失效
- ProxyExpiryService 后台到期扫描任务 + wire 注册
- 账号侧手动回切原代理 + fallback 来源徽章/按钮
- 前端: 创建/编辑表单、列表到期徽章、类型/API/i18n
- ops 告警: proxy_expired_count / proxy_expiring_soon_count 指标
- 导入导出携带有效期/回退字段(备用按 name 映射)
- 补全测试 stub + 集成测试 + review 问题修复
2026-06-08 00:01:30 +08:00
CoolCoolTomato 1a86c6ce12 fix: enforce exclusive group access for api keys 2026-06-07 20:46:03 +08:00
NeckBozia 16bc876935 fix(usage): sync 5h ResetsAt to SessionWindowEnd and zero expired window
active poll 拿到新 5h ResetsAt 时只回写了 Extra.session_window_utilization,
没回写 SessionWindowEnd column;estimateSetupTokenUsage 读这个 column 作为 5h
窗口结束时间,导致被动采样模式下 dashboard 显示 utilization > 0 但 reset
时间渲染为「现在」。

- syncActiveToPassive 增加 UpdateSessionWindowEnd 回写
- estimateSetupTokenUsage 抄齐 Codex 分支的过期归零 guard,避免 active
  poll 没回写时 UI 渲染矛盾
- UsageProgressBar 区分「待刷新」/「现在」语义;i18n key 中英补齐
2026-06-07 15:23:45 +08:00
feitianbubu 029b6d61a2 feat(usage): 聚合统计拆分缓存创建与命中 token 2026-06-06 22:32:37 +08:00
Wesley Liddick 56a6f53848 Merge pull request #2999 from Pluviobyte/codex/sub2api-scheduler-snapshot-sync
fix: sync scheduler snapshots after account state updates
2026-06-06 14:27:31 +08:00
Wesley Liddick 5824359e97 Merge pull request #3062 from visa2/fix/multi-instance-bg-job-leader-lock
fix(service): gate periodic background jobs with a leader lock for multi-instance
2026-06-06 13:59:42 +08:00
Wesley Liddick b76f9524ba Merge pull request #3040 from bwliangc/feat/ops-ttft-sample-weighting
fix(ops): 运维监控首 token 延迟(TTFT)按流式样本数加权
2026-06-06 13:48:51 +08:00
Wesley Liddick 6f18f74030 Merge pull request #2970 from ghostg00/compat/redis-replicate-commands
compat(redis): make TIME-using Lua scripts work on Redis 3.2-4.x
2026-06-06 11:23:54 +08:00
shaw bba86f97d8 fix(repo): make userRepo.Delete reuse caller's transaction
userRepository.Delete used the r.client.Tx(ctx)+ErrTxStarted pattern,
which only reuses a transaction when r.client is itself a tx-bound
client. With the singleton base client (how Wire injects repos),
r.client.Tx() always starts a NEW independent transaction and ignores
any tx carried in the context.

As a result, AdminService.DeleteUser's transaction (added in #3047)
only covered the API-key deletions; the user soft-delete escaped into a
separate, immediately-committed transaction. If the outer commit failed
after the user tx had committed, the user was deleted while its API keys
stayed active, reintroducing the unrecoverable state of #3021.

Add a leading dbent.TxFromContext check (mirroring DeleteWithAudit) so
Delete joins an externally-provided transaction and lets the caller
commit/rollback. Extract the deletion body into deleteUser(ctx, exec,
id). All three original paths are preserved; callers passing a plain ctx
are unaffected.

Add user_repo_delete_atomicity_integration_test.go: builds repos with
the base client and a manual outer tx, asserting that delete-keys and
delete-user roll back together on abort and commit together on success.
2026-06-06 10:28:28 +08:00
visa2andClaude Opus 4.8 362f9e77bf fix(service): gate periodic background jobs with a leader lock for multi-instance
Three periodic background jobs ran on every instance with no cross-instance
coordination, multiplying their cost (and side effects) by the replica count:

- DashboardAggregationService.runScheduledAggregation: N× heavy GROUP BY
  aggregation queries every minute plus watermark write races.
- PaymentOrderExpiryService.runOnce: N× upstream payment-provider reconcile/
  expiry API calls per pending order.
- SubscriptionExpiryService.sendExpiryReminders: N× full active-subscription
  scans every minute and potential duplicate reminder emails.

Add a LeaderLockCache abstraction so only one instance runs each job per cycle:

- The interface lives in the service layer; the Redis-backed implementation
  (SetNX + compare-and-delete release) lives in the repository layer, so the
  service package keeps its depguard "must not import redis" boundary intact.
- tryAcquireSingletonLeaderLock prefers the cache and falls back to a Postgres
  advisory lock when Redis errors, mirroring the Ops background services. When
  neither backend is configured the job runs ungated, preserving single-instance
  and test behavior (no self-lockout: the lock is released every cycle).

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-06-05 22:47:24 +08:00
方程 705fe7d880 fix(admin): delete user api keys with user 2026-06-05 16:23:56 +08:00
DaydreamCodingandClaude Opus 4.8 cfb195c7b2 feat(usage): 记录并展示失败请求(用户端+管理端)
- 记录失败请求并在用户端/管理端展示;分类下拉改用统一 Select 组件
- 模型过滤改后端 ILIKE 模糊匹配;新增「Key 名称」列(含已删除标记)与按 Key 过滤;时间列移至末列
- 用户可见「已删除 key 失败请求」:OpsErrorLogFilter 加 MatchDeletedKeyOwner,用户侧归属
  放宽为 (user_id OR deleted_key_owner_user_id),让 key 原所有者能看到删除 key 后继续请求
  导致的认证失败记录(他人仍 NotFound,不泄露存在性)
- 迁移 148:ops_error_logs 用户+时间索引

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 14:00:57 +08:00
DaydreamCodingandClaude Opus 4.8 ddf063352a feat(ops): 错误日志 key 归因与早退字段补全
让 /admin/ops 错误详情正确归因 API key 并补全早退场景字段,合并三项改动:

- 鉴权早退补全用户/分组/平台字段:引入 ops fallback key(ContextKeyOpsFallbackAPIKey),
  apiKey 一加载成功即写入,覆盖分组停用/删除、Key 停用/过期/额度、用户停用、IP 限制等早退
  路径;ops 错误日志改用 getOpsAPIKey(正式 key 优先、回退键兜底),不改「已鉴权」语义。
- 已删除 key 归因(迁移 145):删除 key 时同一事务写 deleted_api_key_audits 映射,认证失败
  时用明文反查命中原所有者,错误详情展示「已删除 Key 所有者」「尝试的 Key 前缀」。
- 有效 key 报错快照前缀(迁移 147):对绑定有效 key 的错误,落库时快照明文前 8 位到
  api_key_prefix(与 attempted_key_prefix 互斥),key 之后被删仍保留报错当时真实前缀。

均仅对上线后新产生的错误/删除生效。

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 14:00:57 +08:00
Wesley Liddick 57bae985a7 Merge pull request #2711 from wucm667/fix/db-pool-enforce-conn-lifetime-floor
fix(db-pool): 强制连接生命期下限,缓解 lib/pq watchCancel 协程泄漏
2026-06-05 13:55:07 +08:00
bwlcandClaude Opus 4.8 69b4654510 fix(ops): weight TTFT percentiles by streaming sample count
TTFT (first_token_ms) is only recorded for streaming requests, but the
ops dashboard weighted merged TTFT percentiles by success_count (all
successful requests, streaming + non-streaming). When non-streaming
traffic was present this diluted/skewed the merged TTFT figures shown
for longer (pre-aggregated) time ranges; the realtime path was exact.

Add a per-bucket ttft_sample_count (rows that actually recorded
first_token_ms) to ops_metrics_hourly / ops_metrics_daily and weight all
TTFT percentile merges by it instead of success_count:

- hourly/daily pre-agg upserts populate and propagate ttft_sample_count;
  daily TTFT p50/p90/avg now weighted by ttft_sample_count.
- dashboard hourly-row merge and cross-segment combine weight TTFT by
  the streaming sample count; queryUsageLatency returns it for raw
  head/tail fragments.

duration stays weighted by success_count (recorded for every request);
p95/p99/max keep the conservative MAX merge (weight-independent).

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-06-05 10:58:44 +08:00
Pluviobyte b6c0706e30 fix: sync scheduler snapshots after account state clears 2026-06-03 14:18:12 +08:00