Commit Graph
999 Commits
Author SHA1 Message Date
superman2003 1dedb2097d fix(grok): persist quota exhaustion as rate limit 2026-07-11 20:41:32 +08:00
Bestony 1c02158c2a perf(migrations): index latest API key IP lookups 2026-07-11 17:32:06 +08:00
Bestony 80b7a8d4cb perf(keys): bound latest IP lookup per key 2026-07-11 17:31:30 +08:00
Wesley Liddick 6dd3274aaf Merge pull request #3948 from superman2003/fix/gpt56-billing-usage-integrity
fix(openai): correct GPT-5.6 billing and usage gaps
2026-07-10 15:45:18 +08:00
Wesley Liddick 6a2063b374 Merge pull request #3883 from ShuminFu/fix/setup-token-auto-refresh
fix: 后台自动刷新纳入 setup-token 账号(修复 8h 后 401)
2026-07-10 15:44:54 +08:00
superman2003 de28eba3c8 fix(openai): harden GPT-5.6 billing and usage 2026-07-10 15:13:11 +08:00
Shumin a495d5e302 test: 更新断言以覆盖 setup-token 纳入后台刷新
- ListOAuthRefreshCandidates SQL 断言由 type = 'oauth' 改为
  type IN ('oauth', 'setup-token')。
- ClaudeTokenRefresher.CanRefresh 增加 anthropic setup-token → true 用例。
2026-07-10 02:27:33 -04:00
Wesley Liddick 8b96acde97 Merge pull request #3898 from Arron196/feat/gpt-5.6-cache-billing-stats
feat(openai): 支持 GPT-5.6 系列缓存写入计费与统计
2026-07-10 13:57:29 +08:00
superman2003 fc66a30ffc fix: harden billing concurrency and payment recovery 2026-07-10 10:56:49 +08:00
benjamin 5c15d32ff7 test(repository): stabilize expired lock reconciliation 2026-07-10 10:18:50 +08:00
Shumin 99da308196 fix: 后台自动刷新纳入 setup-token 账号
setup-token 的 access_token 同为 8h 短期令牌(expires_in=28800),
此前被后台刷新服务排除,到期后请求返回 401 authentication_error。
放开 CanRefresh 与候选查询的 type='oauth' 限制,与手动刷新入口
(account.IsOAuth()) 保持一致;实际刷新仍由 NeedsRefresh 基于
expires_at 门控并在分布式锁下执行,不会过度刷新。
2026-07-09 11:05:05 -04:00
shaw 1c2e6503c6 feat: 版本徽章新增近3个历史版本在线回退与手动回退指引
后端:
- UpdateService 新增 ListRollbackVersions/RollbackToVersion,回退目标强制
  限定为比当前版本更旧的近3个正式版本(排除当前/更新/预发布/草稿),
  越界返回 400 ROLLBACK_VERSION_NOT_ALLOWED;下载复用既有 HTTPS 域名
  白名单 + checksum 校验 + 原子换二进制管线
- GitHubReleaseClient 新增 FetchRecentReleases(releases 列表 API)
- 新增 GET /admin/system/rollback-versions;POST /admin/system/rollback
  支持可选 {"version"} body,无 body 保持原 .backup 本地回退行为不变
- 端点均在 /admin 组 adminAuth(IsAdmin+TokenVersion)保护内

前端:
- VersionBadge「已是最新版本」状态下新增版本回退面板:radio 卡片选择
  近3个版本,手动回退方式按部署形态 tab 切换(脚本部署 curl 命令 /
  Docker 镜像 tag 指引),支持一键复制;release 构建提供一键回退并
  复用重启流程;源码构建仅提示不支持在线回退
- 下拉根元素重置 whitespace-normal,修复 sidebar-brand 的
  white-space:nowrap 继承导致的长文本溢出裁切

测试:服务层过滤/排序/上限/非法目标拒绝、handler 双模式与鉴权错误映射、
GitHub client 列表拉取、前端 API 请求体行为共 16 个新增用例
2026-07-09 22:30:04 +08:00
Wesley Liddick 10102807e2 Merge pull request #3869 from Birditch/feat/admin-user-role-and-token-ranking
feat(admin): 用户角色管理 + 用户 Token 排行
2026-07-09 19:50:01 +08:00
BirditchandClaude Opus 4.8 b062b36646 feat(admin): 用量记录页新增"用户 Token 排行"面板
- 在 /admin/usage 新增按用户聚合的 Token 排行表(输入/输出/缓存/总 Token、请求数、费用)
- 支持按列排序、邮箱搜索、Top N (20/50/100/200),点击行下钻到该用户
- 复用现有筛选(用户/时间/模型/分组等)与既有 KPI 卡片、逐条日志
- 后端对 /admin/dashboard/user-breakdown 做加法扩展(向后兼容):
  - UserBreakdownItem 增加 input_tokens/output_tokens/cache_tokens
  - 新增 sort_by(白名单排序,防注入,缺省仍按 actual_cost)
- 新增 sort_by 转发单元测试;更新 UsageView.spec 稳定桩

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 06:04:22 -04:00
shaw 49fb18e073 Merge origin/main into worktree/brave-valley-9578
Resolve conflict in api_key_repo.go: keep both ListAllByUserID (this PR)
and attachLastUsedIPs/latestUsageLogIPs (main), and have ListAllByUserID
attach last-used IPs so the current_concurrency sort path keeps the
last_used_ip column populated.
2026-07-09 17:17:16 +08:00
shaw d4952154ff fix: bill Grok video per second and harden video usage logging
Follow-up fixes for the #3775 audit findings:

- Bill Grok video generation per second of output, matching the xAI rate
  card: parse the request duration (1-15s, upstream default 8s) and compute
  cost as per-second price x duration x count. The built-in rate card values
  were already xAI per-second prices but were previously charged per video,
  undercharging up to 15x with a user-controlled duration.
- Group video_price_* fields are now documented and surfaced as per-second
  rates (USD/s); admin UI labels, placeholders and hints updated accordingly.
- Persist video_count/video_resolution/video_duration_seconds on usage_logs
  (migration 172) so video billing is auditable, and exempt any row with
  video_count > 0 from the image_size check constraint: a video billed via a
  token-mode channel price produces billing_mode='token' with image_count=1
  and no image_size, which the previous constraint rejected, dropping the
  whole billing transaction.
- Only refetch the group in apiKeyWithFreshGroupMediaPricing when the group
  object actually looks like it is missing media pricing fields (both media
  multipliers zero and all prices nil, impossible for a normally loaded
  group), removing a per-usage DB query for groups without overrides.
- Frontend: drop the unused admin.groups.mediaPricing locale block, map
  cleared price inputs to null (create) / -1 (update, cleared via backend
  normalizePrice) instead of sending "" that failed *float64 unmarshalling,
  and align video price placeholders with the text-to-video default model
  (grok-imagine-video 0.05/0.07, 1080p only on 1.5 at 0.25).
2026-07-09 15:38:59 +08:00
Wesley Liddick 9ba0fb3084 Merge pull request #3775 from heathermhuang/codex/grok-media-pricing-labels
fix: add Grok video pricing controls
2026-07-09 15:03:38 +08:00
Bestony@Homelab 5debe1db33 feat(keys): sort by current concurrency 2026-07-08 15:51:22 +08:00
wucm667 7a11b39d6d fix(api-key): check usage log rows close 2026-07-08 15:36:12 +08:00
wucm667 e0d149d511 feat(api-key): show last used IP 2026-07-08 15:26:54 +08:00
Heatherm Huang 889b657451 test: accept casted video billing constraint 2026-07-08 13:50:49 +08:00
Heatherm Huang 4d702e3234 fix: split Grok image and video pricing 2026-07-08 13:50:49 +08:00
shaw db3bd9971e refactor(repository): 纯移动拆分 usage_log_repo.go(4701→212行) 2026-07-08 08:49:21 +08:00
shaw b5604afd3d test(batch-image): 集成测试适配 ReplaceBatchImageItemsForJob 的 indexing 状态守卫
守卫要求 job 处于 indexing 状态才允许重建 item 表;原测试在 created
状态下直接调用被 409 拒绝。改为先断言非 indexing 状态的冲突行为,
再经 created→submitted→indexing 合法链推进后执行原有断言。
2026-07-07 20:13:28 +08:00
shaw 80a229bce5 fix(batch-image): 修复审计发现的计费死锁、状态机与队列原子性缺陷
修复 PR #3768 批量图像 MVP 合并后审计报告中的全部问题:

结算与计费(高危):
- 所有 SETTLEMENT_* 失败(超冻结/计数非法/manifest 冲突/定价缺失/扣费失败)
  统一计入 retry_count 并在耗尽时释放冻结转 failed,消灭 settling 无限
  requeue 导致的冻结余额永久锁死
- 耗尽出口的释放指纹统一为 RequestHash,与 processor/Cancel/recovery 一致;
  release 遇同 request id 指纹冲突视为幂等成功,治愈历史毒消息
- 管理端校验 hold_multiplier >= discount_multiplier,定价快照对存量脏数据钳制
- 释放前校验 per-job hold claim(dedup+归档表),杜绝幻影释放

索引对账(高危):
- provider 输出与提交 custom_id 集对账:未知条目丢弃并记事件,
  漏项补 PROVIDER_RESULT_MISSING 失败行,保证 success+fail == item_count

提交与恢复(高危):
- 提交前转 uploading 并在 provider.Submit 期间心跳刷新 updated_at;
  恢复扫描改为原子复核(FailStaleUnsubmittedBatchImageJob),
  消灭慢提交被误杀退款而上游任务照常计费的孤儿场景
- 上游任务创建成功但本地状态推进失败时,尽力取消上游并清理输入
- recovery 释放失败时入队交由 worker releaseTerminalHold 兜底重试

队列与并发(中危):
- Enqueue(SetNX+LPush)与 Reserve(BRPop+ZAdd)均改为 Lua 原子脚本,
  消灭崩溃窗口导致 job 脱离队列、被 7 天 inflight 键锁死
- 锁冲突按 LockConflictDelay 重新入队(原直接丢弃需等 10 分钟 stale 恢复)
- 处理期间心跳:active zset 续期(ZAddXX 防幽灵成员)+ 锁 TTL 续期
- ReplaceBatchImageItemsForJob 增加 indexing 状态守卫,防掉队 worker 重写账目

存量回归(中危):
- image-only 定价条目(仅图片价无 token 价)恢复 token 计费 fail-closed,
  不再按 $0 计费;图片计费路径不受影响
- 鉴权余额门槛恢复 balance <= 0 语义,MinimumBalanceReserve 不再作硬 403

加固:
- ZIP max_items 钳制到管理员上限;Submit 补齐 Platform==Gemini 校验;
  gemini downloadUri 跟随前做 host 白名单校验
- 批量客户端改用共享 httpclient(拨号/TLS/响应头超时有界)
- 审计点名的忽略错误(MarkDownloaded/SettlementFailed/AppendEvent 等)改为记日志
2026-07-07 18:53:56 +08:00
Turtle_Li 83d7b0e440 fix: isolate duplicate item integration test 2026-07-07 11:24:04 +08:00
Turtle_Li 5494261902 Merge remote-tracking branch 'origin/main' into feature/batch-image-foundation
# Conflicts:
#	.github/audit-exceptions.yml
2026-07-07 10:53:17 +08:00
Turtle_Li aff148167c fix: address batch image ci failures 2026-07-07 10:50:36 +08:00
shaw 7650cce5a4 fix(redis-index): harden index-driven cleanup introduced in #3762
- 启动清理覆盖 score 已过期的索引成员,并新增 marker 门控的一次性
  遗留 wait 键清扫,消除重启后幻影等待计数挤占 maxWait 的问题
- UMQ 争锁失败路径回填观测到的持有者锁索引(PTTL=-1 异常锁以已到期
  候选入索引),恢复被 SCAN 删除后丢失的孤儿锁兜底完整性
- 后台清理同时回收 account 与 user 两个活跃索引的到期候选,修复
  user 索引过期成员无回收路径导致的无界增长
- acquire/incrementWait Lua 脚本返回 {result, now},索引写复用脚本
  内 Redis 时间,热路径每次操作减少一次 TIME 往返;清理循环外提
  now 并批量写索引
- 删除无生产调用方的 GetActiveAccountLoadMap 及配套死代码
- 索引维护失败增加 warn 日志,不再静默丢弃
- 移除根目录工单文档 REDIS_SCAN_ARCHITECTURE_OPTIMIZATION.md
2026-07-07 10:04:38 +08:00
shaw 7c8e13208b style: replace interface{} with any per golangci-lint gofmt rewrite rules 2026-07-07 09:32:15 +08:00
shaw d37d216f9d Merge remote-tracking branch 'origin/main' into pr-3762
# Conflicts:
#	backend/internal/repository/concurrency_cache.go
#	backend/internal/repository/concurrency_cache_integration_test.go
2026-07-07 09:22:44 +08:00
Turtle_Li 1b07fe821a merge: sync batch image branch with origin main 2026-07-07 03:27:11 +08:00
jjaw 5089c303e1 优化 Redis SCAN 清理架构 2026-07-06 20:27:13 +08:00
Wesley Liddick d1d3400b69 Merge pull request #3645 from bestony/worktree/lucky-harbor-dbe4
feat(keys): add api key concurrency stats
2026-07-06 16:45:15 +08:00
Turtle_Li 9703ca9d33 merge: sync batch image foundation with upstream main 2026-07-06 13:40:09 +08:00
Turtle_Li d8e96f0f97 fix: bound batch image settlement retries 2026-07-06 13:33:53 +08:00
Turtle_Li 8fab636998 feat: complete batch image workflow 2026-07-06 12:22:04 +08:00
linshuboy f26ca5661e feat: add OpenAI advanced scheduler controls
Related: #1089, #408, #123
2026-07-05 17:24:38 +08:00
Wesley Liddick 0c53d4f334 Merge pull request #3661 from jianjianai/fix/group-capacity-batch-hotpath
fix: 优化用户组容量汇总批量统计性能
2026-07-04 09:20:02 +08:00
Wesley Liddick 8e07ec55db Merge pull request #3685 from jianjianai/fix/optimize-slot-cleanup
fix: 优化并发槽位定时清理性能
2026-07-04 09:19:34 +08:00
Turtle_Li a994fbd77a feat: add batch image MVP 2026-07-04 05:30:50 +08:00
DaydreamCoding ebbdc70311 feat(usage): 错误请求对齐用量明细(UI/排序/筛选/列设置)
错误请求列表(/admin/usage 错误 tab、Ops 弹窗、用户端 /usage 错误 tab)
对齐用量明细的交互与信息密度。Squash of:

- feat(usage): 错误请求全面对齐用量明细(UI/列序/排序/筛选/列设置/新列)
- refactor(usage): 错误表提取共享徽章工具与 IP 批量工具条,后端排序解析归并 SetSort
- feat(usage): /admin/usage 错误请求新增分类过滤
- fix(ops): 错误列表 phase=upstream 过滤生效,守卫豁免改为显式 opt-in
- fix(ops): 错误列表用户列回退显示已删除 KEY 所有者
- fix(usage): 错误请求状态码排序对齐展示/过滤,筛选项改固定常用码
2026-07-03 23:02:27 +08:00
shaw a1b2b32e08 fix: prevent silent usage_logs drops under queue overflow (#3656)
高并发下扣费成功但 usage_logs 被双层异步队列静默丢弃(单用户实测丢失 49%),
导致消费排行与对账缺口。根因是 6a685727 为避免溢出时单行 INSERT 踩踏数据库,
把三处"队列满"改成了立即终态丢弃。

改为有界阻塞背压,兼顾数据不丢与批量写入模式不变:

- CreateBestEffort/createBatched 入队移除 default 立即丢弃分支,
  队列满时阻塞等待(受 detached 15s ctx 期限约束)
- writeUsageLogBestEffort 收到 dropped 不再直接放弃,统一走
  repo.Create 同步兜底(仍经批处理器;ctx 耗尽时换新 detached 窗口),
  重复写入由 ON CONFLICT DO NOTHING 幂等防护
- worker 池默认溢出策略 sample→sync(viper 默认 + 池常量),
  溢出时提交方内联执行(提交点在响应写出后,不阻塞客户端),
  显式配置 sample/drop 的部署不受影响
- 附带修复 ensure*Batcher 的存量数据竞争:channel nil 检查移入 sync.Once

正常负载零行为变化;突发时短暂等待替代永久丢失;数据库只见批量 INSERT。
2026-07-03 21:02:32 +08:00
jjaw f3a3a0869d 优化并发槽位清理 2026-07-03 21:02:31 +08:00
jianjian c620ad6a36 fix: align group capacity SQL with target schema 2026-07-03 08:44:39 +00:00
jianjian 72ccd1b111 fix: batch group capacity summaries 2026-07-02 20:22:17 +00:00
jianjian 3f2ef60468 fix: optimize ops realtime account stats 2026-07-02 20:04:47 +00:00
Wesley Liddick 821399ade1 Merge pull request #3622 from deqiying/feat/subscription-revoke-restore
支持恢复已撤销订阅
2026-07-02 17:34:20 +08:00
Wesley Liddick 51b5142af7 Merge pull request #3612 from StarryKira/codex/fix-user-model-stats-requested-model
[codex] fix user model stats requested aggregation
2026-07-02 17:32:19 +08:00
Wesley Liddick 634b0eee48 Merge pull request #3430 from chaochaoweb3/codex/remove-claude-oauth-expires-in
fix: omit expires_in from Claude OAuth token exchange
2026-07-02 17:32:09 +08:00