Commit Graph
818 Commits
Author SHA1 Message Date
benjamin e9fb5983cd fix(billing): 默认关闭 OpenAI 长上下文计费 2026-07-13 23:32:16 +08:00
hongheshan-svgandClaude Opus 4.8 98027cdded fix(openai): enable HTTP/2 keep-alive PING to evict dead Codex connections
Codex/OpenAI upstream switched from a WebSocket pool to HTTP/2. The
outbound H2 transport set neither ReadIdleTimeout nor ResponseHeaderTimeout
(the OpenAI profile forces ResponseHeaderTimeout=0), so a pooled H2
connection silently killed by a proxy/NAT becomes a "dead connection":
both ends believe it is alive and a request assigned to it hangs until the
OS TCP retransmit timeout (minutes) before the first byte — observed as an
8m37s TTFT with an eventual 200. Occasional (only when a request lands on a
dead pooled conn) and across all groups (shared OpenAI transport); worse on
larger idle pools.

Explicitly configure http2 on the openai_h2 transport and enable active
PING health checks (ReadIdleTimeout=15s, PingTimeout=15s) so dead
connections are detected and evicted at the source, instead of relying on
ResponseHeaderTimeout as an after-the-fact backstop. Scoped to the
openai_h2 path only; Claude/Gemini (default) and h1 modes are untouched.

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
2026-07-13 23:31:18 +08:00
jjaw a0778e9a42 修复调度器使用已消费事件计算延迟 2026-07-13 21:56:47 +08:00
superman2003 c896cacf6d feat(grok): improve free quota probing and usage 2026-07-13 19:49:56 +08:00
benjamin 3e4d48e010 Merge remote-tracking branch 'upstream/main' into fix/api-double-billing
# Conflicts:
#	frontend/src/components/account/EditAccountModal.vue
2026-07-13 18:06:44 +08:00
benjamin a0ac5e0240 fix(billing): 默认开启 OpenAI 长上下文计费 2026-07-13 17:55:01 +08:00
Bestony@Homelab c8cfc93632 fix(openai-ws): bound ingress session lifecycle 2026-07-13 15:32:42 +08:00
Wesley Liddick baab0adf7c Merge pull request #4075 from jianjianai/codex/fix-scheduler-cache-invalid-time-upstream
fix(scheduler): skip cache records with unencodable times
2026-07-13 14:42:05 +08:00
Wesley Liddick 90bff0ea17 Merge pull request #4050 from bestony/perf/keys-last-used-ip-query
perf(keys): bound latest IP lookup per key
2026-07-13 14:13:11 +08:00
benjamin 0d9c140bc2 Merge upstream/main into fix/api-double-billing 2026-07-13 11:10:33 +08:00
Wesley Liddick b73d8c3efe Merge pull request #4009 from heathermhuang/codex/fix-recent-grok-issues
fix: expand Grok API, CLI, billing, and setup support
2026-07-13 10:36:11 +08:00
shaw 64a2a31729 fix(billing): 修复复审发现的三处按次计费问题
- HIGH: GetByKeyForAuth 的 WithGroup 显式 Select 补 web_search_price_per_call 列,
  否则 auth 快照里分组覆盖价恒为 nil,覆盖价/免费配置在实际计费中静默失效
- MEDIUM: 按次搜索倍率改用不含高峰因子的基础倍率(与 image/video 按次不变式
  及分组表单价格预览承诺一致),calculateOpenAIRecordUsageCost 新增独立倍率参数
- LOW: 计费门槛从 <400 收紧为严格 2xx,1xx/3xx 透传不计费
2026-07-13 10:14:53 +08:00
Heatherm Huang c4ff604e93 test(grok): cover OAuth chat permission identity 2026-07-13 10:11:33 +08:00
Heatherm Huang ce3f12bbff test(grok): cover CLI identity at transport boundary 2026-07-13 10:11:33 +08:00
Heatherm Huang f187f08ae3 fix(grok): harden OAuth routing and CLI version guard 2026-07-13 10:11:33 +08:00
Heatherm Huang 038b25c0b1 fix(grok): resolve recent integration issues 2026-07-13 10:09:53 +08:00
shaw 7cbb36f278 feat(billing): Codex alpha/search 网页搜索按次计费
- alpha/search 成功请求(上游 2xx)按次计费落 usage_logs(billing_mode=per_request),
  上游错误透传/failover 不计费;使用 mandatory 池提交,池满同步兜底不丢扣费
- 单价默认 0.01 USD/次(官方 $10/1000 次),分组新增 web_search_price_per_call
  覆盖价(0=免费,负数/留空=默认价),实际扣费叠加分组费率倍数(与 token 口径一致)
- 分组字段全链路:ent schema + 迁移 174 + 端口快照 v15 + admin 创建/更新 + DTO
- 前端分组表单(openai 平台)新增单次价格配置,实时预览应用当前倍率后的单次价格
- 该端点鉴权维持仅 OpenAI 分组(非 OpenAI 分组 404)
2026-07-13 09:54:51 +08:00
benjamin 92dcfb5ebc fix(billing): 按账号控制 OpenAI 长上下文计费 2026-07-13 09:47:19 +08:00
jjaw fe184f8c33 修复调度缓存异常时间阻塞
Refs Wei-Shaw/sub2api#60

Refs Wei-Shaw/sub2api#1608
2026-07-12 05:54:46 +08:00
superman2003 1dedb2097d fix(grok): persist quota exhaustion as rate limit 2026-07-11 20:41:32 +08:00
Bestony 1c02158c2a perf(migrations): index latest API key IP lookups 2026-07-11 17:32:06 +08:00
Bestony 80b7a8d4cb perf(keys): bound latest IP lookup per key 2026-07-11 17:31:30 +08:00
Wesley Liddick 6dd3274aaf Merge pull request #3948 from superman2003/fix/gpt56-billing-usage-integrity
fix(openai): correct GPT-5.6 billing and usage gaps
2026-07-10 15:45:18 +08:00
Wesley Liddick 6a2063b374 Merge pull request #3883 from ShuminFu/fix/setup-token-auto-refresh
fix: 后台自动刷新纳入 setup-token 账号(修复 8h 后 401)
2026-07-10 15:44:54 +08:00
superman2003 de28eba3c8 fix(openai): harden GPT-5.6 billing and usage 2026-07-10 15:13:11 +08:00
Shumin a495d5e302 test: 更新断言以覆盖 setup-token 纳入后台刷新
- ListOAuthRefreshCandidates SQL 断言由 type = 'oauth' 改为
  type IN ('oauth', 'setup-token')。
- ClaudeTokenRefresher.CanRefresh 增加 anthropic setup-token → true 用例。
2026-07-10 02:27:33 -04:00
Wesley Liddick 8b96acde97 Merge pull request #3898 from Arron196/feat/gpt-5.6-cache-billing-stats
feat(openai): 支持 GPT-5.6 系列缓存写入计费与统计
2026-07-10 13:57:29 +08:00
superman2003 fc66a30ffc fix: harden billing concurrency and payment recovery 2026-07-10 10:56:49 +08:00
benjamin 5c15d32ff7 test(repository): stabilize expired lock reconciliation 2026-07-10 10:18:50 +08:00
Shumin 99da308196 fix: 后台自动刷新纳入 setup-token 账号
setup-token 的 access_token 同为 8h 短期令牌(expires_in=28800),
此前被后台刷新服务排除,到期后请求返回 401 authentication_error。
放开 CanRefresh 与候选查询的 type='oauth' 限制,与手动刷新入口
(account.IsOAuth()) 保持一致;实际刷新仍由 NeedsRefresh 基于
expires_at 门控并在分布式锁下执行,不会过度刷新。
2026-07-09 11:05:05 -04:00
shaw 1c2e6503c6 feat: 版本徽章新增近3个历史版本在线回退与手动回退指引
后端:
- UpdateService 新增 ListRollbackVersions/RollbackToVersion,回退目标强制
  限定为比当前版本更旧的近3个正式版本(排除当前/更新/预发布/草稿),
  越界返回 400 ROLLBACK_VERSION_NOT_ALLOWED;下载复用既有 HTTPS 域名
  白名单 + checksum 校验 + 原子换二进制管线
- GitHubReleaseClient 新增 FetchRecentReleases(releases 列表 API)
- 新增 GET /admin/system/rollback-versions;POST /admin/system/rollback
  支持可选 {"version"} body,无 body 保持原 .backup 本地回退行为不变
- 端点均在 /admin 组 adminAuth(IsAdmin+TokenVersion)保护内

前端:
- VersionBadge「已是最新版本」状态下新增版本回退面板:radio 卡片选择
  近3个版本,手动回退方式按部署形态 tab 切换(脚本部署 curl 命令 /
  Docker 镜像 tag 指引),支持一键复制;release 构建提供一键回退并
  复用重启流程;源码构建仅提示不支持在线回退
- 下拉根元素重置 whitespace-normal,修复 sidebar-brand 的
  white-space:nowrap 继承导致的长文本溢出裁切

测试:服务层过滤/排序/上限/非法目标拒绝、handler 双模式与鉴权错误映射、
GitHub client 列表拉取、前端 API 请求体行为共 16 个新增用例
2026-07-09 22:30:04 +08:00
Wesley Liddick 10102807e2 Merge pull request #3869 from Birditch/feat/admin-user-role-and-token-ranking
feat(admin): 用户角色管理 + 用户 Token 排行
2026-07-09 19:50:01 +08:00
BirditchandClaude Opus 4.8 b062b36646 feat(admin): 用量记录页新增"用户 Token 排行"面板
- 在 /admin/usage 新增按用户聚合的 Token 排行表(输入/输出/缓存/总 Token、请求数、费用)
- 支持按列排序、邮箱搜索、Top N (20/50/100/200),点击行下钻到该用户
- 复用现有筛选(用户/时间/模型/分组等)与既有 KPI 卡片、逐条日志
- 后端对 /admin/dashboard/user-breakdown 做加法扩展(向后兼容):
  - UserBreakdownItem 增加 input_tokens/output_tokens/cache_tokens
  - 新增 sort_by(白名单排序,防注入,缺省仍按 actual_cost)
- 新增 sort_by 转发单元测试;更新 UsageView.spec 稳定桩

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
2026-07-09 06:04:22 -04:00
shaw 49fb18e073 Merge origin/main into worktree/brave-valley-9578
Resolve conflict in api_key_repo.go: keep both ListAllByUserID (this PR)
and attachLastUsedIPs/latestUsageLogIPs (main), and have ListAllByUserID
attach last-used IPs so the current_concurrency sort path keeps the
last_used_ip column populated.
2026-07-09 17:17:16 +08:00
shaw d4952154ff fix: bill Grok video per second and harden video usage logging
Follow-up fixes for the #3775 audit findings:

- Bill Grok video generation per second of output, matching the xAI rate
  card: parse the request duration (1-15s, upstream default 8s) and compute
  cost as per-second price x duration x count. The built-in rate card values
  were already xAI per-second prices but were previously charged per video,
  undercharging up to 15x with a user-controlled duration.
- Group video_price_* fields are now documented and surfaced as per-second
  rates (USD/s); admin UI labels, placeholders and hints updated accordingly.
- Persist video_count/video_resolution/video_duration_seconds on usage_logs
  (migration 172) so video billing is auditable, and exempt any row with
  video_count > 0 from the image_size check constraint: a video billed via a
  token-mode channel price produces billing_mode='token' with image_count=1
  and no image_size, which the previous constraint rejected, dropping the
  whole billing transaction.
- Only refetch the group in apiKeyWithFreshGroupMediaPricing when the group
  object actually looks like it is missing media pricing fields (both media
  multipliers zero and all prices nil, impossible for a normally loaded
  group), removing a per-usage DB query for groups without overrides.
- Frontend: drop the unused admin.groups.mediaPricing locale block, map
  cleared price inputs to null (create) / -1 (update, cleared via backend
  normalizePrice) instead of sending "" that failed *float64 unmarshalling,
  and align video price placeholders with the text-to-video default model
  (grok-imagine-video 0.05/0.07, 1080p only on 1.5 at 0.25).
2026-07-09 15:38:59 +08:00
Wesley Liddick 9ba0fb3084 Merge pull request #3775 from heathermhuang/codex/grok-media-pricing-labels
fix: add Grok video pricing controls
2026-07-09 15:03:38 +08:00
Bestony@Homelab 5debe1db33 feat(keys): sort by current concurrency 2026-07-08 15:51:22 +08:00
wucm667 7a11b39d6d fix(api-key): check usage log rows close 2026-07-08 15:36:12 +08:00
wucm667 e0d149d511 feat(api-key): show last used IP 2026-07-08 15:26:54 +08:00
Heatherm Huang 889b657451 test: accept casted video billing constraint 2026-07-08 13:50:49 +08:00
Heatherm Huang 4d702e3234 fix: split Grok image and video pricing 2026-07-08 13:50:49 +08:00
shaw db3bd9971e refactor(repository): 纯移动拆分 usage_log_repo.go(4701→212行) 2026-07-08 08:49:21 +08:00
shaw b5604afd3d test(batch-image): 集成测试适配 ReplaceBatchImageItemsForJob 的 indexing 状态守卫
守卫要求 job 处于 indexing 状态才允许重建 item 表;原测试在 created
状态下直接调用被 409 拒绝。改为先断言非 indexing 状态的冲突行为,
再经 created→submitted→indexing 合法链推进后执行原有断言。
2026-07-07 20:13:28 +08:00
shaw 80a229bce5 fix(batch-image): 修复审计发现的计费死锁、状态机与队列原子性缺陷
修复 PR #3768 批量图像 MVP 合并后审计报告中的全部问题:

结算与计费(高危):
- 所有 SETTLEMENT_* 失败(超冻结/计数非法/manifest 冲突/定价缺失/扣费失败)
  统一计入 retry_count 并在耗尽时释放冻结转 failed,消灭 settling 无限
  requeue 导致的冻结余额永久锁死
- 耗尽出口的释放指纹统一为 RequestHash,与 processor/Cancel/recovery 一致;
  release 遇同 request id 指纹冲突视为幂等成功,治愈历史毒消息
- 管理端校验 hold_multiplier >= discount_multiplier,定价快照对存量脏数据钳制
- 释放前校验 per-job hold claim(dedup+归档表),杜绝幻影释放

索引对账(高危):
- provider 输出与提交 custom_id 集对账:未知条目丢弃并记事件,
  漏项补 PROVIDER_RESULT_MISSING 失败行,保证 success+fail == item_count

提交与恢复(高危):
- 提交前转 uploading 并在 provider.Submit 期间心跳刷新 updated_at;
  恢复扫描改为原子复核(FailStaleUnsubmittedBatchImageJob),
  消灭慢提交被误杀退款而上游任务照常计费的孤儿场景
- 上游任务创建成功但本地状态推进失败时,尽力取消上游并清理输入
- recovery 释放失败时入队交由 worker releaseTerminalHold 兜底重试

队列与并发(中危):
- Enqueue(SetNX+LPush)与 Reserve(BRPop+ZAdd)均改为 Lua 原子脚本,
  消灭崩溃窗口导致 job 脱离队列、被 7 天 inflight 键锁死
- 锁冲突按 LockConflictDelay 重新入队(原直接丢弃需等 10 分钟 stale 恢复)
- 处理期间心跳:active zset 续期(ZAddXX 防幽灵成员)+ 锁 TTL 续期
- ReplaceBatchImageItemsForJob 增加 indexing 状态守卫,防掉队 worker 重写账目

存量回归(中危):
- image-only 定价条目(仅图片价无 token 价)恢复 token 计费 fail-closed,
  不再按 $0 计费;图片计费路径不受影响
- 鉴权余额门槛恢复 balance <= 0 语义,MinimumBalanceReserve 不再作硬 403

加固:
- ZIP max_items 钳制到管理员上限;Submit 补齐 Platform==Gemini 校验;
  gemini downloadUri 跟随前做 host 白名单校验
- 批量客户端改用共享 httpclient(拨号/TLS/响应头超时有界)
- 审计点名的忽略错误(MarkDownloaded/SettlementFailed/AppendEvent 等)改为记日志
2026-07-07 18:53:56 +08:00
Turtle_Li 83d7b0e440 fix: isolate duplicate item integration test 2026-07-07 11:24:04 +08:00
Turtle_Li 5494261902 Merge remote-tracking branch 'origin/main' into feature/batch-image-foundation
# Conflicts:
#	.github/audit-exceptions.yml
2026-07-07 10:53:17 +08:00
Turtle_Li aff148167c fix: address batch image ci failures 2026-07-07 10:50:36 +08:00
shaw 7650cce5a4 fix(redis-index): harden index-driven cleanup introduced in #3762
- 启动清理覆盖 score 已过期的索引成员,并新增 marker 门控的一次性
  遗留 wait 键清扫,消除重启后幻影等待计数挤占 maxWait 的问题
- UMQ 争锁失败路径回填观测到的持有者锁索引(PTTL=-1 异常锁以已到期
  候选入索引),恢复被 SCAN 删除后丢失的孤儿锁兜底完整性
- 后台清理同时回收 account 与 user 两个活跃索引的到期候选,修复
  user 索引过期成员无回收路径导致的无界增长
- acquire/incrementWait Lua 脚本返回 {result, now},索引写复用脚本
  内 Redis 时间,热路径每次操作减少一次 TIME 往返;清理循环外提
  now 并批量写索引
- 删除无生产调用方的 GetActiveAccountLoadMap 及配套死代码
- 索引维护失败增加 warn 日志,不再静默丢弃
- 移除根目录工单文档 REDIS_SCAN_ARCHITECTURE_OPTIMIZATION.md
2026-07-07 10:04:38 +08:00
shaw 7c8e13208b style: replace interface{} with any per golangci-lint gofmt rewrite rules 2026-07-07 09:32:15 +08:00
shaw d37d216f9d Merge remote-tracking branch 'origin/main' into pr-3762
# Conflicts:
#	backend/internal/repository/concurrency_cache.go
#	backend/internal/repository/concurrency_cache_integration_test.go
2026-07-07 09:22:44 +08:00