Abeautifulsnow c300336281 feat(ai): context window management and token usage tracking ...

* @
feat(ai): add conversation context management and token usage tracking (Phase 4)

## Background
Phase 4 of AI agent loop upgrade: token usage extraction from LLM responses,
cost estimation display, and automatic context window management (summarization
when approaching limits).

## Changes

### Backend (Rust)
- New token_usage::TokenUsage — cross-turn token accumulation
- New agent_loop::maybe_compact — proactive context compaction: token estimation
  → cut-point selection → LLM summarization → fallback summary
  → tool-pair integrity protection → drain/insert
- New agent_loop::context_window_for_model — auto-detect context window by model
- New agent_loop::is_context_length_error — detect upstream context-length errors
- New agent_loop::compact_tool_result_for_context — JSON/text sampling for large results
- Modified agent_loop::run_agent_loop — pre-turn compaction check, retry on context error
- Modified agent_events::AgentEnd — split total_tokens into input_tokens/output_tokens
- New agent_events::ContextCompacted — compaction event with metrics
- New AiConfig.context_window — user-configurable context window size
- New AiChatMessage.kind — context summary message marker
- Modified stream_with_tools & 3 provider stream functions — extract TokenUsage
  - Claude: message_start(input) + message_delta(output)
  - OpenAI: stream_options { include_usage } → final usage chunk
  - Gemini: usageMetadata per chunk, keep last

### Frontend (TypeScript/Vue)
- AiAssistant.vue: token + cost display row, estimateCost() with 7 model families,
  contextCompacted UI step, pendingCompaction deferred apply,
  messagesForAgentHistory post-compaction history reconstruction
- EditorSettingsDialog.vue: contextWindow input (min=1000, auto-detect)
- settingsStore.ts: AiConfig.contextWindow field
- tauri.ts: AgentEvent type updates, AiChatMessage.kind
- i18n: 6 new keys in en.ts and zh-CN.ts

## Fixed Issues
- messages[0] dropped from LLM context after compaction
- estimate_text_tokens hard threshold causing 2.8x estimate jump
- context_compacted event field semantics mismatch with frontend insert position
- validate_summary English-only keywords causing fallback on non-English LLMs
- Stale agentTokens after conversation switch
@

* remove cost computation logic

2026-06-15 18:55:22 +08:00

..

db

fix: normalize channel_binding=require to prefer for postgres

2026-06-15 17:44:00 +08:00

external

feat: make duckdb optional behind duckdb-bundled feature

2026-06-10 18:30:14 +08:00

models

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

schema

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

sql_dialect

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

table_structure_sql

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

agent_catalog.rs

fix(mongodb): support legacy driver fallback

2026-06-15 14:07:31 +08:00

agent_connection.rs

fix(mongodb): support legacy driver fallback

2026-06-15 14:07:31 +08:00

agent_events.rs

feat(ai): context window management and token usage tracking

2026-06-15 18:55:22 +08:00

agent_kv.rs

feat(etcd): add agent-backed etcd support

2026-06-08 00:58:42 +08:00

agent_loop.rs

feat(ai): context window management and token usage tracking

2026-06-15 18:55:22 +08:00

agent_manager.rs

fix: update gbase default agent key test to gbase8a

2026-06-14 12:22:08 +08:00

agent_runtime.rs

feat(agent): unify oracle driver

2026-06-12 16:34:42 +08:00

agent_service.rs

fix(agent): retry and rename fallback when removing old JRE on Windows

2026-06-13 01:56:29 +08:00

agent_tools.rs

feat(ai-agent): Phase 3 — explain_query tool, parallel/sequential execution, DB gating (#1187)

2026-06-15 13:14:59 +08:00

ai.rs

feat(ai): context window management and token usage tracking

2026-06-15 18:55:22 +08:00

cloud_sync.rs

fix(gbase): support 8a and 8s profiles

2026-06-14 10:16:24 +08:00

connection_secrets.rs

fix(gbase): support 8a and 8s profiles

2026-06-14 10:16:24 +08:00

connection.rs

fix(databend): enable editing and procedures

2026-06-15 17:43:44 +08:00

csv_export.rs

Preserve ordered SSH and proxy connection layers (#802)

2026-06-06 23:22:29 +08:00

data_compare.rs

feat: optimize data compare memory usage

2026-06-11 11:57:22 +08:00

data_grid_neo4j_sql.rs

refactor: extract Neo4j and TDengine SQL logic from data_grid_sql.rs

2026-05-25 23:21:47 +08:00

data_grid_sql.rs

fix(databend): enable editing and procedures

2026-06-15 17:43:44 +08:00

data_grid_tdengine_sql.rs

refactor: extract Neo4j and TDengine SQL logic from data_grid_sql.rs

2026-05-25 23:21:47 +08:00

database_capabilities.rs

feat(sidebar): reveal local db file in OS file manager

2026-06-13 16:00:47 +08:00

database_export.rs

fix(postgres): handle tsvector columns

2026-06-15 16:59:01 +08:00

database_search_sql.rs

refactor(sql): move builders to Rust

2026-05-23 01:28:52 +08:00

db_admin_sql.rs

fix(iris): clone table with valid false predicate

2026-06-15 14:23:40 +08:00

driver_runtime.rs

feat: add driver runtime manager

2026-06-04 23:20:15 +08:00

history.rs

feat(history): track database activity

2026-05-10 14:18:20 +08:00

jdbc.rs

feat(jdbc): install drivers from Maven bundles

2026-06-11 21:41:37 +08:00

lib.rs

feat(ai): context window management and token usage tracking

2026-06-15 18:55:22 +08:00

mongo_ops.rs

fix(elasticsearch): apply document filters in browser

2026-06-11 18:56:38 +08:00

object_source_sql.rs

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

path_utils.rs

feat: support H2 file connections

2026-06-08 20:09:27 +08:00

plugins.rs

fix(windows-build): ssh-agent err when building on windows

2026-06-11 17:29:40 +08:00

query_cancel.rs

fix: cancel DuckDB queries via duckdb_interrupt instead of only cancelling async future

2026-06-04 17:20:13 +08:00

query_execution_sql.rs

feat(ai-agent): Phase 3 — explain_query tool, parallel/sequential execution, DB gating (#1187)

2026-06-15 13:14:59 +08:00

query_result_sql.rs

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

query.rs

fix(data): preserve unsafe int64 display

2026-06-15 15:14:38 +08:00

redis_ops.rs

fix(redis): batch SCAN server-side to reduce fetch-all latency

2026-06-15 15:01:13 +08:00

saved_sql.rs

feat(sidebar): support nested groups

2026-06-11 22:35:00 +08:00

schema_diff.rs

feat(schema-diff): professional DDL diff view with deployment-aware colors and custom themes (#1112)

2026-06-12 17:14:05 +08:00

schema.rs

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

sql_analysis.rs

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

sql_dialect.rs

refactor(sql): split dialect helpers

2026-06-10 19:53:36 +08:00

sql_editability.rs

Preserve ordered SSH and proxy connection layers (#802)

2026-06-06 23:22:29 +08:00

sql_file_import.rs

Preserve ordered SSH and proxy connection layers (#802)

2026-06-06 23:22:29 +08:00

sql_risk.rs

添加 Manticore Search 支持 (#1145)

2026-06-13 20:23:36 +08:00

sql.rs

fix(query): skip comment-only SQL execution

2026-06-15 14:18:40 +08:00

sqlite_backup.rs

feat(sqlite): add database backup action

2026-06-13 18:39:12 +08:00

storage.rs

feat(settings): split driver store paths

2026-06-13 17:27:58 +08:00

table_export.rs

Preserve ordered SSH and proxy connection layers (#802)

2026-06-06 23:22:29 +08:00

table_import.rs

fix(import): accept JSON files with UTF-8 BOM

2026-06-08 17:07:17 +08:00

table_structure_sql.rs

feat(mysql): edit foreign keys and triggers

2026-06-12 12:29:46 +08:00

text_export.rs

feat(export): move text exports to Rust

2026-05-22 15:41:33 +08:00

token_usage.rs

feat(ai): context window management and token usage tracking

2026-06-15 18:55:22 +08:00

transfer.rs

feat: add QuestDB support

2026-06-14 22:58:05 +08:00

types.rs

feat(schema-diff): professional DDL diff view with deployment-aware colors and custom themes (#1112)

2026-06-12 17:14:05 +08:00

update.rs

fix(updates): guard portable builds from installer updates (#550)

2026-05-30 19:15:37 +08:00

xlsx_export.rs

feat: support double-clicking .db/.sqlite/.duckdb files to open in IDE

2026-05-29 17:25:52 +08:00