Abeautifulsnow
c300336281
* @
feat(ai): add conversation context management and token usage tracking (Phase 4)
## Background
Phase 4 of AI agent loop upgrade: token usage extraction from LLM responses,
cost estimation display, and automatic context window management (summarization
when approaching limits).
## Changes
### Backend (Rust)
- New token_usage::TokenUsage — cross-turn token accumulation
- New agent_loop::maybe_compact — proactive context compaction: token estimation
→ cut-point selection → LLM summarization → fallback summary
→ tool-pair integrity protection → drain/insert
- New agent_loop::context_window_for_model — auto-detect context window by model
- New agent_loop::is_context_length_error — detect upstream context-length errors
- New agent_loop::compact_tool_result_for_context — JSON/text sampling for large results
- Modified agent_loop::run_agent_loop — pre-turn compaction check, retry on context error
- Modified agent_events::AgentEnd — split total_tokens into input_tokens/output_tokens
- New agent_events::ContextCompacted — compaction event with metrics
- New AiConfig.context_window — user-configurable context window size
- New AiChatMessage.kind — context summary message marker
- Modified stream_with_tools & 3 provider stream functions — extract TokenUsage
- Claude: message_start(input) + message_delta(output)
- OpenAI: stream_options { include_usage } → final usage chunk
- Gemini: usageMetadata per chunk, keep last
### Frontend (TypeScript/Vue)
- AiAssistant.vue: token + cost display row, estimateCost() with 7 model families,
contextCompacted UI step, pendingCompaction deferred apply,
messagesForAgentHistory post-compaction history reconstruction
- EditorSettingsDialog.vue: contextWindow input (min=1000, auto-detect)
- settingsStore.ts: AiConfig.contextWindow field
- tauri.ts: AgentEvent type updates, AiChatMessage.kind
- i18n: 6 new keys in en.ts and zh-CN.ts
## Fixed Issues
- messages[0] dropped from LLM context after compaction
- estimate_text_tokens hard threshold causing 2.8x estimate jump
- context_compacted event field semantics mismatch with frontend insert position
- validate_summary English-only keywords causing fallback on non-English LLMs
- Stale agentTokens after conversation switch
@
* remove cost computation logic
95 lines
3.2 KiB
Rust
95 lines
3.2 KiB
Rust
use serde::{Deserialize, Serialize};
|
|
|
|
/// Events emitted by the agent loop, streamed to the frontend via SSE.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
#[serde(tag = "type", rename_all = "snake_case")]
|
|
pub enum AgentEvent {
|
|
/// A turn in the agent loop has started.
|
|
TurnStart { turn: u32 },
|
|
/// Text delta from the LLM response.
|
|
TextDelta { delta: String },
|
|
/// Reasoning/thinking delta (for models that support it).
|
|
ReasoningDelta { delta: String },
|
|
/// The LLM wants to call a tool.
|
|
ToolCallStart { tool_call_id: String, tool_name: String, args: serde_json::Value },
|
|
/// A tool execution has completed.
|
|
ToolCallEnd { tool_call_id: String, tool_name: String, result: serde_json::Value, is_error: bool },
|
|
/// A turn in the agent loop has ended.
|
|
TurnEnd { turn: u32 },
|
|
/// The agent loop has finished.
|
|
AgentEnd { input_tokens: Option<u32>, output_tokens: Option<u32> },
|
|
/// Context was compacted to stay within context window limits.
|
|
ContextCompacted {
|
|
summary: String,
|
|
summary_tokens: u32,
|
|
compacted_messages: usize,
|
|
estimated_before: u32,
|
|
estimated_after: u32,
|
|
},
|
|
/// An error occurred during the agent loop.
|
|
Error { message: String },
|
|
}
|
|
|
|
/// A unified tool call extracted from any provider's response.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct ToolCall {
|
|
pub id: String,
|
|
pub name: String,
|
|
pub arguments: serde_json::Value,
|
|
}
|
|
|
|
/// Result of executing a tool.
|
|
#[derive(Debug, Clone, Serialize, Deserialize)]
|
|
pub struct ToolResult {
|
|
pub tool_call_id: String,
|
|
pub tool_name: String,
|
|
pub content: String,
|
|
pub is_error: bool,
|
|
/// Structured explain data (QueryResult) for explain_query tool results.
|
|
/// Set only by execute_explain_query; None for all other tools.
|
|
#[serde(skip_serializing_if = "Option::is_none")]
|
|
pub explain_data: Option<serde_json::Value>,
|
|
}
|
|
|
|
/// Definition of a tool available to the agent.
|
|
#[derive(Debug, Clone)]
|
|
pub struct ToolDefinition {
|
|
pub name: &'static str,
|
|
pub description: &'static str,
|
|
pub parameters: serde_json::Value,
|
|
pub read_only: bool,
|
|
/// Whether this tool can be executed in parallel with other tools.
|
|
/// false = must run sequentially (e.g. execute_query).
|
|
pub parallel_ok: bool,
|
|
}
|
|
|
|
impl ToolDefinition {
|
|
/// Convert to OpenAI-compatible tool JSON for the API request.
|
|
pub fn to_openai_tool(&self) -> serde_json::Value {
|
|
serde_json::json!({
|
|
"type": "function",
|
|
"function": {
|
|
"name": self.name,
|
|
"description": self.description,
|
|
"parameters": self.parameters
|
|
}
|
|
})
|
|
}
|
|
/// Convert to Anthropic-compatible tool JSON.
|
|
pub fn to_anthropic_tool(&self) -> serde_json::Value {
|
|
serde_json::json!({
|
|
"name": self.name,
|
|
"description": self.description,
|
|
"input_schema": self.parameters
|
|
})
|
|
}
|
|
/// Convert to Gemini-compatible function declaration.
|
|
pub fn to_gemini_tool(&self) -> serde_json::Value {
|
|
serde_json::json!({
|
|
"name": self.name,
|
|
"description": self.description,
|
|
"parameters": self.parameters
|
|
})
|
|
}
|
|
}
|