feat(core): split foundational capabilities into dedicated crates
This commit is contained in:
402 files changed
+4386
-3367
No files matched your search
@@ -0,0 +1,126 @@
|
||||
# DBX Rust 模块边界
|
||||
|
||||
## 依赖方向
|
||||
|
||||
`dbx-core` 是应用编排层,不再承载所有底层实现。桌面、Web、CLI、MCP
|
||||
继续通过 core 调用连接、查询、导入导出等业务;独立工具可直接使用下层 crate。
|
||||
|
||||
```text
|
||||
desktop / web / cli / mcp
|
||||
│
|
||||
dbx-core
|
||||
├── dbx-drivers ── dbx-sql ── dbx-types
|
||||
│ ├── dbx-types
|
||||
│ ├── dbx-platform
|
||||
│ └── dbx-sqlite-worker (protocol, no runtime defaults)
|
||||
├── dbx-plugin-runtime ── dbx-types + dbx-platform
|
||||
├── dbx-ai-provider ── dbx-platform
|
||||
├── dbx-formats
|
||||
├── dbx-sql / dbx-types
|
||||
└── dbx-platform
|
||||
```
|
||||
|
||||
下层 crate 不得反向依赖 core,包括 build 和 dev 依赖。涉及多层的测试放在
|
||||
`dbx-core/tests/`,避免为了测试把数据库连接引入 SQL、类型或格式 crate。
|
||||
`scripts/core-architecture.test.mjs` 校验 workspace 依赖、功能转发、源码归属、
|
||||
构建输入和前端测试引用路径;CI 的 Rust 检查执行此守卫。
|
||||
|
||||
## 所有权
|
||||
|
||||
| Crate | 拥有的实现 | 不应引入 |
|
||||
| --- | --- | --- |
|
||||
| `dbx-types` | 连接配置、数据库身份、查询与元数据 DTO、JS 安全 JSON、MQTT/GridFS 数据记录 | 连接池、应用状态、驱动 SDK |
|
||||
| `dbx-sql` | 解析、方言注册与加载、SQL 风险、DDL/DML、结构差异计划 | 查询执行、凭据、core 业务 |
|
||||
| `dbx-drivers` | 原生驱动、隧道、JDBC/Agent 运行与分发、超时取消、SQLite 元数据读取 | 导入/迁移编排、应用配置持久化 |
|
||||
| `dbx-formats` | CSV/XLSX/文本/时间格式与 ZIP 编码 | 数据库请求、连接管理 |
|
||||
| `dbx-ai-provider` | 模型与 CLI 适配器、流式协议、事件和 token 用量 | 数据库工具执行和 Agent 业务循环 |
|
||||
| `dbx-plugin-runtime` | manifest、签名与安装、Marketplace、子进程会话及 host 请求 | 数据库驱动实现、core 应用状态 |
|
||||
| `dbx-platform` | 进程、路径、代理、下载、版本比较与共享用户提示网关 | 数据库身份和业务规则 |
|
||||
|
||||
`dbx-sql` 包含方言文件加载/监听,因此不是完全无 I/O 的函数库;边界是“不执行数据库业务”。
|
||||
平台的 `host-prompts`、`downloads` 和类型库的 `mq-admin` 等能力显式启用,
|
||||
避免只使用类型或纯格式的消费者意外加载驱动或应用层。
|
||||
|
||||
## Core 业务目录
|
||||
|
||||
```text
|
||||
dbx-core/src/
|
||||
lib.rs 对外模块与旧 API 兼容导出
|
||||
connection/ 连接路由、凭据、运行配置、JDBC 配置、任务监督
|
||||
query/ 查询编排、取消、文档/Redis/HBase 操作、事务与对象缓存
|
||||
schema/ 元数据编排、运行时 SQLite 表重建
|
||||
data/ 导入导出、迁移、比较、备份、文档与脚本工作流
|
||||
ai/ 数据库 Agent 循环、工具、解释、模板与 MCP 策略
|
||||
admin/ Nacos、Consul、MQ、MQTT
|
||||
persistence/ 本地存储、历史、保存的 SQL、配置与云同步
|
||||
safety/ 生产安全、写入解锁和风险指标
|
||||
host/ 更新、变更日志和外部应用集成
|
||||
db/ 驱动兼容导出与 Cloudflare D1 业务接口覆盖
|
||||
```
|
||||
|
||||
目录内存放实际实现,不以 `#[path]` 把旧根目录伪装成新目录。已存在但未声明的
|
||||
实验文件不会因迁移而自动启用。
|
||||
|
||||
Cloudflare D1 的批量导入依赖迁移逻辑,保留在 `data/cloudflare_d1/`;HTTP 驱动、
|
||||
SQL 限制与词法处理位于 drivers。SQLite 表重建执行留在 core,SQL 生成位于 sql。
|
||||
数据库查询驱动的 CSV 导出编排留在 core,CSV 编码位于 formats。
|
||||
|
||||
## 兼容性
|
||||
|
||||
- 原有 `dbx_core::models`、`dbx_core::types`、`dbx_core::db`、`dbx_core::sql*`、
|
||||
`dbx_core::ai`、`dbx_core::plugins` 及业务模块路径继续通过重导出可用。
|
||||
- 同一 DTO、连接池、提示请求和插件类型不复制定义;旧路径与新 crate 具有相同类型身份。
|
||||
`public_api_compatibility` 测试覆盖这些跨层身份。
|
||||
- 协议、序列化字段、默认配置、数据库语义、安装目录与进程生命周期不因搬迁而改变。
|
||||
原测试随实现迁移;依赖 core 的跨层测试移入 core 集成测试目录。
|
||||
- `test-support` 仅提供原有 stub/测试钩子,由 dev-dependency 启用,不在生产依赖启用。
|
||||
- 模块路径产生的日志 target 和 Rust 调试类型名会随 crate/目录变化。自定义
|
||||
`RUST_LOG=dbx_core=debug` 不会覆盖新 crate;需要加入
|
||||
`dbx_drivers=debug,dbx_sql=debug,dbx_plugin_runtime=debug,dbx_ai_provider=debug,dbx_platform=debug`
|
||||
等目标。默认桌面日志级别及 Web 默认过滤策略保持原样。
|
||||
|
||||
## Features 与资源
|
||||
|
||||
core 保留原来的 default feature 集合,并向实际实现 crate 转发
|
||||
`duckdb-sidecar`、`dynamodb`、`mq-admin`、`openapi` 与 SQLite 能力。
|
||||
`sqlite-bundled`、`sqlite-sqlcipher`、`sqlite-multiple-ciphers` 是不同的 SQLite
|
||||
后端选择,不要用 workspace `--all-features` 代替合法组合测试。
|
||||
|
||||
- `dbx-types/build.rs` 消费 `plugins/connection-types/`。
|
||||
- `dbx-sql/build.rs` 消费 `plugins/dialects/`。
|
||||
- Pi MCP bridge 随 AI provider;Agent v2 协议 JSON 随 drivers,并与 Java resource 对照。
|
||||
- 数据库文档导出的 JS/CSS 仍在 `dbx-core/assets/`,由 `pnpm build:docs-export` 生成。
|
||||
- Docker 的依赖缓存阶段必须包含所有 workspace manifest 和 stub,实际构建阶段
|
||||
必须包含方言与连接类型目录。Nix 保留整个仓库源码输入。
|
||||
|
||||
## 验证
|
||||
|
||||
本机不启动数据库/引擎或 Docker 实例。单元测试包含内存 SQLite 与本地协议模拟器;
|
||||
真实数据库验证复用 SSH 测试服务器上的专用测试实例,使用 SSH 隧道访问。
|
||||
Linux 的文件权限回归测试应由非 root 用户执行;root 可绕过目录写权限,会让测试
|
||||
前提失效。测试进程的代理应排除 localhost,避免本地 HTTP 模拟器请求被转发。
|
||||
|
||||
```sh
|
||||
pnpm test:architecture
|
||||
cargo check -p dbx-core --no-default-features --all-targets
|
||||
|
||||
env -u DOCKER_CONTEXT \
|
||||
DOCKER_HOST=unix:///tmp/dbx-disabled-docker.sock \
|
||||
RUST_MIN_STACK=33554432 \
|
||||
cargo test -p dbx-core -p dbx-drivers -p dbx-sql \
|
||||
-p dbx-types -p dbx-formats -p dbx-platform \
|
||||
-p dbx-ai-provider -p dbx-plugin-runtime \
|
||||
--no-default-features --features dbx-core/sqlite-bundled --lib
|
||||
|
||||
cargo test -p dbx-core --no-default-features --features sqlite-bundled \
|
||||
--test public_api_compatibility --test connection_url_compatibility \
|
||||
--test agent_recovery_contract
|
||||
```
|
||||
|
||||
交付验证还应覆盖桌面/Web/CLI/MCP 消费者、合法 feature 组合、前端类型与源码契约、
|
||||
Agent 安装/恢复以及远程数据库查询/元数据/导入导出。`#[ignore]` 的 live 测试需要其
|
||||
声明的测试环境变量;未运行的 live 用例不算通过。禁止把静态构建输入检查描述成
|
||||
Docker/Nix 镜像已构建,也不能把单元测试通过描述成所有数据库版本均已实测。
|
||||
|
||||
拆分的直接收益是可独立检查的依赖边界、实现所有权与回归范围;编译时间、内存和
|
||||
运行速度的提升需要同环境 benchmark,不能仅凭目录拆分推断。
|
||||
+13
-1
@@ -4,7 +4,19 @@ Rust crates for DBX live here.
|
||||
|
||||
## Directories
|
||||
|
||||
- `dbx-core/` - shared database core, drivers, schema/query logic, import/export, transfer, and plugin support.
|
||||
- `dbx-core/` - application orchestration: connections, queries, schema operations, data workflows, AI tools, administration, persistence, safety, and host services.
|
||||
- `dbx-types/` - shared connection, query, metadata, and serialization contracts; generated database identities.
|
||||
- `dbx-sql/` - SQL parsing, dialects, analysis, risk classification, DDL/DML generation, and schema diff planning.
|
||||
- `dbx-drivers/` - native database adapters, database Agents, tunnels, execution budgets, and driver lifecycle.
|
||||
- `dbx-formats/` - data formatting and file encoders, independent of database connections.
|
||||
- `dbx-ai-provider/` - AI provider clients, CLI adapters, streaming, and token usage.
|
||||
- `dbx-plugin-runtime/` - plugin packages, signatures, marketplace, subprocess sessions, and host requests.
|
||||
- `dbx-platform/` - shared process, path, proxy, download, version, and host-prompt primitives.
|
||||
- `dbx-web/` - the Docker/web backend service binary published as `dbx-web`.
|
||||
- `dbx-cli/` - the command-line application.
|
||||
- `dbx-mcp/` - the MCP service library and binary.
|
||||
- `dbx-sqlite-worker/` - the isolated SQLite file-host worker and protocol.
|
||||
|
||||
The workspace root is defined in the repository-level `Cargo.toml`.
|
||||
|
||||
See [ARCHITECTURE.md](ARCHITECTURE.md) for dependency rules, compatibility exports, feature selection, and validation commands.
|
||||
@@ -0,0 +1,23 @@
|
||||
[package]
|
||||
name = "dbx-ai-provider"
|
||||
version = "0.1.0"
|
||||
edition = "2021"
|
||||
license = "Apache-2.0"
|
||||
publish = false
|
||||
|
||||
[dependencies]
|
||||
dbx-platform = { path = "../dbx-platform" }
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
serde_json = { version = "1.0", features = ["arbitrary_precision", "preserve_order"] }
|
||||
futures = "0.3"
|
||||
reqwest = { version = "0.12", default-features = false, features = ["json", "stream", "rustls-tls", "socks"] }
|
||||
tokio = { version = "1", features = ["process", "io-util", "sync", "time", "rt", "macros"] }
|
||||
log = "0.4"
|
||||
uuid = { version = "1", features = ["v4", "serde"] }
|
||||
base64 = "0.22"
|
||||
bytes = "1"
|
||||
httpdate = "1"
|
||||
tempfile = "3"
|
||||
|
||||
[dev-dependencies]
|
||||
tokio = { version = "1", features = ["test-util", "rt-multi-thread"] }
|
||||
File renamed without changes.
File renamed without changes.
@@ -2132,7 +2132,7 @@ async fn ollama_model_completion_support(
|
||||
Ok(crate::ai_model_filter::ollama_completion_capability(&data))
|
||||
}
|
||||
|
||||
pub(crate) async fn ollama_selected_model_tool_support(config: &AiConfig) -> Result<Option<bool>, String> {
|
||||
pub async fn ollama_selected_model_tool_support(config: &AiConfig) -> Result<Option<bool>, String> {
|
||||
let model_id = config.model.trim();
|
||||
if model_id.is_empty() {
|
||||
return Err("Model is required".to_string());
|
||||
@@ -2724,7 +2724,7 @@ fn truncate_diagnostic(value: &str, max_chars: usize) -> String {
|
||||
/// its scheme because proxies often echo that token without `Bearer`/`Basic`.
|
||||
/// Other custom header values retain the [`MIN_SECRET_CHARS`] guard so ordinary
|
||||
/// diagnostics are not mangled by short values such as a version marker.
|
||||
pub(crate) fn sensitive_values(config: &AiConfig) -> Vec<String> {
|
||||
pub fn sensitive_values(config: &AiConfig) -> Vec<String> {
|
||||
let mut secrets: Vec<String> = Vec::new();
|
||||
push_secret(&mut secrets, normalized_api_key(config), true);
|
||||
for (name, value) in &config.custom_headers {
|
||||
@@ -2771,7 +2771,7 @@ fn push_secret(secrets: &mut Vec<String>, value: &str, allow_short: bool) {
|
||||
/// body that echoes `?key=…` cannot leak either. This is the single gate every
|
||||
/// provider error string must pass through before it reaches the UI, a log
|
||||
/// target, or a shared report; callers truncate the result as needed.
|
||||
pub(crate) fn redact_secrets(value: &str, secrets: &[String]) -> String {
|
||||
pub fn redact_secrets(value: &str, secrets: &[String]) -> String {
|
||||
let mut redacted = value.to_string();
|
||||
for secret in secrets {
|
||||
if redacted.contains(secret.as_str()) {
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -134,7 +134,7 @@ async fn codex_process_env(config: &AiConfig, command: &CodexCommandSpec) -> Res
|
||||
config,
|
||||
command,
|
||||
node_runtime_dir.as_deref(),
|
||||
crate::update::system_proxy_url().as_deref(),
|
||||
dbx_platform::proxy::system_proxy_url().as_deref(),
|
||||
&inherited_env_keys,
|
||||
)
|
||||
}
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,18 @@
|
||||
#![recursion_limit = "256"]
|
||||
|
||||
pub use dbx_platform::{path_utils, process};
|
||||
|
||||
pub mod agent_events;
|
||||
pub mod ai;
|
||||
pub mod ai_claude_code_cli;
|
||||
pub mod ai_cli_agent;
|
||||
pub mod ai_codebuddy_cli;
|
||||
pub mod ai_codex_cli;
|
||||
pub mod ai_cursor_cli;
|
||||
pub mod ai_effort;
|
||||
pub mod ai_grok_cli;
|
||||
mod ai_model_filter;
|
||||
pub mod ai_opencode_cli;
|
||||
pub mod ai_pi_agent_cli;
|
||||
pub mod ai_qoder_cli;
|
||||
pub mod token_usage;
|
||||
File renamed without changes.
+21
-42
@@ -6,66 +6,51 @@ license = "Apache-2.0"
|
||||
|
||||
[features]
|
||||
default = ["duckdb-sidecar", "dynamodb", "mq-admin", "sqlite-sqlcipher", "system-fonts"]
|
||||
duckdb-sidecar = []
|
||||
dynamodb = ["dep:aws-sdk-dynamodb"]
|
||||
mq-admin = ["dep:rumqttc"]
|
||||
sqlite-sqlcipher = ["rusqlite/bundled-sqlcipher-vendored-openssl"]
|
||||
sqlite-multiple-ciphers = ["dep:libsqlite3-hotbundle"]
|
||||
duckdb-sidecar = ["dbx-sql/duckdb-sidecar", "dbx-drivers/duckdb-sidecar"]
|
||||
dynamodb = ["dbx-drivers/dynamodb"]
|
||||
mq-admin = ["dep:rumqttc", "dbx-types/mq-admin", "dbx-drivers/mq-admin"]
|
||||
sqlite-sqlcipher = ["rusqlite/bundled-sqlcipher-vendored-openssl", "dbx-drivers/sqlite-sqlcipher"]
|
||||
sqlite-multiple-ciphers = ["dbx-drivers/sqlite-multiple-ciphers"]
|
||||
# Keeps bundled SQLite for consumers that enable no other sqlite source
|
||||
# (dbx-cli, dbx-mcp); do not combine with the cipher features.
|
||||
sqlite-bundled = ["rusqlite/bundled"]
|
||||
openapi = ["dep:utoipa"]
|
||||
sqlite-bundled = ["rusqlite/bundled", "dbx-drivers/sqlite-bundled"]
|
||||
openapi = ["dbx-types/openapi", "dbx-sql/openapi"]
|
||||
system-fonts = ["font-kit"]
|
||||
|
||||
[dependencies]
|
||||
dbx-drivers = { path = "../dbx-drivers", default-features = false }
|
||||
dbx-sql = { path = "../dbx-sql", default-features = false }
|
||||
dbx-plugin-runtime = { path = "../dbx-plugin-runtime" }
|
||||
dbx-formats = { path = "../dbx-formats" }
|
||||
dbx-types = { path = "../dbx-types", default-features = false }
|
||||
dbx-ai-provider = { path = "../dbx-ai-provider" }
|
||||
dbx-platform = { path = "../dbx-platform", features = ["host-prompts", "downloads"] }
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
serde_json = { version = "1.0", features = ["arbitrary_precision", "preserve_order"] }
|
||||
semver = "1"
|
||||
ed25519-dalek = "2"
|
||||
fs2 = "0.4"
|
||||
utoipa = { version = "5.5", optional = true }
|
||||
unicode-width = "0.2"
|
||||
unicode-ident = "1"
|
||||
serde_yaml_ng = "0.10"
|
||||
json5 = "0.4"
|
||||
regex = "1"
|
||||
winnow = "1.0"
|
||||
rayon = "1"
|
||||
percent-encoding = "2"
|
||||
encoding_rs = "0.8"
|
||||
log = "0.4"
|
||||
tracing = "0.1"
|
||||
tracing-subscriber = "0.3"
|
||||
tokio = { version = "1", features = ["full"] }
|
||||
tokio-util = { version = "0.7", features = ["compat"] }
|
||||
chrono = { version = "0.4", features = ["serde"] }
|
||||
httpdate = "1"
|
||||
rust_decimal = { version = "1", features = ["serde"] }
|
||||
anyhow = "1"
|
||||
uuid = { version = "1", features = ["v4", "serde"] }
|
||||
tokio-postgres = { version = "0.7", features = ["with-chrono-0_4", "with-uuid-1", "with-serde_json-1"] }
|
||||
deadpool-postgres = { version = "0.14", features = ["rt_tokio_1"] }
|
||||
tokio-postgres-rustls = "0.13"
|
||||
postgres-openssl = "0.5"
|
||||
openssl = { version = "0.10", features = ["vendored"] }
|
||||
webpki-roots = "0.26"
|
||||
rusqlite = { version = "0.32", default-features = false, features = ["load_extension", "backup", "functions", "column_decltype"] }
|
||||
libsqlite3-hotbundle = { git = "https://github.com/zipg/libsqlite3-hotbundle.git", rev = "7f12f203b4f4694a273410bd9b02d77f82b6114e", features = ["cipher-sqlcipher"], optional = true }
|
||||
dbx-sqlite-worker = { path = "../dbx-sqlite-worker", default-features = false }
|
||||
mysql_async = { version = "0.37", default-features = false, features = ["default-rustls", "client_ed25519", "chrono", "rust_decimal"] }
|
||||
sqlparser = { version = "0.62.0", features = ["visitor"] }
|
||||
redis = { version = "0.32.2", features = ["tokio-comp", "tls-rustls", "tls-rustls-insecure", "tokio-rustls-comp", "sentinel", "cluster-async"] }
|
||||
rustls = { version = "0.23", features = ["aws-lc-rs"] }
|
||||
rustls-pemfile = "2.2"
|
||||
tiberius = { version = "0.12.3", default-features = false, features = ["tds73", "chrono", "rust_decimal", "rustls", "sql-browser-tokio"] }
|
||||
reqwest = { version = "0.12", default-features = false, features = ["cookies", "json", "stream", "rustls-tls", "socks"] }
|
||||
prometheus-parse = "=0.2.5"
|
||||
futures = "0.3"
|
||||
mongodb = { version = "3.2.5", features = ["socks5-proxy"] }
|
||||
aws-sdk-dynamodb = { version = "=1.104.0", optional = true }
|
||||
rumqttc = { version = "0.24", features = ["websocket"], optional = true }
|
||||
russh = "0.60"
|
||||
pageant = "0.2"
|
||||
portpicker = "0.1.1"
|
||||
csv = "1"
|
||||
calamine = { version = "0.30.1", features = ["dates"] }
|
||||
@@ -86,27 +71,21 @@ flate2 = "1"
|
||||
crc = "3"
|
||||
sha1 = "0.10"
|
||||
tar = "0.4"
|
||||
zstd = "0.13"
|
||||
sysinfo = { version = "0.32", features = ["system"] }
|
||||
serde_yaml = "0.9"
|
||||
minijinja = "2"
|
||||
notify = { version = "7", default-features = false, features = ["macos_kqueue"] }
|
||||
arc-swap = "1"
|
||||
insta = { version = "1", features = ["json", "glob"] }
|
||||
tempfile = "3"
|
||||
|
||||
[target.'cfg(windows)'.dependencies]
|
||||
windows-sys = { version = "0.61", features = ["Win32_Storage_FileSystem", "Win32_Foundation", "Win32_Security"] }
|
||||
|
||||
[dev-dependencies]
|
||||
zstd = "0.13"
|
||||
dbx-drivers = { path = "../dbx-drivers", default-features = false, features = ["test-support"] }
|
||||
dbx-plugin-runtime = { path = "../dbx-plugin-runtime", features = ["test-support"] }
|
||||
dbx-platform = { path = "../dbx-platform", features = ["test-support"] }
|
||||
tokio = { version = "1", features = ["test-util"] }
|
||||
|
||||
[target.'cfg(all(unix, not(target_os = "redox")))'.dependencies]
|
||||
nix = { version = "0.31.3", features = ["user"] }
|
||||
|
||||
[dev-dependencies]
|
||||
mysql_common = { git = "https://github.com/t8y2/rust_mysql_common.git", rev = "47740d85a8277167a5717afac785d37b46a36296", default-features = false }
|
||||
tokio = { version = "1", features = ["test-util"] }
|
||||
tokio-openssl = "0.6"
|
||||
|
||||
[build-dependencies]
|
||||
serde = { version = "1.0", features = ["derive"] }
|
||||
serde_json = "1.0"
|
||||
serde_yaml_ng = "0.10"
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,6 @@
|
||||
pub mod consul;
|
||||
#[cfg(feature = "mq-admin")]
|
||||
pub mod mq;
|
||||
#[cfg(feature = "mq-admin")]
|
||||
pub mod mqtt;
|
||||
pub mod nacos;
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -8,10 +8,10 @@
|
||||
//! src-tauri/src/commands/mqtt_cmd.rs (Tauri command 入口)
|
||||
//! │
|
||||
//! ▼
|
||||
//! crates/dbx-core/src/mqtt/service.rs (共享核心逻辑)
|
||||
//! crates/dbx-core/src/admin/mqtt/service.rs (共享核心逻辑)
|
||||
//! │
|
||||
//! ▼
|
||||
//! crates/dbx-core/src/mqtt/client.rs (rumqttc 客户端封装)
|
||||
//! crates/dbx-core/src/admin/mqtt/client.rs (rumqttc 客户端封装)
|
||||
//! │
|
||||
//! ▼
|
||||
//! MQTT Broker (EMQX / Mosquitto / HiveMQ / ...)
|
||||
@@ -19,4 +19,4 @@
|
||||
|
||||
pub mod client;
|
||||
pub mod service;
|
||||
pub mod types;
|
||||
pub use dbx_types::mqtt as types;
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,8 @@
|
||||
pub use dbx_ai_provider::ai::*;
|
||||
|
||||
pub mod agent_explain;
|
||||
pub mod agent_kv;
|
||||
pub mod agent_loop;
|
||||
pub mod agent_tools;
|
||||
pub mod mcp_policy;
|
||||
pub mod prompt_template;
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -1,3 +1,10 @@
|
||||
pub mod connection_secrets;
|
||||
pub mod driver_runtime;
|
||||
pub mod jdbc;
|
||||
pub mod runtime_config;
|
||||
pub mod session_credentials;
|
||||
pub mod task_supervisor;
|
||||
|
||||
use std::collections::{HashMap, HashSet};
|
||||
use std::path::PathBuf;
|
||||
use std::sync::atomic::{AtomicBool, AtomicUsize, Ordering};
|
||||
File renamed without changes.
File renamed without changes.
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,7 @@
|
||||
pub use dbx_drivers::db::cloudflare_d1::*;
|
||||
|
||||
mod import;
|
||||
#[cfg(test)]
|
||||
mod transfer_tests;
|
||||
|
||||
pub(crate) use import::{build_import_insert_batches, build_streaming_import_insert_batch};
|
||||
File renamed without changes.
File renamed without changes.
@@ -0,0 +1,111 @@
|
||||
use serde::{Deserialize, Serialize};
|
||||
use std::fs::File;
|
||||
use std::io::{BufWriter, Write};
|
||||
|
||||
use crate::connection::AppState;
|
||||
use crate::models::connection::DatabaseType;
|
||||
use crate::query::{execute_sql_statement_with_options, QueryExecutionOptions};
|
||||
use crate::sql_dialect::{build_table_data_select_sql, TableDataSelectSqlOptions};
|
||||
|
||||
pub use dbx_formats::csv_export::*;
|
||||
|
||||
const TABLE_DATA_EXPORT_PAGE_SIZE: usize = 10_000;
|
||||
|
||||
#[derive(Debug, Clone, Serialize, Deserialize)]
|
||||
#[serde(rename_all = "camelCase")]
|
||||
pub struct TableCsvExportOptions {
|
||||
pub file_path: String,
|
||||
pub connection_id: String,
|
||||
pub database: String,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub schema: Option<String>,
|
||||
pub table_name: String,
|
||||
#[serde(default)]
|
||||
pub columns: Vec<String>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub page_size: Option<usize>,
|
||||
#[serde(default, skip_serializing_if = "Option::is_none")]
|
||||
pub timeout_secs: Option<u64>,
|
||||
#[serde(default)]
|
||||
pub csv_quote_mode: CsvQuoteMode,
|
||||
}
|
||||
|
||||
async fn connection_database_type(state: &AppState, connection_id: &str) -> Result<DatabaseType, String> {
|
||||
state
|
||||
.configs
|
||||
.read()
|
||||
.await
|
||||
.get(connection_id)
|
||||
.map(|config| config.db_type)
|
||||
.ok_or_else(|| format!("Connection config not found: {connection_id}"))
|
||||
}
|
||||
|
||||
pub async fn export_table_data_csv_core(state: &AppState, options: TableCsvExportOptions) -> Result<u64, String> {
|
||||
let database_type = connection_database_type(state, &options.connection_id).await?;
|
||||
let page_size = options.page_size.unwrap_or(TABLE_DATA_EXPORT_PAGE_SIZE).max(1);
|
||||
let mut writer =
|
||||
BufWriter::new(File::create(&options.file_path).map_err(|err| format!("Failed to write CSV file: {err}"))?);
|
||||
writer.write_all("\u{FEFF}".as_bytes()).map_err(|err| err.to_string())?;
|
||||
|
||||
let mut offset = 0usize;
|
||||
let mut rows_exported = 0u64;
|
||||
let mut wrote_header = false;
|
||||
|
||||
loop {
|
||||
let sql = build_table_data_select_sql(TableDataSelectSqlOptions {
|
||||
database_type: Some(database_type),
|
||||
schema: options.schema.clone(),
|
||||
table_name: options.table_name.clone(),
|
||||
table_type: None,
|
||||
primary_keys: Vec::new(),
|
||||
columns: options.columns.clone(),
|
||||
fallback_order_columns: Vec::new(),
|
||||
order_by: None,
|
||||
limit: Some(page_size),
|
||||
offset: Some(offset),
|
||||
where_input: None,
|
||||
include_row_id: false,
|
||||
..Default::default()
|
||||
});
|
||||
let result = execute_sql_statement_with_options(
|
||||
state,
|
||||
&options.connection_id,
|
||||
&options.database,
|
||||
&sql,
|
||||
options.schema.as_deref(),
|
||||
None,
|
||||
QueryExecutionOptions {
|
||||
max_rows: Some(page_size),
|
||||
timeout_secs: options.timeout_secs,
|
||||
..Default::default()
|
||||
},
|
||||
)
|
||||
.await?;
|
||||
|
||||
if !wrote_header {
|
||||
write_csv_text_row(&mut writer, result.columns, options.csv_quote_mode)?;
|
||||
wrote_header = true;
|
||||
}
|
||||
|
||||
let fetched = result.rows.len();
|
||||
if fetched == 0 {
|
||||
break;
|
||||
}
|
||||
for row in result.rows {
|
||||
writer.write_all(b"\n").map_err(|err| err.to_string())?;
|
||||
write_csv_value_row(&mut writer, row, options.csv_quote_mode)?;
|
||||
}
|
||||
|
||||
rows_exported += fetched as u64;
|
||||
if fetched < page_size {
|
||||
break;
|
||||
}
|
||||
offset += fetched;
|
||||
}
|
||||
|
||||
if rows_exported == 0 {
|
||||
writer.write_all(b"\n").map_err(|err| err.to_string())?;
|
||||
}
|
||||
writer.flush().map_err(|err| err.to_string())?;
|
||||
Ok(rows_exported)
|
||||
}
|
||||
File renamed without changes.
File renamed without changes.
Loaded 100 of 402 files, more files were not shown because too many files have changed in this diff.
Show more
Reference in new issue
Block a user