12 KiB
Data Transfer: Tree Object Selection and Transfer Modes Design
Date: 2026-08-01
Summary
The data transfer dialog (apps/desktop/src/components/transfer/DataTransferDialog.vue) currently lets the user pick tables from a flat checkbox list and copy table data between connections. Non-table objects (views, stored procedures, functions, triggers, sequences, events) are only copied for PostgreSQL→PostgreSQL transfers, and then only as an all-or-nothing schema sweep performed after the table data copy.
This change turns the table picker into a tree that groups selectable objects by type (tables, views, stored procedures, functions, triggers, sequences, events), and adds an explicit transfer mode: structure and data (default), structure only, and data only. Non-table object DDL copying is implemented for PostgreSQL and PG-family databases (Kingbase, GaussDB, KWDB, OpenGauss), MySQL, Dameng, and Oracle. Cross-database-family object DDL copy is out of scope for the first version.
Goals
- Select transfer content via a tree grouped by object type, with per-group select-all and object-level checkboxes.
- Support three transfer modes: structure and data, structure only, data only.
- Copy non-table object DDL for PostgreSQL/PG-family, MySQL, Dameng, and Oracle, within the same database family.
- Skip objects that already exist on the target side (by name), and report the skip count on completion.
- Preserve the existing table data transfer behavior (append/overwrite/upsert write modes, FK dependency ordering, ownership preview, target-table-name case handling).
Non-goals
- Cross-family non-table object DDL transfer (e.g., MySQL view → PostgreSQL view). The tree disables non-table groups with a hint in that case.
- Object transfer for databases outside the supported family set (SQLite, ClickHouse, SQL Server, MongoDB, Hive, Spark, etc.).
- Editing object DDL inside the transfer dialog (users use the object source editor for that).
- Differential/sync behavior beyond "skip existing" (no replacement of changed objects).
Existing Design to Reuse
crates/dbx-core/src/transfer.rs— table data transfer core:TransferRequest,transfer_table, FK dependency sorting (sort_tables_by_fk_dependency), progress reporting (TransferProgress), DDL generation (generate_create_table_ddl), column type mapping (map_column_type).crates/dbx-core/src/transfer.rs:5020—transfer_postgres_schema_objects: existing PostgreSQL schema object DDL copy (views, materialized views, procedures, functions, sequences, triggers, types, packages, synonyms + ownership/grants/policies). Reused and extended with per-object selection filtering.crates/dbx-core/src/object_source_sql.rs—build_executable_object_source_statements,ObjectSourceKind,EditableObjectSourceSqlInput: per-database object source DDL machinery (already used by the PG object transfer).crates/dbx-web/src/routes/transfer.rs— transfer orchestration route, progress channel, per-object failure handling.apps/desktop/src/lib/backend/api.ts—listObjects(connectionId, database, schema, objectTypes?, filter?, limit?, offset?, catalog?)returningObjectInfo[];TransferRequesttypes.apps/desktop/src/lib/database/databaseObjectCapabilities.ts— per-database sidebar object kind lists.
Design
1. Backend protocol (crates/dbx-core/src/transfer.rs, crates/dbx-web/src/routes/transfer.rs)
Extend TransferRequest:
#[serde(rename_all = "camelCase")]
pub enum TransferContent {
#[default]
StructureAndData,
StructureOnly,
DataOnly,
}
#[serde(rename_all = "SCREAMING_SNAKE_CASE")] // matches SidebarObjectKind conventions
pub enum TransferObjectKind {
Table, View, MaterializedView, Procedure, Function, Trigger, Sequence, Event,
}
pub struct TransferObjectSelection {
pub object_type: TransferObjectKind,
pub names: Vec<String>,
}
New fields on TransferRequest:
content: TransferContent— defaults toStructureAndData. Replaces thecreate_tableboolean (create_table == (content != DataOnly)); the field is kept for wire compatibility but the frontend stops sending it.objects: Vec<TransferObjectSelection>— non-table object selections. The table selection stays in the existingtables: Vec<String>field (also mirrored as aTransferObjectKind::Tableentry when the tree sends it).
Validation (in the route, before transfer starts):
DataOnlywith non-emptyobjects→ error "仅数据模式不传输非表对象".StructureOnlywith emptytablesand emptyobjects→ error (nothing to do).- Non-table objects are rejected when source and target database families differ (see family rules below).
- Non-table object names are validated like table names (non-empty, no NUL bytes).
TransferMode (append/overwrite/upsert) is unchanged; it only takes effect when the content includes data (StructureAndData or DataOnly).
2. Database family rules
| Family | Database types | Non-table object DDL support |
|---|---|---|
| mysql | mysql | yes (views, procedures, functions, triggers, events) |
| postgres | postgres, kingbase, gaussdb, kwdb, opengauss | yes |
| oracle | oracle, dameng, oceanbase-oracle | yes (views, procedures, functions, triggers, sequences) |
| other | everything else | no — tree groups disabled, backend rejects |
is_same_transfer_family(source, target) — a new pure function in transfer.rs, unit-tested. Cross-family transfers keep table data copy working exactly as today, with non-table groups disabled in the UI.
Object type availability per family (drives which tree groups render):
| Object kind | mysql | postgres family | oracle/dameng |
|---|---|---|---|
| TABLE | ✓ | ✓ | ✓ |
| VIEW | ✓ | ✓ | ✓ |
| MATERIALIZED_VIEW | ✗ | ✓ | ✓ |
| PROCEDURE | ✓ | ✓ | ✓ |
| FUNCTION | ✓ | ✓ | ✓ |
| TRIGGER | ✓ | ✓ | ✓ |
| SEQUENCE | ✗ (MariaDB only, first version: no) | ✓ | ✓ |
| EVENT | ✓ | ✗ | ✗ |
3. Object listing (backend)
Extend the existing listObjects path (per-database object listing in crates/dbx-core/src/schema.rs and the agent-side listing) so the transfer tree can fetch objects grouped by type:
- PostgreSQL family: already supported kinds (TABLE, VIEW, MATERIALIZED_VIEW, PROCEDURE, FUNCTION, SEQUENCE) via
pg_catalog; add TRIGGER listing (pg_triggerjoinpg_class/pg_namespace, names of triggers in the schema). - MySQL: add TRIGGER (
information_schema.TRIGGERSfiltered byEVENT_OBJECT_SCHEMA) and EVENT (information_schema.EVENTS); VIEW/PROCEDURE/FUNCTION listing already exists. - Oracle/Dameng: add TRIGGER and SEQUENCE (
ALL_OBJECTSfiltered byOBJECT_TYPE); VIEW/PROCEDURE/FUNCTION listing already exists (Dameng already hasSYS.SYSOBJECTS-based listing).
The Event kind is transfer-only: it is added to the backend ObjectSourceKind/listing mapping but not to the sidebar object capability lists (the sidebar does not gain new node types in this change).
4. Non-table object DDL copy (backend)
New transfer_objects flow in crates/dbx-core/src/transfer.rs, invoked from the route after table data transfer and after the existing PG schema-dependency step. Execution order:
- tables (existing flow, FK sorted)
- sequences
- views, materialized views
- functions
- procedures
- triggers
- events
DDL fetching per family:
- PostgreSQL family: reuse
get_postgres_schema_object_sources_for_transfer+get_postgres_materialized_view_sources_for_transfer+ newget_postgres_trigger_sources_for_transfer-style queries, filtered by the selected names. Existing schema rewrite (rewrite_postgres_routine_schema) andbuild_executable_object_source_statementsapply. The current "transfer everything in the schema" behavior becomes "transfer only selected objects"; when the request carries no object selection the step is skipped entirely (no behavior surprise for table-only transfers). - MySQL: VIEW/PROCEDURE/FUNCTION DDL via
SHOW CREATE(existingmysql_object_source_sql); TRIGGER DDL assembled frominformation_schema.TRIGGERS(CREATE TRIGGERwith ACTION_TIMING/EVENT_MANIPULATION/EVENT_OBJECT_TABLE/ACTION_STATEMENT — the per-table trigger query atcrates/dbx-core/src/db/mysql.rs:4239is the basis); EVENT DDL assembled frominformation_schema.EVENTS; stripDEFINER=clauses; rewrite the schema qualifier from source schema to target schema when they differ. - Oracle/Dameng: fetch DDL via
DBMS_METADATA.GET_DDL('<TYPE>', '<name>')(the existing object-source machinery already implements per-type metadata DDL for these databases; reuse those code paths where possible); rewrite schema qualifiers; execute per statement.
Target-side existence check, per object, immediately before executing its DDL:
- PostgreSQL family:
SELECT 1 FROM pg_class/pg_proc ...by schema+name. - MySQL:
information_schemalookups (VIEWS, ROUTINES, TRIGGERS, EVENTS). - Oracle/Dameng:
ALL_OBJECTS(orSYS.SYSOBJECTSfor Dameng) by schema+name.
Existing objects are skipped and counted; the count is included in the final summary progress message ("skipped N existing objects"). Failed objects follow the existing pattern: recorded in the failure list, transfer continues, final status reflects the failures.
Progress: reuse TransferProgress with table = "schema object: <name>" (already the convention used by the PG object transfer). The DataOnly mode skips the whole object-DDL step.
5. Frontend (apps/desktop/src/components/transfer/DataTransferDialog.vue)
- Replace the flat table list with a tree:
- Top-level groups rendered from the source database family capability table (tables, views, materialized views, procedures, functions, triggers, sequences, events as applicable).
- Group header checkbox = select all / clear group; per-object checkboxes; expand/collapse per group; a search filter box filters object names across groups.
- Groups that cannot be transferred (cross-family, unsupported kind, or
DataOnlymode) are rendered disabled with a tooltip explaining why. - Table selection stays in
tables; non-table selections serialize intoobjects.
- Transfer mode selector: 结构和数据 / 仅结构 / 仅数据 (i18n keys in the
transfer.*namespace).DataOnly: non-table groups disabled ("仅数据模式不传输非表对象"); table group enabled.StructureOnly: data write mode (append/overwrite/upsert) selector hidden/disabled; ownership preview and target-table-name case options only apply when data is copied (ownership preview only relevant withStructureAndData; keep the existing behavior for that mode).- The
createTablecheckbox is removed.
- Load object lists per group from
listObjectswith the per-kind object type filter; loading states per group; error states surface inline.
6. Error handling
- Validation errors from the route are shown inline in the dialog (existing error display pattern).
- Per-object DDL failures: existing
failed_tablesmechanism extended to carry object entries; final statusErrorwith the failure list, transfer continues for remaining objects. - Cancellation: existing
transfer_idcancellation checks are threaded through the object-DDL loop (mirroring the PG object transfer loop).
7. Testing
Backend unit tests (crates/dbx-core/src/transfer.rs test module + crates/dbx-core/tests/live_postgres_transfer.rs):
is_same_transfer_familymatrix.- Request validation (DataOnly + objects, cross-family objects).
- Per-family object DDL statement generation (MySQL
SHOW CREATEhandling with DEFINER stripping; schema rewriting). - Target existence check SQL per family.
- Live PG transfer: object filtering by selection, skip-existing behavior,
StructureOnly(no data rows copied),DataOnly(no DDL executed).
Frontend tests (apps/desktop/src/components/transfer/__tests__/):
- Tree group rendering per database family.
- Group select-all, object toggle, search filtering.
- Disable rules:
DataOnlymode, cross-family, unsupported kinds. - Request serialization (
tables+objects+content).