# Database Documentation and DBML Export Design Date: 2026-08-02 ## Summary DBX can inspect a schema but cannot document it. There is no way to produce a browsable, shareable description of a database, and no occurrence of DBML anywhere in the repository. This change adds a **documentation subsystem** built around one idea: `SchemaSnapshot` is the product, and every output is a serializer over it. `crates/dbx-core/src/docs/` collects a normalized snapshot of a relational schema, merges it with user-authored notes and table groups, and serializes it to two artifacts — a `.dbml` interchange file and a self-contained HTML documentation site in the style of [dbdocs.io](https://dbdocs.io/Holistics/Ecommerce). The documentation UI is a **standalone Vue application that makes no backend calls**. The same component tree renders inside DBX as a dialog and inside an exported single-file HTML page. Generation is available headless through `dbx docs` and `dbx dbml`, so documentation can be regenerated in CI and schema drift can fail a build. Schema versioning (a Changelog tab and per-table "Last Updated") is explicitly deferred, but the snapshot is version-stamped and storable from day one so that adding it later is additive rather than a refactor. ## Goals - Collect a normalized `SchemaSnapshot` for any relational engine DBX already supports, reusing the existing `schema::providers` metadata layer. - Serialize the snapshot to valid DBML, including `TableGroup` blocks, `Enum` blocks, and `Ref` cardinality. - Render dbdocs-style documentation: project overview, per-group table index with inline markdown notes, per-table pages with columns/indexes/bidirectional relationships, an ER diagram with group containers, and client-side search. - Let users author table notes, column notes, project notes, and **table groups** in DBX, persisted in a human-readable, git-committable file. - Export a self-contained HTML file that works over `file://` with no network access. - Provide `dbx docs` and `dbx dbml` CLI verbs producing byte-identical output to the desktop app. - Degrade visibly, never silently, when an engine cannot supply metadata. ## Non-goals - **Schema versioning / Changelog / "Last Updated"** — deferred. Live databases mostly cannot report a DDL timestamp (PostgreSQL has none; MySQL's `information_schema.tables.update_time` is NULL for most InnoDB tables), so this is a derivative of stored snapshot history, not of metadata. Section 12 defines the seam. - **DBML import / parsing.** The serializer is one-way. A `.dbml` file is an output, never a source of truth. - **Non-relational engines** (MongoDB, Redis, Elasticsearch, InfluxDB, HBase). DBML has no vocabulary for them and schema inference from documents is a separate feature. - **Non-table objects** — triggers, procedures, functions, sequences, packages, grants, partitions. `ObjectInfo` exists for these but none map to DBML and dbdocs does not document them. `format_version` makes adding them additive. - **Write-back to the database.** DBX never issues `COMMENT ON` as a side effect of documentation. - **Publishing/hosting.** DBX writes a file; distributing it is the user's business. ## Existing Design to Reuse - `crates/dbx-core/src/types.rs:57-141` — `TableInfo`, `ColumnInfo`, `IndexInfo`, `ForeignKeyInfo`. These are serde mirrors of the TS interfaces in `apps/desktop/src/types/database.ts` and are **composed directly** into the snapshot rather than re-declared. - `crates/dbx-core/src/schema/providers/native.rs` — `list_databases`, `list_schemas`, `list_tables`, `get_columns`, `list_foreign_keys`, `list_indexes`, already dispatching across every supported engine. - `crates/dbx-core/src/schema/normalization.rs` — identifier normalization; the home for per-engine case folding of annotation keys. - `crates/dbx-core/src/table_structure_sql/dialect.rs` — `capabilities_for(database_type).comment`, the authority on whether an engine supports comments at all. - `crates/dbx-core/src/schema_diff.rs:772` — `RenameCandidate { source_name, target_name, score, column_jaccard, type_similarity }` plus `jaccard_similarity` and `column_type_similarity`. Reused to *suggest* re-mapping orphaned notes after a rename. - `crates/dbx-core/src/database_export.rs` — the established progress-callback and cancellation shape. - `apps/desktop/src/lib/diagram/erDiagram.ts` — `layoutDiagramTables` and `diagramZoom.ts`. This module imports **only types**, so the standalone docs bundle can import it without pulling in DBX runtime code. `buildDiagramRelationships` / `foreignKeySourceCardinality` are the reference implementation for the Rust port in Section 3. - `apps/desktop/src/lib/export/diagramSvgExport.ts` — existing SVG serialization for diagram export. - `apps/desktop/src/styles/globals.css:1310+` — the DBX token palette (strictly achromatic, `oklch(… 0 0)`), from which the docs theme subset is derived. - `marked@18` and `shiki@4` are already dependencies. **No new dependency is required by this design.** ## Design ### 1. Snapshot model (`crates/dbx-core/src/docs/snapshot.rs`) ```rust use crate::types::{ColumnInfo, ForeignKeyInfo, IndexInfo}; #[derive(Serialize, Deserialize)] pub struct SchemaSnapshot { pub format_version: u32, // 1 pub project: ProjectMeta, pub tables: Vec, pub relationships: Vec, pub groups: Vec, pub enums: Vec, pub warnings: Vec, } pub struct ProjectMeta { pub name: String, pub database_type: String, pub database: Option, pub schemas: Vec, pub generated_at: String, // RFC3339 pub note: Option, // markdown, the landing page } pub struct DocTable { pub schema: Option, pub name: String, pub kind: TableKind, // Table | View | MaterializedView pub columns: Vec, pub indexes: Vec, pub foreign_keys: Vec, pub group_id: Option, pub note: Option, pub note_source: NoteSource, // Database | Local | None pub shadowed_note: Option, // db comment a local note overrides pub column_notes: BTreeMap, pub estimated_rows: Option, pub view_definition: Option, } pub struct TableGroup { pub id: String, // slug, stable across renames pub name: String, pub hue: u16, // 0-359 — see Section 4 pub note: Option, } pub struct Relationship { pub id: String, pub name: Option, pub from: FieldRef, pub to: FieldRef, pub cardinality: Cardinality, pub on_update: Option, pub on_delete: Option, } pub struct FieldRef { pub schema: Option, pub table: String, pub column: String } pub struct ColumnNote { pub note: String, pub source: NoteSource, pub shadowed: Option } pub struct DocEnum { pub schema: Option, pub name: String, pub values: Vec, pub note: Option, pub synthesized: bool } // true for MySQL inline ENUM(...) pub enum TableKind { Table, View, MaterializedView } pub enum NoteSource { Database, Local, None } pub enum Cardinality { OneToOne, ManyToOne } pub enum SnapshotWarning { TableSkipped { table: String, reason: String }, NoForeignKeyMetadata { engine: String }, CommentsUnsupported { engine: String }, OrphanedNotes { count: usize }, DbmlOmitted { table: String, item: String, reason: String }, } ``` Four decisions are embedded here: - **`format_version`** guards viewer/data skew. The exported HTML pins a snapshot beside a viewer built from the same commit, but an in-app viewer may load a snapshot produced by an older DBX. One integer prevents a class of blank-render bugs and makes future additions additive. - **`note_source` and `shadowed_note`** exist so the UI can show *where* prose came from. When a local note shadows a database comment, both are available; the original is shown on hover. Without this, a DBA later writing a better `COMMENT ON` would have their improvement silently hidden forever. - **`warnings`** is the honest-degradation channel (Section 10). - **Enums are first-class** because DBML has native `enum` blocks. `ColumnInfo.enum_values` is already populated for MySQL inline `ENUM(...)`; PostgreSQL named enums are collected from `pg_type`/`pg_enum`. MySQL inline enums are synthesized into a named enum per `table_column`. This is the one place the snapshot normalizes rather than passes through. ### 2. Collection (`crates/dbx-core/src/docs/collector.rs`) v1 fetches through the **existing** provider functions, fanned out with a bounded semaphore (8 concurrent). No new per-engine SQL. This is deliberately the naive-but-correct version: it works for every supported relational engine on day one because it reuses code already proven against all of them. A 400-table schema costs roughly 1200 queries — bounded, cancellable, and measurable. The optimization path is targeted rather than speculative: PostgreSQL and MySQL can collect all columns, FKs and indexes for a schema in three `information_schema` sweeps instead of 3×N. That belongs in a `collector::bulk` specialization behind the same interface, added only where a benchmark under `scripts/bench/` shows it matters. Progress is reported through an `impl Fn(DocsProgress)` callback and cancellation via a checked `AtomicBool`, matching `database_export.rs`. ### 3. Relationship inference (`crates/dbx-core/src/docs/relations.rs`) A port of the data half of `erDiagram.ts`. The split is deliberate: - **Data logic moves to Rust** — given foreign keys and unique indexes, is an edge 1:1 or N:1? This is the `foreignKeySourceCardinality` check: a relationship is `OneToOne` only when the source columns contain a unique key, otherwise `ManyToOne`. - **Layout stays in the renderer** — `layoutDiagramTables` is a view concern and remains in TypeScript, imported by the docs bundle. This keeps the port to roughly 100 lines rather than a rewrite of the diagram feature. `SchemaDiagramDialog.vue` is **not modified** by this change; once `relations.rs` is proven it becomes the obvious candidate to consume the same inference, removing the duplicate. ### 4. Annotations and table groups (`crates/dbx-core/src/docs/annotations.rs`) **The notes file is the store** — not a cache, not an export. `dbx.db` persists exactly one thing per connection: the path to this file. ```jsonc // default: /docs-notes/.json // overridable per connection to e.g. ./docs/dbx-docs.json inside a repository { "format_version": 1, "project": { "name": "Ecommerce", "note": "# Overview\n\nMarkdown." }, "groups": [ { "id": "order-management", "name": "Order Management", "hue": 28, "note": "Everything from checkout to carrier handoff." } ], "tables": { "core.orders": { "group": "order-management", "note": "One row per completed checkout.", "columns": { "status": { "note": "State machine; only moves forward." } } } } } ``` This shape is required by the CI goal. If notes lived only in `dbx.db`, `dbx docs` in CI would render documentation with every note missing — polished pages, empty of the prose that justifies them. As a file, notes are also reviewable in pull requests, which is arguably better than `COMMENT ON` (invisible to code review). `DesktopSettings.saved_sql_sync_dir` is the existing precedent for a user-chosen path. **Table groups** are cross-schema domain groupings, authored by the user, stored here alongside notes, and serialized to DBML `TableGroup` blocks. A table belongs to at most one group; unassigned tables render under `(no group)`. Grouping is documentation, not configuration — it never affects what is collected. **Group colour is stored as a hue integer (0–359), not a hex string.** Lightness and chroma are fixed per theme in CSS and only the hue varies per group: ```css .group { --h: 28; } /* light */ --group-c: oklch(0.55 0.15 var(--h)); --group-tint: oklch(0.965 0.022 var(--h)); /* dark */ --group-c: oklch(0.76 0.13 var(--h)); --group-tint: oklch(0.27 0.038 var(--h)); ``` Storing a hex colour the dbdocs way would force either an unreadable dark mode (pastels tuned for a white page turn to mud on `rgb(19 20 22)`) or two colours per group that the user must choose twice. One hue guarantees legible contrast on both grounds by construction, and converts to hex on DBML export. **Group editor UI.** Groups are created and edited in a dialog reached from the group header on the wiki index, the group badge on a table page, and the diagram group popover. Fields: name, note (markdown), colour, and member tables. The colour control is **12 curated swatches plus a full 0–359 hue slider** — every hue is reachable, but lightness and chroma remain theme-controlled, so no selection can produce an illegible group. The dialog renders a **two-ground preview**, showing the group chrome on a forced-light and a forced-dark tile simultaneously, because the user may be editing in one theme while a colleague reads the exported HTML in the other. A native hex/RGB picker is deliberately not offered. It would let a user choose a colour that disappears against one of the two grounds, and the only remedies are to render something different from what the picker showed, or to ship an unreadable group. Hue-plus-fixed-lightness makes the illegible state unrepresentable rather than merely discouraged. On save, the group is written to the notes file. Since colour is one integer, a group's full definition is a four-field object and diffs cleanly in review. **Keys and renames.** Keys are `schema.table` and `schema.table.column`, case-folded per engine (PostgreSQL lower, Oracle upper, MySQL per `lower_case_table_names`) in `schema/normalization.rs`. When a table or column is renamed its note orphans. **User prose is never silently deleted.** Orphans remain in the file, the collector emits `SnapshotWarning::OrphanedNotes { count }`, and the dialog shows an "Unmatched notes" panel. For each orphan, `schema_diff::RenameCandidate` is used to **suggest** the most likely new owner along with its confidence score; the user confirms, re-maps manually, or deletes. Suggestion plus confirmation, never automatic re-attachment — a wrong auto-guess would attach misleading documentation to the wrong column, which is worse than an honest orphan. **Merge precedence** is `local_annotation ?? database_comment`, recorded in `note_source`. On engines where `capabilities_for(db_type).comment == false` (SQLite, ClickHouse) the annotation layer is the only prose channel, and the UI states this once rather than offering an inert "read from database" affordance. ### 5. DBML serialization (`crates/dbx-core/src/docs/dbml.rs`) A hand-written serializer, roughly 250 lines. The `@dbml/core` npm package is unusable from Rust and the grammar is small enough not to warrant a dependency. ```dbml Project ecommerce { database_type: 'PostgreSQL' Note: '''Generated by DBX on 2026-08-02''' } Enum order_status { pending shipped cancelled } TableGroup order_management [color: #c2410c] { core.orders core.order_items Note: '''Everything from checkout to carrier handoff.''' } Table core.orders { id integer [pk, increment] user_id integer [not null, note: 'Owning customer'] status order_status [not null, default: 'pending'] placed_at timestamptz [default: `now()`] Indexes { (user_id, placed_at) [name: 'idx_orders_user_placed'] } Note: '''One row per completed checkout.''' } Ref fk_orders_user: core.orders.user_id > core.users.id [delete: restrict] ``` Rules: - **Types pass through verbatim.** DBML does not validate type names, so `timestamptz`, `jsonb` and `numeric(10,2)` survive. Where `data_type` arrives bare, precision is reconstructed from `character_maximum_length` / `numeric_precision` / `numeric_scale`. - **Cardinality comes from `relations.rs`**: `>` for many-to-one, `-` for one-to-one. - **Schema qualification is conditional** — bare names for a single-schema snapshot (it reads better), fully qualified for multi-schema, since DBML has no `search_path`. - **Escaping** — identifiers quoted when not `[A-Za-z_][A-Za-z0-9_]*`; notes always in `'''triple quotes'''` so apostrophes cannot break the file; expression defaults in backticks, literals in single quotes. - **Group colour** is converted from hue to hex at the theme-neutral lightness. **DBML is lossy and this is documented in the output.** It cannot express check constraints, partial or filtered indexes (`IndexInfo.filter`), included columns, collations, or generated columns. Composite foreign keys are emitted in the grouped `(a, b) > (c, d)` form, whose support across dbdocs/dbdiagram is uneven. Every omission produces a `SnapshotWarning`, and the HTML documentation renders the omitted item with a "not in DBML" marker. The asymmetry is intentional: **DBML is the interchange format; the HTML documentation is the complete record.** Treating DBML as a source of truth would silently drop constraints. ### 6. Docs viewer (`apps/desktop/src/docs/`) ``` main.ts — standalone entry; reads window.__DBX_DOCS__, mounts DocsApp.vue — root; hash routing components/ DocsShell.vue — sidebar + main DocsSidebar.vue — Group by: Schemas | Table Groups switch, trees, enums DocsSearch.vue — ⌘K palette, client-side fuzzy over tables/columns/groups WikiIndex.vue — stats, project note, per-group table index TablePage.vue — orchestrates the sections below ColumnTable.vue — columns grid; inline note editing when !readonly IndexList.vue RelationshipList.vue GroupHeader.vue EnumPage.vue GroupEditorDialog.vue — name/note/hue/members; rendered only when !readonly ErdView.vue — pan/zoom SVG with group containers; reuses layoutDiagramTables() NoteMarkdown.vue — marked → sanitized HTML WarningBanner.vue — renders snapshot.warnings __tests__/ — fixture snapshots in, rendered output asserted ``` **The contract: this directory makes no backend calls.** No `api.*`, no Tauri, no Pinia stores. Snapshot in, DOM out. This is what allows the same code to run inside an HTML file on a machine that has never installed DBX, and what makes the entire documentation UI testable without a database. Editing is expressed as a `readonly` prop — `true` in exported HTML, `false` in-app. Components emit `@update:note`, `@update:group` and `@update:table-group`; `DatabaseDocsDialog.vue` performs the writes. `GroupEditorDialog.vue` is not rendered at all when `readonly`, so the exported HTML never ships an editor that cannot save. Six further decisions: - **Hash routing** (`#/table/core.orders`), not the History API, which breaks over `file://`. Also makes every table deep-linkable. - **Raw HTML is stripped from markdown.** `marked` removed `sanitize` in v5 and DOMPurify is not in the tree. Notes come partly from `COMMENT ON` values living in the database, which is not always under the document author's control, and the output is a file people forward to colleagues. `marked` is configured to escape raw HTML: markdown formatting works, `