dbx/agents/metadata-constraint-coverage.tsv at agents-v0.2.47

mirror of https://github.com/t8y2/dbx.git synced 2026-10-27 00:03:52 +00:00

Files

T

jischeng c3893bf314 feat(spark): add Apache Spark database driver with catalog support

* feat(spark): add Apache Spark database driver

Spark Thrift Server speaks the HiveServer2 protocol, so the Spark driver
reuses the hive-jdbc agent runtime. Registers spark (agentKey=spark,
defaultPort=10015) across the manifest, DatabaseType enum, agent catalog,
SQL dialects, connection dialog, and a SparkAgent built on AbstractJdbcAgent.

* feat(spark): add catalog support and official logo

SparkAgent now supports Spark 3.4+ multi-catalog (Paimon, Lance, Iceberg)
via catalog=<name> in url_params, mirroring the StarRocks catalog flow:
- afterConnect switches to the configured catalog via USE <catalog>
- listDatabases returns the catalog name when SHOW DATABASES is empty
  (catalogs like Lance expose tables at the catalog root with no databases)
- listTables uses SHOW TABLES IN <catalog>[.<schema>] uniformly, working
  for both catalog-root tables (Lance) and catalog.schema tables (Paimon)
- setSchemaSQL qualifies USE with the catalog to prevent resetting to
  spark_catalog on schema switches
- buildJdbcUrl omits database from the URL path

Also replaces the placeholder lightning icon with the official Apache Spark
logo (spark-logo.png from spark.apache.org).

* fix(spark): use catalog-qualified DESCRIBE for column retrieval

getColumnsFromDescribe used USE <schema> + DESCRIBE <table>, which
generated USE `catalog`.`catalog` when the sidebar passed the catalog
name as the schema (Lance case where the catalog is the only node).
This failed and fell back to JDBC metadata which also returned nothing,
resulting in an empty DDL (CREATE TABLE ... ()).

Now uses DESCRIBE `catalog`.`table` directly when a catalog is
configured, bypassing USE entirely. Also fixes setSchemaSQL to treat
schema == catalog as catalog-only (no .schema suffix).

* fix(ci): add spark to metadata coverage matrix and BRIDGE_REQUIRED_TYPES

- agents/metadata-constraint-coverage.tsv: register spark as
  intentional-fallback (SHOW TABLES/SHOW DATABASES, like Hive/TDengine)
- packages/node-core/src/diagnostics.ts: add spark to
  BRIDGE_REQUIRED_TYPES (mcpMode=bridge, matches manifest)

* fix(ci): add spark to DBX_CONNECTION_TYPE_DESCRIPTION in mcp-server

The driver-manifest test asserts every manifest dbType appears in the
MCP server's connection type description string.

2026-07-07 14:36:22 +08:00

3.0 KiB

Raw Permalink Blame History

1	driver	strategy	scope	reason
2	access	intentional-fallback	java-jdbc-metadata	Uses Access JDBC metadata; no portable stable server-side paging API, common constraints filter locally.
3	bigquery	native-pushdown	java-sql	Uses dataset INFORMATION_SCHEMA.TABLES with type, normal-name filter, stable order, and literal LIMIT/OFFSET.
4	cassandra	intentional-fallback	java-cql	Uses Cassandra system_schema; portable LIKE/fuzzy paging is not safe across versions, common constraints filter locally.
5	dameng	native-pushdown	java-sql	Uses ALL_OBJECTS with type, filter, stable order, and LIMIT/OFFSET with legacy fallback.
6	databend	native-pushdown	java-sql	Uses system procedures for routine pushdown and shared table metadata constraints for table-like objects.
7	db2	native-pushdown	java-sql	Uses SYSCAT metadata with type, filter, stable order, and OFFSET/FETCH.
8	gbase8a	native-pushdown	java-sql	Uses information_schema with type, filter, stable order, and LIMIT/OFFSET.
9	gbase8s	native-pushdown	java-sql	Uses systables with type, filter, stable order, and SKIP/FIRST.
10	goldendb	native-pushdown	java-sql	Uses information_schema with type, filter, stable order, and LIMIT/OFFSET.
11	h2	native-pushdown	java-sql	Uses INFORMATION_SCHEMA with type, filter, stable order, and LIMIT/OFFSET.
12	hive	intentional-fallback	java-sql	Uses Hive metadata paths where stable server-side fuzzy paging is not portable, common constraints filter locally.
13	informix	native-pushdown	java-sql	Uses systables/sysprocedures with type, filter, stable order, and SKIP/FIRST.
14	kingbase	native-pushdown	java-sql	Uses sys_catalog/information_schema with type, filter, stable order, and LIMIT/OFFSET for both compatibility modes.
15	kylin	intentional-fallback	java-jdbc-metadata	Uses JDBC metadata from Kylin driver; no portable server-side paging API, common constraints filter locally.
16	neo4j	intentional-fallback	java-graph-metadata	Uses JDBC metadata with Cypher fallback; label discovery cannot guarantee relational-style filtered paging, common constraints filter locally.
17	oceanbase-oracle	native-pushdown	java-sql	Uses Oracle-compatible metadata SQL with type, filter, stable order, and ROWNUM paging.
18	oracle-go	native-pushdown	native-go	Uses Oracle-compatible metadata SQL with type, filter, stable order, and ROWNUM paging, with legacy profile guard.
19	snowflake	native-pushdown	java-sql	Uses INFORMATION_SCHEMA with type, filter, stable order, and literal LIMIT/OFFSET.
20	spark	intentional-fallback	java-show-metadata	Uses SHOW TABLES/SHOW DATABASES; stable server-side fuzzy paging is not portable, common constraints filter locally.
21	sundb	native-pushdown	java-sql	Uses information_schema with type, filter, stable order, and LIMIT/OFFSET.
22	tdengine	intentional-fallback	java-show-metadata	Uses SHOW STABLES/TABLES; stable server-side fuzzy paging is not portable, common constraints filter locally.
23	trino	native-pushdown	java-sql	Uses information_schema.tables with type, filter, stable order, and literal LIMIT/OFFSET.
24	xugu	native-pushdown	native-go	Uses ALL_TABLES/ALL_VIEWS metadata with type, filter, stable order, and ROWNUM paging.

3.0 KiB Raw Permalink Blame History

3.0 KiB

Raw Permalink Blame History