mirror of
https://github.com/t8y2/dbx.git
synced 2026-10-20 00:17:44 +00:00
jischeng
c3893bf314
* feat(spark): add Apache Spark database driver Spark Thrift Server speaks the HiveServer2 protocol, so the Spark driver reuses the hive-jdbc agent runtime. Registers spark (agentKey=spark, defaultPort=10015) across the manifest, DatabaseType enum, agent catalog, SQL dialects, connection dialog, and a SparkAgent built on AbstractJdbcAgent. * feat(spark): add catalog support and official logo SparkAgent now supports Spark 3.4+ multi-catalog (Paimon, Lance, Iceberg) via catalog=<name> in url_params, mirroring the StarRocks catalog flow: - afterConnect switches to the configured catalog via USE <catalog> - listDatabases returns the catalog name when SHOW DATABASES is empty (catalogs like Lance expose tables at the catalog root with no databases) - listTables uses SHOW TABLES IN <catalog>[.<schema>] uniformly, working for both catalog-root tables (Lance) and catalog.schema tables (Paimon) - setSchemaSQL qualifies USE with the catalog to prevent resetting to spark_catalog on schema switches - buildJdbcUrl omits database from the URL path Also replaces the placeholder lightning icon with the official Apache Spark logo (spark-logo.png from spark.apache.org). * fix(spark): use catalog-qualified DESCRIBE for column retrieval getColumnsFromDescribe used USE <schema> + DESCRIBE <table>, which generated USE `catalog`.`catalog` when the sidebar passed the catalog name as the schema (Lance case where the catalog is the only node). This failed and fell back to JDBC metadata which also returned nothing, resulting in an empty DDL (CREATE TABLE ... ()). Now uses DESCRIBE `catalog`.`table` directly when a catalog is configured, bypassing USE entirely. Also fixes setSchemaSQL to treat schema == catalog as catalog-only (no .schema suffix). * fix(ci): add spark to metadata coverage matrix and BRIDGE_REQUIRED_TYPES - agents/metadata-constraint-coverage.tsv: register spark as intentional-fallback (SHOW TABLES/SHOW DATABASES, like Hive/TDengine) - packages/node-core/src/diagnostics.ts: add spark to BRIDGE_REQUIRED_TYPES (mcpMode=bridge, matches manifest) * fix(ci): add spark to DBX_CONNECTION_TYPE_DESCRIPTION in mcp-server The driver-manifest test asserts every manifest dbType appears in the MCP server's connection type description string.
12 lines
339 B
Groovy
12 lines
339 B
Groovy
dependencies {
|
|
implementation fileTree(dir: 'libs', include: ['*.jar'])
|
|
implementation 'org.apache.hive:hive-jdbc:4.0.1:standalone'
|
|
implementation 'org.slf4j:slf4j-nop:1.7.30'
|
|
}
|
|
|
|
tasks.named('shadowJar') {
|
|
manifest {
|
|
attributes('Agent-Label': 'Apache Spark', 'Main-Class': 'com.dbx.agent.spark.SparkAgent')
|
|
}
|
|
}
|