dbeaver / dbeaver/dbeaver

Simba Spark JDBC/Databricks plugin incompatibility with other Spark environments

Open
#38,958 2 comments 0 reactions 0 assignees View on GitHub
new driver
Dominant language
Java
Stars
51.8k
Forks
4.4k
Avg merge
3d 13h
Merged PRs (30d)
183

Description

### Description

The Simba Spark JDBC driver can be used with Spark environments other than Databricks. When creating a generic JDBC driver/connection, it appears DBeaver will still invoke the Databricks plugin for the Simba Spark JDBC driver. The Databricks plugin makes specific assumptions about the Spark environment and causes issues with other Spark environments.

### DBeaver Version

DBeaver Community 25.1.1

### Operating System

Windows 11 Enterprise

### Database and driver

1. Oracle AIDP (aka IDL) Spark environment
2. Simba Spark JDBC Driver 2.6.18

### Steps to reproduce

1. In the top menu: navigate to Database -> Driver Manager
2. Click New
3. In Settings tab: Fill in Driver Name, Driver Type as "Generic" and Class Name as "com.simba.spark.jdbc.Driver"
4. In Libraries Tab, click Add File and add the driver then click OK
5. In the top menu: navigate to Database -> New Database Connection
6. Select the driver created in the previous steps and fill in the JDBC URL
7. Click Finish and the driver will connect to the data source and DBeaver will use the DBeaver Azure Databricks Model when connecting
8. The tables show up but the schema name is set to NULL instead of the correct schema name

### Additional context

Databricks exposes the table schema name as "databaseName" while Oracle AIDP/IDL exposes the table schema name as "namespace". It seems that even when you create a Generic JDBC driver/connection and use the Simba Spark JDBC driver, DBeaver will automatically associate it with the Databricks plugin.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.