Simba Spark JDBC/Databricks plugin incompatibility with other Spark environments
- Dominant language
- Java
- Stars
- 51.8k
- Forks
- 4.4k
- Avg merge
- 3d 13h
- Merged PRs (30d)
- 183
Description
### Description
The Simba Spark JDBC driver can be used with Spark environments other than Databricks. When creating a generic JDBC driver/connection, it appears DBeaver will still invoke the Databricks plugin for the Simba Spark JDBC driver. The Databricks plugin makes specific assumptions about the Spark environment and causes issues with other Spark environments.
### DBeaver Version
DBeaver Community 25.1.1
### Operating System
Windows 11 Enterprise
### Database and driver
1. Oracle AIDP (aka IDL) Spark environment
2. Simba Spark JDBC Driver 2.6.18
### Steps to reproduce
1. In the top menu: navigate to Database -> Driver Manager
2. Click New
3. In Settings tab: Fill in Driver Name, Driver Type as "Generic" and Class Name as "com.simba.spark.jdbc.Driver"
4. In Libraries Tab, click Add File and add the driver then click OK
5. In the top menu: navigate to Database -> New Database Connection
6. Select the driver created in the previous steps and fill in the JDBC URL
7. Click Finish and the driver will connect to the data source and DBeaver will use the DBeaver Azure Databricks Model when connecting
8. The tables show up but the schema name is set to NULL instead of the correct schema name
### Additional context
Databricks exposes the table schema name as "databaseName" while Oracle AIDP/IDL exposes the table schema name as "namespace". It seems that even when you create a Generic JDBC driver/connection and use the Simba Spark JDBC driver, DBeaver will automatically associate it with the Databricks plugin.
Contributor guide
Assessment
This issue has not been assessed yet.