[Java][FlightSQL] JDBC Driver fails when Looker formats the URL with ':direct=' instead of '://'
- Ngôn ngữ chính
- Java
- Star
- 94
- Fork
- 152
- Merge trung bình
- 3 ngày 16 giờ
- Pull request đã merge (30 ngày)
- 11
Mô tả
### Describe the enhancement requested
I am running a customer-hosted Looker instance and trying to connect to Dremio using the
Apache Arrow Flight SQL JDBC driver (`flight-sql-jdbc-driver`), registered in Looker via a
`custom_jdbc_config.yml` entry so the native Dremio dialect can be kept:
```yaml
- name: dremio
dir_name: dremio
module_path: org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver
override_jdbc_url_subprotocol: arrow-flight-sql
```
`override_jdbc_url_subprotocol` only swaps the *subprotocol token* Looker plugs into its
existing, dialect-specific URL template for Dremio — it does not rewrite the rest of the
template. Looker's native Dremio dialect builds connection strings as:
```
jdbc:dremio:direct=:
```
so after the subprotocol override, Looker generates:
```
jdbc:arrow-flight-sql:direct=:
```
instead of the driver's expected format:
```
jdbc:arrow-flight-sql://:
```
Because `ArrowFlightJdbcDriver` parses the URL with `java.net.URI.create(...)` and strictly
expects the `://` delimiter to separate scheme from host, it cannot extract a host from the
`:direct=` form, and the connection fails.
I tried a workaround by switching Looker's dialect dropdown to "Trino" (which emits the
standard `://` format). While queries did run correctly over Arrow Flight, it broke the
Looker SQL Runner UI, since Looker then sends Trino-specific metadata queries to Dremio.
To use this driver properly with Looker, the native Dremio dialect needs to stay selected,
which means the driver needs to tolerate Looker's `:direct=` URL format.
### Steps to Reproduce / Current Behavior
Passing a connection string containing `:direct=` to the driver:
```
jdbc:arrow-flight-sql:direct=:
```
fails instantly with:
```text
Java::JavaSql::SQLException: URL must have a host. Expected format: jdbc:arrow-flight-sql://[host][:port][?param1=value&...]
```
### Fix
Implemented and verified locally against `flight/flight-sql-jdbc-core`. Added a normalization
step to `ArrowFlightJdbcDriver.java` that rewrites the legacy `:direct=` delimiter to `://`
before any URL parsing occurs, applied consistently at every entry point that parses the
connection string (`acceptsURL`, `connect`, `getUrlsArgs`):
```java
// Some BI tools (e.g. Looker) generate connection strings using the legacy
// "jdbc:arrow-flight-sql:direct=:" syntax instead of the standard "://" delimiter.
// Normalize it so the URL can still be parsed as a valid URI.
private static final String LEGACY_DIRECT_DELIMITER = ":direct=";
private static String normalizeUrl(final String url) {
if (url != null && url.contains(LEGACY_DIRECT_DELIMITER)) {
return url.replace(LEGACY_DIRECT_DELIMITER, "://");
}
return url;
}
```
- `acceptsURL(String url)` now normalizes before checking the `jdbc:arrow-flight-sql://` /
`jdbc:arrow-flight://` prefixes.
- `connect(String url, Properties info)` normalizes once up front and uses the normalized URL
for both argument parsing (`getUrlsArgs`) and `ArrowFlightConnection.createNewConnection`.
- `getUrlsArgs(String url)` normalizes defensively as well, since it is `@VisibleForTesting`
and may be called directly.
This is a purely additive, backward-compatible change — URLs that don't contain `:direct=`
are returned unmodified.
### Testing / Verification
Added two regression tests to `ArrowFlightJdbcDriverTest`:
- `testDriverUrlParsingMechanismShouldAcceptLegacyDirectDelimiter` — verifies `acceptsURL` and
`getUrlsArgs` correctly parse host/port/params from a `jdbc:arrow-flight-sql:direct=...` URL.
- `testShouldConnectWhenProvidedWithLegacyDirectDelimiterUrl` — verifies a live connection can
be established end-to-end using the `:direct=` URL format against the test Flight SQL server.
Full `ArrowFlightJdbcDriverTest` suite: 18/18 tests passing, 0 failures, 0 errors.
Also validated end-to-end against a real customer-hosted Looker instance connecting to Dremio
with the native Dremio dialect (config above) — connection succeeded after applying the fix.
### Additional deployment note (not part of this fix, but relevant to anyone testing it with Looker)
The `flight-sql-jdbc-driver` shaded jar is built with Java 17 bytecode by default
(`maven.compiler.release=17` in the root POM), but Looker's bundled JRE is Java 11. Loading
the default build in Looker fails with:
```
UnsupportedClassVersionError: ... class file version 61.0, this version of the Java Runtime
only recognizes class file versions up to 55.0
```
To build a Looker-compatible jar, rebuild the driver and its Arrow-owned dependency closure
(`arrow-format`, `arrow-memory-core`, `arrow-memory-netty(-buffer-patch)`, `arrow-vector`,
`flight-core`, `flight-sql`, `flight-sql-jdbc-core`, `flight-sql-jdbc-driver`) with:
```bash
mvn -pl format,memory/memory-core,memory/memory-netty,memory/memory-netty-buffer-patch,vector,flight/flight-core,flight/flight-sql,flight/flight-sql-jdbc-core,flight/flight-sql-jdbc-driver \
-DskipTests -Dmaven.compiler.release=11 -Dmaven.compiler.source=11 -Dmaven.compiler.target=11 \
clean install
```
(`clean` is required — Maven's incremental compiler otherwise skips recompilation when only
compiler flags change.) This may be worth calling out in the driver's documentation for users
deploying it to JVM-hosted BI tools that ship an older bundled runtime.
Hướng dẫn đóng góp
Hướng nghiên cứu
Bắt đầu trong flight/flight-sql/flight-sql-jdbc-driver tại ArrowFlightJdbcDriver.java, sau đó đọc ArrowFlightJdbcDriverTest và các bài kiểm thử phân tích URL của nó. Tái hiện trường hợp legacy :direct= và xác minh rằng acceptsURL, getUrlsArgs và connect xử lý host, port và các tham số, đồng thời bộ kiểm thử hiện có vẫn tiếp tục thành công.
Do mô hình lập chỉ mục viết ra từ nội dung của issue.
Đánh giá
- Công nghệ
- java
- Lĩnh vực
- backend, databases
- Loại issue
- Lỗi
- Độ khó
- 2/5
- Thời gian dự kiến
- 1-3 giờ
- Mức độ hoạt động
- Ít trao đổi
- Độ rõ ràng
- Đặc tả rõ ràng
- Mức phù hợp với người mới
- 38/100