apache / apache/arrow-java

[Java][FlightSQL] JDBC Driver fails when Looker formats the URL with ':direct=' instead of '://'

Offen
#1,203 7 Kommentare 0 Reaktionen 0 zugewiesene Personen Auf GitHub ansehen
Type: enhancement
Vorherrschende Sprache
Java
Sterne
94
Forks
152
Ø Merge
3 T. 16 Std.
Gemergte PRs (30 T.)
11

Beschreibung

### Describe the enhancement requested

I am running a customer-hosted Looker instance and trying to connect to Dremio using the
Apache Arrow Flight SQL JDBC driver (`flight-sql-jdbc-driver`), registered in Looker via a
`custom_jdbc_config.yml` entry so the native Dremio dialect can be kept:

```yaml
- name: dremio
dir_name: dremio
module_path: org.apache.arrow.driver.jdbc.ArrowFlightJdbcDriver
override_jdbc_url_subprotocol: arrow-flight-sql
```

`override_jdbc_url_subprotocol` only swaps the *subprotocol token* Looker plugs into its
existing, dialect-specific URL template for Dremio — it does not rewrite the rest of the
template. Looker's native Dremio dialect builds connection strings as:

```
jdbc:dremio:direct=:
```

so after the subprotocol override, Looker generates:

```
jdbc:arrow-flight-sql:direct=:
```

instead of the driver's expected format:

```
jdbc:arrow-flight-sql://:
```

Because `ArrowFlightJdbcDriver` parses the URL with `java.net.URI.create(...)` and strictly
expects the `://` delimiter to separate scheme from host, it cannot extract a host from the
`:direct=` form, and the connection fails.

I tried a workaround by switching Looker's dialect dropdown to "Trino" (which emits the
standard `://` format). While queries did run correctly over Arrow Flight, it broke the
Looker SQL Runner UI, since Looker then sends Trino-specific metadata queries to Dremio.

To use this driver properly with Looker, the native Dremio dialect needs to stay selected,
which means the driver needs to tolerate Looker's `:direct=` URL format.

### Steps to Reproduce / Current Behavior

Passing a connection string containing `:direct=` to the driver:

```
jdbc:arrow-flight-sql:direct=:
```

fails instantly with:

```text
Java::JavaSql::SQLException: URL must have a host. Expected format: jdbc:arrow-flight-sql://[host][:port][?param1=value&...]
```

### Fix

Implemented and verified locally against `flight/flight-sql-jdbc-core`. Added a normalization
step to `ArrowFlightJdbcDriver.java` that rewrites the legacy `:direct=` delimiter to `://`
before any URL parsing occurs, applied consistently at every entry point that parses the
connection string (`acceptsURL`, `connect`, `getUrlsArgs`):

```java
// Some BI tools (e.g. Looker) generate connection strings using the legacy
// "jdbc:arrow-flight-sql:direct=:" syntax instead of the standard "://" delimiter.
// Normalize it so the URL can still be parsed as a valid URI.
private static final String LEGACY_DIRECT_DELIMITER = ":direct=";

private static String normalizeUrl(final String url) {
if (url != null && url.contains(LEGACY_DIRECT_DELIMITER)) {
return url.replace(LEGACY_DIRECT_DELIMITER, "://");
}
return url;
}
```

- `acceptsURL(String url)` now normalizes before checking the `jdbc:arrow-flight-sql://` /
`jdbc:arrow-flight://` prefixes.
- `connect(String url, Properties info)` normalizes once up front and uses the normalized URL
for both argument parsing (`getUrlsArgs`) and `ArrowFlightConnection.createNewConnection`.
- `getUrlsArgs(String url)` normalizes defensively as well, since it is `@VisibleForTesting`
and may be called directly.

This is a purely additive, backward-compatible change — URLs that don't contain `:direct=`
are returned unmodified.

### Testing / Verification

Added two regression tests to `ArrowFlightJdbcDriverTest`:

- `testDriverUrlParsingMechanismShouldAcceptLegacyDirectDelimiter` — verifies `acceptsURL` and
`getUrlsArgs` correctly parse host/port/params from a `jdbc:arrow-flight-sql:direct=...` URL.
- `testShouldConnectWhenProvidedWithLegacyDirectDelimiterUrl` — verifies a live connection can
be established end-to-end using the `:direct=` URL format against the test Flight SQL server.

Full `ArrowFlightJdbcDriverTest` suite: 18/18 tests passing, 0 failures, 0 errors.

Also validated end-to-end against a real customer-hosted Looker instance connecting to Dremio
with the native Dremio dialect (config above) — connection succeeded after applying the fix.

### Additional deployment note (not part of this fix, but relevant to anyone testing it with Looker)

The `flight-sql-jdbc-driver` shaded jar is built with Java 17 bytecode by default
(`maven.compiler.release=17` in the root POM), but Looker's bundled JRE is Java 11. Loading
the default build in Looker fails with:

```
UnsupportedClassVersionError: ... class file version 61.0, this version of the Java Runtime
only recognizes class file versions up to 55.0
```

To build a Looker-compatible jar, rebuild the driver and its Arrow-owned dependency closure
(`arrow-format`, `arrow-memory-core`, `arrow-memory-netty(-buffer-patch)`, `arrow-vector`,
`flight-core`, `flight-sql`, `flight-sql-jdbc-core`, `flight-sql-jdbc-driver`) with:

```bash
mvn -pl format,memory/memory-core,memory/memory-netty,memory/memory-netty-buffer-patch,vector,flight/flight-core,flight/flight-sql,flight/flight-sql-jdbc-core,flight/flight-sql-jdbc-driver \
-DskipTests -Dmaven.compiler.release=11 -Dmaven.compiler.source=11 -Dmaven.compiler.target=11 \
clean install
```

(`clean` is required — Maven's incremental compiler otherwise skips recompilation when only
compiler flags change.) This may be worth calling out in the driver's documentation for users
deploying it to JVM-hosted BI tools that ship an older bundled runtime.

Beitragsleitfaden

Beitragsleitfaden öffnen

Rechercherichtung

Beginne in flight/flight-sql/flight-sql-jdbc-driver bei ArrowFlightJdbcDriver.java und lies anschließend ArrowFlightJdbcDriverTest sowie dessen Tests zur URL-Analyse. Reproduziere den alten :direct= Fall und verifiziere, dass acceptsURL, getUrlsArgs und connect den Host, den Port und die Parameter verarbeiten, während die bestehende Suite weiterhin erfolgreich durchläuft.

Vom Indexierungsmodell aus dem Issue-Text verfasst.

Bewertung

Tech-Stack
java
Bereich
backend, databases
Issue-Typ
Bug
Schwierigkeit
2/5
Geschätzter Aufwand
1-3 Stunden
Aktivitätsstatus
Ruhig
Klarheit
Klar beschrieben
Anfängerfreundlichkeit
38/100

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.