googleapis / googleapis/google-cloud-java

[java-spanner-jdbc] Bigquery 2.9.16: Using this connector in Spark is resulting in all values in the spark dataframe being the column names

Offen
#12,281 1 Kommentar 0 Reaktionen 1 zugewiesene Person Zugewiesen an @olavloite Auf GitHub ansehen
api: spanner
Vorherrschende Sprache
Java
Sterne
2.1k
Forks
1.2k
Ø Merge
1 T. 23 Std.
Gemergte PRs (30 T.)
154

Beschreibung

Thanks for stopping by to let us know something could be better!

**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.

Please run down the following list and make sure you've tried the usual "quick fixes":

- Search the issues already opened: https://github.com/googleapis/java-spanner-jdbc/issues
- Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform

If you are still having issues, please include as much information as possible:

#### Environment details

1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
General, Core, and Other are also allowed as types
2. OS type and version: Mac 12.0
3. Java version: 20.0.1
4. version(s):

#### Steps to reproduce

1. Hook this connector up in a spark job with the following code:
```scala
def querySpanner[T](sqlQuery: String)(implicit spark: SparkSession, enc: org.apache.spark.sql.Encoder[T]): Dataset[T] = {
val url = "jdbc:cloudspanner:/projects/your-project-id/instances/your-instance-id/databases/your-database-id?credentials=$jsonKeyFilePath"

// Read data using Spark
val df = spark.read
.format("jdbc")
.option("url", url)
.option("dbtable", "myTable")
.option("driver", "com.google.cloud.spanner.jdbc.JdbcDriver")
.load()

// Convert DataFrame to Dataset
df.as[T]
}
```

2. See that it is returning the values as if every row is the column name:
+----+---+
|name| id|
+----+---+
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
+----+---+
only showing top 20 rows

My query is: "SELECT name, id FROM OrgInfoV2"

#### Any additional information below

It seems similar to this issue:
https://stackoverflow.com/questions/66983401/spark-mariadb-jdbc-sql-query-returns-column-names-instead-of-column-values
or this one:
https://stackoverflow.com/questions/63177736/spark-read-as-jdbc-return-all-rows-as-columns-name

where it appears to be issues with the driver

Following these steps guarantees the quickest resolution possible.

Thanks!

Beitragsleitfaden

Beitragsleitfaden öffnen

Bewertung

Dieses Issue wurde noch nicht bewertet.

Neue Issues direkt in Ihr Postfach

Eine kurze Übersicht über anfängerfreundliche GitHub-Issues.