googleapis / googleapis/google-cloud-java
[java-spanner-jdbc] Bigquery 2.9.16: Using this connector in Spark is resulting in all values in the spark dataframe being the column names
- Langage dominant
- Java
- Étoiles
- 2.1k
- Forks
- 1.2k
- Merge moyen
- 1 j 23 h
- PR mergées (30 j)
- 154
Description
Thanks for stopping by to let us know something could be better!
**PLEASE READ**: If you have a support contract with Google, please create an issue in the [support console](https://cloud.google.com/support/) instead of filing on GitHub. This will ensure a timely response.
Please run down the following list and make sure you've tried the usual "quick fixes":
- Search the issues already opened: https://github.com/googleapis/java-spanner-jdbc/issues
- Check for answers on StackOverflow: http://stackoverflow.com/questions/tagged/google-cloud-platform
If you are still having issues, please include as much information as possible:
#### Environment details
1. Specify the API at the beginning of the title. For example, "BigQuery: ...").
General, Core, and Other are also allowed as types
2. OS type and version: Mac 12.0
3. Java version: 20.0.1
4. version(s):
#### Steps to reproduce
1. Hook this connector up in a spark job with the following code:
```scala
def querySpanner[T](sqlQuery: String)(implicit spark: SparkSession, enc: org.apache.spark.sql.Encoder[T]): Dataset[T] = {
val url = "jdbc:cloudspanner:/projects/your-project-id/instances/your-instance-id/databases/your-database-id?credentials=$jsonKeyFilePath"
// Read data using Spark
val df = spark.read
.format("jdbc")
.option("url", url)
.option("dbtable", "myTable")
.option("driver", "com.google.cloud.spanner.jdbc.JdbcDriver")
.load()
// Convert DataFrame to Dataset
df.as[T]
}
```
2. See that it is returning the values as if every row is the column name:
+----+---+
|name| id|
+----+---+
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
|name| id|
+----+---+
only showing top 20 rows
My query is: "SELECT name, id FROM OrgInfoV2"
#### Any additional information below
It seems similar to this issue:
https://stackoverflow.com/questions/66983401/spark-mariadb-jdbc-sql-query-returns-column-names-instead-of-column-values
or this one:
https://stackoverflow.com/questions/63177736/spark-read-as-jdbc-return-all-rows-as-columns-name
where it appears to be issues with the driver
Following these steps guarantees the quickest resolution possible.
Thanks!
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.