apache / apache/gluten

[VL] Result mismatch when get_json_object with wildcard path

Open
#8,758 6 comments 0 reactions 1 assignee Claimed by @philo-he View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

VL (Velox)

### Bug description

Query
`select get_json_object(value, '$.store.basket[*]') FROM test`
Spark Result
` Array([[[1,2,{"b":"y","a":"x"}],[3,4],[5,6]]])`
Gluten Result
`Array([null])`
```
scala> val json =
| """
| |{"store":{"fruit":[{"weight":8,"type":"apple"},{"weight":9,"type":"pear"}],
| |"basket":[[1,2,{"b":"y","a":"x"}],[3,4],[5,6]],"book":[{"author":"Nigel Rees",
| |"title":"Sayings of the Century","category":"reference","price":8.95},
| |{"author":"Herman Melville","title":"Moby Dick","category":"fiction","price":8.99,
| |"isbn":"0-553-21311-3"},{"author":"J. R. R. Tolkien","title":"The Lord of the Rings",
| |"category":"fiction","reader":[{"age":25,"name":"bob"},{"age":26,"name":"jack"}],
| |"price":22.99,"isbn":"0-395-19395-8"}],"bicycle":{"price":19.95,"color":"red"}},
| |"email":"amy@only_for_json_udf_test.net","owner":"amy","zip code":"94025",
| |"fb:testid":"1234"}
| |""".stripMargin
json: String =
"
{"store":{"fruit":[{"weight":8,"type":"apple"},{"weight":9,"type":"pear"}],
"basket":[[1,2,{"b":"y","a":"x"}],[3,4],[5,6]],"book":[{"author":"Nigel Rees",
"title":"Sayings of the Century","category":"reference","price":8.95},
{"author":"Herman Melville","title":"Moby Dick","category":"fiction","price":8.99,
"isbn":"0-553-21311-3"},{"author":"J. R. R. Tolkien","title":"The Lord of the Rings",
"category":"fiction","reader":[{"age":25,"name":"bob"},{"age":26,"name":"jack"}],
"price":22.99,"isbn":"0-395-19395-8"}],"bicycle":{"price":19.95,"color":"red"}},
"email":"amy@only_for_json_udf_test.net","owner":"amy","zip code":"94025",
"fb:testid":"1234"}
"

scala> Seq(json).toDF("value").createOrReplaceTempView("test")

scala> spark.sql("select get_json_object(value, '$.store.basket[*]') FROM test").collect
res7: Array[org.apache.spark.sql.Row] = Array([[[1,2,{"b":"y","a":"x"}],[3,4],[5,6]]])

scala> spark.conf.set("spark.sql.optimizer.excludedRules", "org.apache.spark.sql.catalyst.optimizer.ConvertToLocalRelation,org.apache.spark.sql.catalyst.optimizer.ConstantFolding,org.apache.spark.sql.catalyst.optimizer.NullPropagation")

scala> spark.sql("select get_json_object(value, '$.store.basket[*]') FROM test").collect
res9: Array[org.apache.spark.sql.Row] = Array([null])
```

### Spark version

None

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

```bash

```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.