"Cannot infer common argument type for comparison operation Union..."
- Dominant language
- Rust
- Stars
- 9.3k
- Forks
- 2.4k
- Avg merge
- 3d 7h
- Merged PRs (30d)
- 344
Description
### Describe the bug
See https://github.com/datafusion-contrib/datafusion-functions-json/pull/3
I have a union defined by
```rs
DataType::Union(
UnionFields::new(
vec![0, 1, 2, 3, 4, 5, 6],
vec![
Field::new("null", DataType::Boolean, true),
Field::new("bool", DataType::Boolean, false),
Field::new("int", DataType::Int64, false),
Field::new("float", DataType::Float64, false),
Field::new("string", DataType::Utf8, false),
Field::new("array", DataType::Utf8, false),
Field::new("object", DataType::Utf8, false),
]
),
UnionMode::Sparse,
)
```
When I try to compare it to an integer with `json_get(json_data, 'foo')=123`, I get the error:
```
called `Result::unwrap()` on an `Err` value: Plan("Cannot infer common argument type for comparison operation Union([(0, Field { name: \"null\", data_type: Boolean, nullable: true, dict_id: 0, dict_is_ordered: false, metadata: {} }), (1, Field { name: \"bool\", data_type: Boolean, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }), (2, Field { name: \"int\", data_type: Int64, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }), (3, Field { name: \"float\", data_type: Float64, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }), (4, Field { name: \"string\", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }), (5, Field { name: \"array\", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} }), (6, Field { name: \"object\", data_type: Utf8, nullable: false, dict_id: 0, dict_is_ordered: false, metadata: {} })], Sparse) = Int64")
```
Is there any way to rewire the logic plan to tell datafusion how to implement these comparisons?
If not, I might have to abandon the `json_get` method, and instead implemtn `json_get_str`, `json_get_int` etc., which would be unfortunate.
I tried implementing `FunctionRewrite`, but the error occurs before it's called.
### To Reproduce
see tests in https://github.com/datafusion-contrib/datafusion-functions-json/pull/3
### Expected behavior
_No response_
### Additional context
_No response_
Contributor guide
Research direction
Start with the tests referenced in datafusion-functions-json/pull/3 and trace comparison type inference before FunctionRewrite, since the reported error occurs first. Confirm the intended behavior for json_get(json_data, 'foo')=123 and add a regression test that covers the Union-versus-Int64 comparison.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100