apache / apache/datafusion

Should can_coerce_from promote Utf8 to LargeUtf8?

Open
#13,325 3 comments 1 reaction 0 assignees View on GitHub
bug
Dominant language
Rust
Stars
9.3k
Forks
2.4k
Avg merge
3d 7h
Merged PRs (30d)
344

Description

### Describe the bug

The function [can_coerce_from](https://docs.rs/datafusion/latest/datafusion/logical_expr/type_coercion/functions/fn.can_coerce_from.html) establishes the lossless conversions between data types.

It currently says that LargeUtf8 can be coerced into Utf8. I'm not sure this should be true?

https://github.com/apache/datafusion/blob/0f584c8db1d9c36d8f73818d8bee24384eb4926e/datafusion/expr/src/type_coercion/functions.rs#L768-L769

I suppose I could construct a single row that contains more than i32 bytes of string data. But it also seems generally desirable for DataFusion to promote to the large string/binary type?

### To Reproduce

_No response_

### Expected behavior

Utf8 promotes into LargeUt8, but not vice versa.

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reading datafusion/expr/src/type_coercion/functions.rs around lines 768-769 and the can_coerce_from documentation linked in the issue. Confirm the intended Utf8 and LargeUtf8 conversion direction, then update the relevant behavior and tests so Utf8 promotes to LargeUtf8 but not vice versa.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
databases
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Clearly specified
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.