apache / apache/doris

[Bug] Empty array/map field in Hive text is read as NULL (or errors) instead of an empty collection

Open Beginner friendly
#66,039 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
15.9k
Forks
3.9k
Avg merge
2d 23h
Merged PRs (30d)
520

Description

### Search before asking

- [x] I had searched in the [issues](https://github.com/apache/doris/issues?q=is%3Aissue) and found no similar issues.

### Version

master / 4.0

### What's Wrong?

When reading a Hive text-format table, an empty field for an `ARRAY` or `MAP` column is not deserialized as an empty collection. In `DataTypeArraySerDe::deserialize_one_cell_from_hive_text` and `DataTypeMapSerDe::deserialize_one_cell_from_hive_text` (be/src/core/data_type_serde/), an empty slice returns `Status::InvalidArgument("slice is empty!")`:

- For a **nullable** column, `DataTypeNullableSerDe` swallows that error and inserts NULL, so an empty field silently becomes NULL.
- For a **non-nullable** column, the error propagates and fails the load.

Neither matches Spark/Hive semantics, where an empty complex-type field is an empty collection (`[]` / `{}`), distinct from an explicit NULL (`\N`).

### What You Expected?

An empty Hive-text field for an ARRAY/MAP column should deserialize to an empty collection (`[]` / `{}`). An explicit NULL marker (`\N`) should remain NULL.

### How to Reproduce?

Query a Hive text table (or use the Hive text serde) where an ARRAY/MAP column has an empty field. The value comes back as NULL (nullable column) or the read fails (non-nullable column), instead of an empty collection.

### Anything Else?

Fix: on an empty slice, append an empty collection (a zero-length element) and return OK, instead of returning InvalidArgument. This changes empty **nullable** array/map fields from NULL to `[]`/`{}`; explicit `\N` is unchanged. (Struct is intentionally left out — an "empty struct" has no canonical representation; that can be a follow-up.)

### Are you willing to submit PR?

- [x] Yes I am willing to submit a PR!

### Code of Conduct

- [x] I agree to follow this project's [Code of Conduct](https://www.apache.org/foundation/policies/conduct)

Contributor guide

Open the contributing guide

Research direction

Start in be/src/core/data_type_serde/ at DataTypeArraySerDe::deserialize_one_cell_from_hive_text and DataTypeMapSerDe::deserialize_one_cell_from_hive_text, and inspect how empty slices and the explicit \N marker are handled. Verify the change with Hive text deserialization coverage: empty fields should produce [] or {}, while \N remains NULL and non-nullable loads no longer fail.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
databases
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
76/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.