QuantileState protobuf serialization corrupts data in write/read_column_to_pb
- Dominant language
- Java
- Stars
- 15.9k
- Forks
- 3.9k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 520
Description
### Version
trunk (4b7a09ff6fd)
### What's Wrong?
`DataTypeQuantileStateSerDe::write_column_to_pb` and `read_column_from_pb` use raw memory copy (`get_data_at` / `insert_data`) to serialize/deserialize `QuantileState` values. However, `QuantileState` contains heap-allocated members (`shared_ptr`, `vector`), so copying `sizeof(QuantileState)` bytes of raw memory does not capture the heap data. On deserialization, `reinterpret_cast`-ing the incomplete bytes back into a `QuantileState` results in dangling pointers and corrupted data.
Additionally, `write_column_to_pb` does not set `PGenericType::QUANTILE_STATE` as the type ID in the `PValues` message.
### What You Expected?
`write_column_to_pb` should use `QuantileState::serialize()` to properly serialize the logical value, and `read_column_from_pb` should use `QuantileState::deserialize()` to reconstruct it. The type ID should be set to `PGenericType::QUANTILE_STATE`.
### How to Reproduce?
Run the BE unit test `QuantileStateSerdeTest.writeColumnToPb` on the unfixed code. The test fails because the round-trip through `write_column_to_pb` / `read_column_from_pb` produces corrupted data:
```
Expected equality of these values:
column_quantile_state->get_data_at(j)
except_column->get_data_at(j)
```
The raw memory comparison shows mismatched bytes because the heap data is lost during serialization.
### Are you willing to submit PR?
Yes, I will submit a PR to fix this.
Contributor guide
Assessment
This issue has not been assessed yet.