matrixorigin / matrixorigin/matrixone
[Bug]: BITMAP_COUNT reports malformed bitmap payloads as empty sets
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Description
`BITMAP_COUNT` silently returns `0` for malformed binary input instead of reporting that the value is not a valid roaring bitmap. The same payload passed through `BITMAP_OR_AGG` produces an explicit decoding error.
This makes corrupted bitmap state indistinguishable from a valid empty set.
## Environment
- MatrixOne: latest official `main`, commit `07fdd4ae0f80f287b93fc4b525fd296d5617abc7`
- Deployment: local standalone launch (isolated Log/TN/CN ports and data directory)
## Reproduction
```sql
select bitmap_count(cast('' as varbinary)) empty_payload,
bitmap_count(cast('not-a-bitmap' as varbinary)) text_payload,
bitmap_count(unhex('01020304')) short_binary_payload;
create table payloads(b varbinary);
insert into payloads
select bitmap_construct_agg(v) from
(select cast(1 as bigint unsigned) v) s;
insert into payloads values (cast('not-a-bitmap' as varbinary));
select bitmap_count(bitmap_or_agg(b)) from payloads;
```
## Actual behavior
```text
BITMAP_COUNT('') 0
BITMAP_COUNT('not-a-bitmap') 0
BITMAP_COUNT(0x01020304) 0
BITMAP_OR_AGG mixed payload:
internal error: invalid roaring bitmap cookie 762605422
```
The behavior reproduced identically in 3/3 fresh runs.
## Expected behavior
`BITMAP_COUNT` should return an invalid-input error for malformed bitmap payloads, consistent with `BITMAP_OR_AGG` and with `HLL_CARDINALITY`'s validation of malformed HLL states.
## Code-path analysis
`pkg/sql/plan/function/func_unary.go:BitmapCount` creates a roaring bitmap and calls `UnmarshalBinary`, but its callback discards the error and returns zero. `HllCardinality` immediately below uses an error-returning wrapper and propagates invalid-state errors, showing the missing validation pattern.
## Impact
- corrupted persisted/intermediate bitmap state can be reported as a legitimate zero cardinality;
- monitoring and analytical queries cannot distinguish data corruption from an empty set;
- scalar count and aggregate merge disagree on whether the same payload is valid.
## Suggested regression coverage
- empty, truncated, random, wrong-cookie, and valid-empty serialized payloads;
- VARBINARY/BINARY/BLOB inputs;
- `BITMAP_COUNT` and `BITMAP_OR_AGG` consistency;
- valid construct/serialize/count round trips.
## Duplicate search
Open and closed issues were searched for `BITMAP_COUNT`, invalid payload, roaring cookie, deserialization, and corrupted bitmap; no matching report was found.
Contributor guide
Assessment
This issue has not been assessed yet.