matrixorigin / matrixorigin/matrixone

[Bug]: BITMAP_COUNT reports malformed bitmap payloads as empty sets

Open
#28,865 0 comments 0 reactions 1 assignee Claimed by @XuPeng-SH View on GitHub
kind/bug needs-triage
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Description

`BITMAP_COUNT` silently returns `0` for malformed binary input instead of reporting that the value is not a valid roaring bitmap. The same payload passed through `BITMAP_OR_AGG` produces an explicit decoding error.

This makes corrupted bitmap state indistinguishable from a valid empty set.

## Environment

- MatrixOne: latest official `main`, commit `07fdd4ae0f80f287b93fc4b525fd296d5617abc7`
- Deployment: local standalone launch (isolated Log/TN/CN ports and data directory)

## Reproduction

```sql
select bitmap_count(cast('' as varbinary)) empty_payload,
bitmap_count(cast('not-a-bitmap' as varbinary)) text_payload,
bitmap_count(unhex('01020304')) short_binary_payload;

create table payloads(b varbinary);
insert into payloads
select bitmap_construct_agg(v) from
(select cast(1 as bigint unsigned) v) s;
insert into payloads values (cast('not-a-bitmap' as varbinary));
select bitmap_count(bitmap_or_agg(b)) from payloads;
```

## Actual behavior

```text
BITMAP_COUNT('') 0
BITMAP_COUNT('not-a-bitmap') 0
BITMAP_COUNT(0x01020304) 0

BITMAP_OR_AGG mixed payload:
internal error: invalid roaring bitmap cookie 762605422
```

The behavior reproduced identically in 3/3 fresh runs.

## Expected behavior

`BITMAP_COUNT` should return an invalid-input error for malformed bitmap payloads, consistent with `BITMAP_OR_AGG` and with `HLL_CARDINALITY`'s validation of malformed HLL states.

## Code-path analysis

`pkg/sql/plan/function/func_unary.go:BitmapCount` creates a roaring bitmap and calls `UnmarshalBinary`, but its callback discards the error and returns zero. `HllCardinality` immediately below uses an error-returning wrapper and propagates invalid-state errors, showing the missing validation pattern.

## Impact

- corrupted persisted/intermediate bitmap state can be reported as a legitimate zero cardinality;
- monitoring and analytical queries cannot distinguish data corruption from an empty set;
- scalar count and aggregate merge disagree on whether the same payload is valid.

## Suggested regression coverage

- empty, truncated, random, wrong-cookie, and valid-empty serialized payloads;
- VARBINARY/BINARY/BLOB inputs;
- `BITMAP_COUNT` and `BITMAP_OR_AGG` consistency;
- valid construct/serialize/count round trips.

## Duplicate search

Open and closed issues were searched for `BITMAP_COUNT`, invalid payload, roaring cookie, deserialization, and corrupted bitmap; no matching report was found.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.