matrixorigin / matrixorigin/matrixone
[Compatibility]: JSON aggregates reject BIT and binary values supported by MySQL
- Dominant language
- Go
- Stars
- 1.9k
- Forks
- 311
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 768
Description
## Description
MatrixOne's scalar JSON constructors support `BIT`, `BINARY`, `VARBINARY`, and `BLOB` using MySQL-compatible opaque base64 JSON strings, but `JSON_ARRAYAGG` and `JSON_OBJECTAGG` reject the same values. MySQL 8 accepts them in both JSON aggregates and preserves the same type tags and bytes.
## Environment
- MatrixOne branch: `main`
- MatrixOne commit: `0c3a04f390adaf6281fd592a49778ea5b1155e67`
- MatrixOne deployment: local standalone, 1 CN / 1 TN / 1 LogService
- Reference: local MySQL `8.0.45`
## Steps to reproduce
```sql
CREATE DATABASE json_aggregate_binary_repro;
USE json_aggregate_binary_repro;
CREATE TABLE t (
k VARCHAR(8),
b BIT(8),
bin BINARY(4),
vb VARBINARY(4),
bl BLOB
);
INSERT INTO t VALUES
('a', b'10101010', X'41004200', X'410042', X'410042'),
('b', b'00000000', X'00000000', X'', X''),
('c', NULL, NULL, NULL, NULL);
SELECT JSON_ARRAY(b), JSON_ARRAY(bin), JSON_ARRAY(vb), JSON_ARRAY(bl)
FROM t WHERE k = 'a';
SELECT JSON_ARRAYAGG(b), JSON_ARRAYAGG(bin),
JSON_ARRAYAGG(vb), JSON_ARRAYAGG(bl)
FROM t;
SELECT JSON_OBJECTAGG(k, b), JSON_OBJECTAGG(k, bin),
JSON_OBJECTAGG(k, vb), JSON_OBJECTAGG(k, bl)
FROM t;
```
## MatrixOne result
The scalar constructor returns the same values as MySQL:
```text
["base64:type16:qg=="]
["base64:type254:QQBCAA=="]
["base64:type15:QQBC"]
["base64:type252:QQBC"]
```
The aggregate expressions are rejected:
```text
BIT: ERROR 20301 (HY000): invalid input: unsupported type for json aggregate: BIT
BINARY: ERROR 20203 (HY000): invalid argument aggregate function json_arrayagg, bad value [BINARY]
VARBINARY: ERROR 20203 (HY000): invalid argument aggregate function json_arrayagg, bad value [VARBINARY]
BLOB: ERROR 20203 (HY000): invalid argument aggregate function json_objectagg, bad value [VARCHAR BLOB]
```
## MySQL 8.0.45 result
MySQL accepts all four aggregate inputs. Representative values are:
```text
JSON_ARRAYAGG(b)
["base64:type16:qg==", "base64:type16:AA==", null]
JSON_ARRAYAGG(vb)
["base64:type15:QQBC", "base64:type15:", null]
JSON_OBJECTAGG(k, bl)
{"a":"base64:type252:QQBC","b":"base64:type252:","c":null}
```
The tags and payloads match the scalar MatrixOne/MySQL JSON constructors for the same source bytes.
## Scope and repeatability
- BIT(1), BIT(8), BINARY with embedded/zero bytes, empty VARBINARY/BLOB, and NULL were covered.
- Both `JSON_ARRAYAGG` and `JSON_OBJECTAGG` reproduce in 3/3 runs.
- Window aggregation, CTAS, views, and prepared statements are also blocked by the same aggregate type checks.
- ENUM and SET controls aggregate as strings identically in MatrixOne and MySQL.
- UUID and vector controls aggregate correctly in MatrixOne; Geometry aggregate input is rejected by both MatrixOne and MySQL and is not part of this compatibility report.
## Code analysis
`pkg/sql/plan/function/list_agg.go` explicitly rejects BINARY, VARBINARY, and BLOB for both JSON aggregates but admits BIT. The executor conversion functions in `pkg/sql/colexec/aggexec/jsonagg2.go` do not contain a BIT arm, so BIT reaches execution and fails there. Scalar JSON construction already routes these four types through the shared JSON typed-value conversion and emits the required opaque base64 representation.
## Expected behavior
`JSON_ARRAYAGG` and `JSON_OBJECTAGG` should accept BIT and binary-family values and serialize them with the same MySQL-compatible opaque JSON representation already used by MatrixOne's scalar JSON constructors. This should apply consistently to normal, window, CTAS/view, prepared, parallel/merge, and spill paths.
Contributor guide
Assessment
This issue has not been assessed yet.