matrixorigin / matrixorigin/matrixone

[Bug]: invalid UTF-8 bytes can be inserted into utf8mb4 varchar

Open
#25,302 1 comment 0 reactions 1 assignee Claimed by @daviszhen View on GitHub
ai-medium deferred kind/bug
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Bug Report

### Description

MatrixOne accepts invalid UTF-8 byte sequences into a `utf8mb4` `VARCHAR` column. MySQL rejects the same insert with `ERROR 1366 Incorrect string value`.

### Steps to Reproduce

```sql
drop table if exists t_invalid_utf8_probe;

create table t_invalid_utf8_probe (
id int primary key,
s varchar(10) character set utf8mb4,
b varbinary(10)
);

insert into t_invalid_utf8_probe values (1, unhex('C328'), unhex('C328'));

select id, hex(s) as s_hex, hex(b) as b_hex
from t_invalid_utf8_probe
order by id;
```

### MySQL Result

```text
ERROR 1366 (HY000): Incorrect string value: '\xC3(' for column 's' at row 1
```

No row is inserted.

### MatrixOne Result

Tested on latest `main` at `200bfa7e0efb`.

```text
id s_hex b_hex
1 C328 C328
```

### Expected Behavior

For `utf8mb4` text columns, MatrixOne should reject invalid UTF-8 byte sequences, matching MySQL. Binary columns such as `VARBINARY` may store the raw bytes.

### Notes

Found during active MySQL compatibility exploration, Module 4: String / Charset / Collation. I searched existing issues for invalid UTF-8 insert / incorrect string value and did not find a duplicate.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.