matrixorigin / matrixorigin/matrixone

[Bug]: regexp functions and operators accept binary string operands unlike MySQL

Open
#25,299 1 comment 0 reactions 1 assignee Claimed by @ck89119 View on GitHub
kind/bug severity/s1
Dominant language
Go
Stars
1.9k
Forks
311
Avg merge
1d 3h
Merged PRs (30d)
768

Description

## Bug Report

### Description

MatrixOne accepts statically typed binary/nonbinary operand combinations in `REGEXP`, `NOT REGEXP`, and the `REGEXP_*` functions that MySQL 8.4 rejects with `ERROR 3995`.

The compatibility rule is narrower than "reject binary REGEXP":

- static binary + nonbinary, in either operand order: reject with error 3995;
- binary + binary: valid and evaluated with byte semantics;
- ordinary untyped `NULL` + nonbinary: valid and returns `NULL`;
- a statically binary expression such as `CAST(NULL AS BINARY)` + nonbinary: reject with error 3995;
- a prepared parameter bound to a binary value + nonbinary: valid because MySQL exempts parameter markers from the static incompatibility check; execution must preserve binary semantics.

This is different from the existing case-insensitive collation issue. The defect is the missing static binary/nonbinary compatibility check before implicit regexp casts, while valid binary regexp inputs still require byte-oriented execution.

### Steps to Reproduce

```sql
-- Incompatible static binary/nonbinary combinations: should fail with 3995.
select _binary'abc' regexp 'a';
select 'abc' regexp _binary'a';
select regexp_like(_binary'abc', 'a');
select regexp_instr('abc', _binary'b');
select regexp_substr(_binary'abc123', '[0-9]+');
select regexp_replace(_binary'abc123', '[0-9]+', 'X');
select cast(null as binary) regexp 'a';

-- Valid binary/binary input: should use byte semantics.
select _binary'abc' regexp _binary'a';
select hex(regexp_substr(_binary'你好', _binary'.'));

-- Ordinary NULL is exempt from the static incompatibility check.
select null regexp 'a';
```

Prepared parameters must be tested separately from static expressions because MySQL treats parameter markers as an exception. A binary value bound to a prepared parameter with a nonbinary regexp operand is valid.

### MySQL 8.4.8 Result

```text
_binary'abc' REGEXP 'a' -> ERROR 3995
'abc' REGEXP _binary'a' -> ERROR 3995
_binary'abc' REGEXP _binary'a' -> 1
NULL REGEXP 'a' -> NULL
CAST(NULL AS BINARY) REGEXP 'a' -> ERROR 3995
HEX(REGEXP_SUBSTR(_binary'你好', _binary'.')) -> E4
prepared binary parameter REGEXP text -> valid
```

The `E4` result confirms that a valid binary/binary regexp operates on bytes rather than Unicode characters.

MySQL's documentation summarizes the restriction as rejecting binary regexp strings, but the 8.4 implementation checks for incompatible binary/nonbinary combinations and explicitly exempts parameter markers and ordinary `NULL` items:

- [MySQL 8.4 regular expression documentation](https://dev.mysql.com/doc/refman/8.4/en/regexp.html)
- [MySQL 8.4 `item_regexp_func.cc` compatibility check](https://github.com/mysql/mysql-server/blob/8.4/sql/item_regexp_func.cc#L93-L123)

### MatrixOne Result

Tested on `main` at `200bfa7e0efb`.

MatrixOne implicitly casts the incompatible static binary/nonbinary inputs and executes them instead of returning error 3995. The original observations included:

```text
bin_str_regexp_char_pat: 1
char_str_regexp_bin_pat: 1
regexp_like_bin_str: 1
regexp_like_bin_pat: 1
regexp_instr_bin_str: 2
regexp_substr_bin_str: 123
regexp_replace_bin_str: abcX
```

### Expected Behavior

MatrixOne should match MySQL's two-stage contract:

1. During binding/type resolution, reject incompatible static binary/nonbinary regexp operands with error 3995, before implicit conversion. Preserve the parameter-marker and ordinary-`NULL` exceptions.
2. During execution, evaluate valid binary regexp inputs with byte semantics and ordinary text inputs with character/ICU semantics.

The implementation should therefore combine the pre-conversion compatibility/error contract from PR #26724 with the binary metadata and byte-execution foundation from PR #26907, rather than merging two overlapping implementations unchanged or rejecting every binary regexp expression.

### Notes

Found during active MySQL compatibility exploration, Module 4: String / Charset / Collation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.