pingcap / pingcap/tidb

the behavior of invalid utf8 string in json value is incompatible with MySQL

Open
#58,886 0 comments 0 reactions 1 assignee Claimed by @YangKeao View on GitHub
component/json severity/minor sig/sql-infra type/bug
Dominant language
Go
Stars
40.5k
Forks
6.2k
PR merge metrics
PR metrics pending

Description

## Bug Report

Please answer these questions before submitting your issue. Thanks!

### 1. Minimal reproduce step (Required)

```
func TestJSONInvalidUTF8(t *testing.T) {
store := testkit.CreateMockStore(t)
tk := testkit.NewTestKit(t, store)

tk.MustQuery(`SELECT JSON_EXTRACT(?, "$.key")`, "{\"key\":\"\xe0\"}").Check(testkit.Rows("\"\xe0\""))
}
```

MySQL returns `"\xe0"` in this case. The json parser can handle the case when the JSON string is not an valid utf8. The TiDB turns it into `0xFFFD`. It's acceptable.

### 2. What did you expect to see? (Required)

Returns `\xe0` for this SQL and arguments.

### 3. What did you see instead (Required)

Returns `\xFFFD` for this SQL.

### 4. What is your TiDB version? (Required)

```
c199ddfcdf9e1f88a243c2b7847d8f3224446004
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.