apache / apache/gluten

[CH] get_json_object parse result different from spark when have special chars like `\uD835\uDE90\uD835`

Open
#3,500 2 comments 0 reactions 0 assignees View on GitHub
bug triage
Dominant language
Scala
Stars
1.6k
Forks
657
Avg merge
2d 14h
Merged PRs (30d)
80

Description

### Backend

CH (ClickHouse)

### Bug description

Somethings when the json string contains special chars like `\uD835\uDE90\uD835`, which is actually Korean or chinese, when use get json object , the result is different from gluten and spark

by gluten, the result is
![image](https://github.com/oap-project/gluten/assets/37431499/d6d4a68a-9cab-4ab5-839d-948b5d0891cb)

by spark, the result is
![image](https://github.com/oap-project/gluten/assets/37431499/0d84e7c0-1236-4f44-8276-5c29a0be905c)

### Spark version

Spark-3.3.x

### Spark configurations

_No response_

### System information

_No response_

### Relevant logs

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.