jaredly / jaredly/rex-json

Handle Unicode escapes

Open
#7 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
OCaml
Stars
46
Forks
12
PR merge metrics
No merged PRs in 30d

Description

Strings in JSON can contain escaped Unicode characters (i.e. \uXXXX) and the spec says that it's legal for any character to be escaped (even though technically you only need to escape a few of them).

The Java LSP client library (lsp4j) is particularly aggressive about escaping characters, so it immediately runs into this issue. For example, it sends:

```
{
"jsonrpc": "2.0",
"method": "textDocument/didOpen",
"params": {
"textDocument": {
"uri": "file:///Users/mdb/ops/langs/reason/a-reason-react-tutorial/src/Main.re",
"languageId": "re",
"version": 1,
"text": "ReactDOMRe.renderToElementWithId(\u003cTodoApp /\u003e, \"root\");\n"
}
}
}
```

(which escapes < and >) and this fails thusly:

```
Scanf.Scan_failure(\"scanf: bad input at char number 35: \\\"illegal escape character 'u'\\\"\")
```

I hacked the code to turn \u00XX into \xXX just to work around this for now (it would still fail on any non-ASCII escaped Unicode characters). I don't name my variables with UTF-8 smileys. 😁 But a proper decoding of Unicode escapes will be needed at some point.

Contributor guide

No contributing guide indexed for this repository

Research direction

No source file or test is named in the issue. Start by locating the JSON string escape decoder and reproduce the lsp4j example, including \u003c, \u003e, and a non-ASCII escape; done means valid \uXXXX escapes are decoded instead of rejected.

Written by the indexing model from the issue text.

Assessment

Tech stack
ocaml
Domain
backend
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.