hypothesis / hypothesis/client

Error when saving if 32-char prefix/suffix of quote selector ends mid-way through a unicode surrogate pair

Open
#5,028 2 comments 0 reactions 0 assignees View on GitHub
bug New Contributor Friendly
Dominant language
Mustache
Stars
730
Forks
224
PR merge metrics
No merged PRs in 30d

Description

**Steps to reproduce:**

1. On https://github.com/typescript-eslint/typescript-eslint/discussions/6014, activate the extension and try to annotate exactly the text "no-inferrable-types" in the left column of a table
2. Try to save the annotation

**Expected result:** Annotation is saved
**Actual result:** 500 server error from h

There are two issues here:

1. From inspecting the client's store (set `window.debug = true`) we can see that the `TextQuoteSelector` of the new annotation contains a string that ends mid-way through a UTF-16 surrogate pair (see the `suffix` field):

TextQuoteSelector invalid unicode

2. In h, the `POST /api/annotations` API fails when attempting to store the JSON blob in Postgres: https://sentry.io/organizations/hypothesis/issues/2551504044/?project=37293&query=is%3Aunresolved&referrer=issue-stream

```
InvalidTextRepresentation: invalid input syntax for type json
LINE 1: ...cript-eslint/typescript-eslint/discussions/6014', '[{"type":...
^
DETAIL: Unicode low surrogate must follow a high surrogate.
CONTEXT: JSON data, line 1: ...ud83e\uddf1\n\ud83d\udfe9\n\n\n\n\n\n", "suffix":...
```

There are two fixes needed here:

1. The client shouldn't be submitting selectors with invalid Unicode to the server
3. h shouldn't crash with an internal server error. It should either fail with a 4xx error, or silently fix up the invalid Unicode (eg. by ignoring the isolated Unicode high surrogate)

A workaround for users is to change the text selection slightly so that the `prefix` and `suffix` end at a slightly different point.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.