hypothesis / hypothesis/h

Long annotation bodies or quotes (approx >32KB) can cause annotation indexing to fail

Open
#4,751 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
3.2k
Forks
459
Avg merge
27d 1h
Merged PRs (30d)
1

Description

This issue was split out from https://github.com/hypothesis/h/issues/4741 .

If an annotation contains a (stored?) field whose length exceeds 32KB, this can cause annotation indexing to fail. The result is that creating / updating the annotation will succeed, but the annotation will not appear in search results and therefore not show up in the client or activity pages in subsequent requests.

https://sentry.io/hypothesis/h/issues/242530914/

The most common fields this appears to occur in are the annotation's body and quote (`target.selector.exact`).

```
h.tasks.indexer.add_annotation failed with RequestError: TransportError(400, u'RemoteTransportException[[Stegron][inet[/x.x.x.x:y]][indices:data/write/index]]; nested: IllegalArgumentException[Document contains at least one immense term in field="_all" (whose UTF8 encoding is longer than the max length 32766))
```

Original report: https://github.com/hypothesis/h/issues/4741#issuecomment-358996944

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.