Graylog2 / Graylog2/graylog2-server

Handling large message/result set sizes in search

Open
#11,785 6 comments 0 reactions 0 assignees View on GitHub
Low Prio triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

When retrieving and displaying a large result set of messages (as in 150*n mb) during search, we need to make sure that:

- We do not exceed the configured http client buffer size (resulting in `entity content is too long [...] for the configured buffer limit [...]` exceptions otherwise, where the default is 100Mb)
- We do not overload the browser during rendering

Potential strategies could include:

- Always retrieve truncated summaries for messages when fetching result sets for multiple messages
- Retrieving complete messages only when showing message details and only on demand (when expanding a message)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.