Graylog2 / Graylog2/graylog2-server

Streams page calls unpaginated endpoint and loads all streams from database

Open
#21,543 4 comments 0 reactions 0 assignees View on GitHub
bug triaged
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## Expected Behavior

When the Streams page loads, it should only request a single page of streams from the backend API. This ensures that system resources and request time are minimized, and performance is maximized.

## Current Behavior

Currently, the Streams page makes a request to the `GET streams/` backend API endpoint, which loads all streams from the database, and performs additional business logic (permissions check and data translation). In one instance, we observed this request taking up to 2.5 seconds, which is quite a long time for a single request. To exacerbate the problem further, due to https://github.com/Graylog2/graylog2-server/issues/12318, the request was being executed multiple times.

See the backend endpoint here:f
https://github.com/Graylog2/graylog2-server/blob/be2931024a35fd475d9d999842741a0b29dcb3e2/graylog2-server/src/main/java/org/graylog2/rest/resources/streams/StreamResource.java#L278-L284

## Possible Solution

Investigate reworking the frontend Streams page to use the existing `streams/paginated` backend API endpoint. If we are missing any backend API capability needed for the frontend, we can look into adding that too.

## Steps to Reproduce (for bugs)

1. Load the main Streams page with the Network tab of the browser dev tools open.
2. Observe that the `GET streams/` API endpoint is executed.

![Image](https://github.com/user-attachments/assets/a5fdc79b-1ec2-4ac1-bb62-c04b62cf97f4)

![Image](https://github.com/user-attachments/assets/355fed25-6516-4746-bc94-509708889d6e)

## Additional info

I have not done any performance testing to reproduce slow loading times. But, this is issue being submitted, since this seems like an area we should improve, as the Streams collection has been known to grow in certain setups.

## Your Environment

* Graylog Version: Tested in 6.2-SNAPSHOT and 6.0.10.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.