chaoss / chaoss/grimoirelab-hatstall

Hatstall scalability

Open
#13 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
HTML
Stars
11
Forks
16
PR merge metrics
No merged PRs in 30d

Description

The current hatstall design does not scale. When there are more than 10,000 identities in the SH database, the performance is not good enough. And with 100,000 identities is unusable. In this ticket a new design will be proposed and a plan for its implementation defined.

* Instead of loading all identities in SortingHat, only a limited number of identities will be loaded. Then, with pagination, the rest of the identities will be loaded.
* The list of identities to be loaded will be defined using some kind of searching/filtering, so in general, the pagination won't be done in all identities in SortingHat database but only in a subset.

This is just a proposal to be discussed and it will evolve to final design that will be implemented.

Contributor guide

Open the contributing guide

Research direction

No files, tests, or entry points are named. Start by reviewing how Hatstall currently loads identities from the SortingHat database, then define the proposed search/filtering and pagination design. Done means an agreed design and an implementation plan for scaling beyond 10,000 identities.

Written by the indexing model from the issue text.

Assessment

Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.