Materials-Consortia / Materials-Consortia/optimade-python-tools
Rethink default MongoDB count strategy
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 91
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
#2247 changed the default MongoDB count timeout to 0.5 seconds, which means it will usually fail quickly for any sizable query. This has limited utility in the case where the user actually just wants to know the number of results without pagination. I suggest a hybrid strategy where the MongoDB timeout is allowed to be quite large for the first page of results, but then is very small for all subsequent pages. This should make pagination significantly faster, and savvy clients can just cache the value of the number of results from the first response they get.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reading issue #2247 and locating the MongoDB count and pagination implementation in the repository. Trace how the first page and subsequent pages are identified, then determine how the proposed timeout policy should apply. Done means the strategy is implemented with coverage for both first-page and later-page behavior.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- mongodb, python
- Domain
- backend-api-design, databases
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100