pull request files collection could be more efficient
- Dominant language
- Python
- Stars
- 13
- Forks
- 17
- Avg merge
- 6h 59m
- Merged PRs (30d)
- 1
Description
We might be able to squeeze a lot more performance from our api calls in PR files collection
Sonnet 4.6 has identified that we are constructing one graphql call per pull request:
https://github.com/chaoss/CollectOSS/blob/96adf3a4d68725db21622673ee6613693c0f5ace/collectoss/tasks/github/pull_requests/files_model/core.py#L52-L69
Some of these may return very few files.
Other places in the code already batch multiple queries into one graphQL call:
https://github.com/chaoss/CollectOSS/blob/96adf3a4d68725db21622673ee6613693c0f5ace/collectoss/tasks/frontend.py#L207-L212
We should do the same in pull request files to help make things faster (less waiting for network calls)
Contributor guide
Research direction
Start with collectoss/tasks/github/pull_requests/files_model/core.py at lines 52-69, then compare the batching approach in collectoss/tasks/frontend.py at lines 207-212. Confirm how pull request files are collected and identify the current per-pull-request GraphQL calls. Done means equivalent requests are batched to reduce network waits while preserving the collected files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- graphql, python
- Domain
- data-engineering, performance
- Issue type
- Refactor
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 68/100