RealDevSquad / RealDevSquad/website-backend
[RFC] Update github_created_at key for existing user
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 74
- Forks
- 276
- Avg merge
- 1d 26m
- Merged PRs (30d)
- 14
Description
Purpose:
Currently, we are storing the github_created_at key only for new users but we have to update the previous users also.
Limitation
Due to the large DB, we can not get all users at once. Also if we are trying to update data by calling GitHub API, we also get a rate limit error updating all users at once so I have to keep in mind that also.
GitHub rate limit:
- Unauthenticated Requests: 60 requests per hour for most endpoints.
- Authenticated Requests: 5000 requests per hour for most endpoints
PR I have worked and got suggestions to change the approach: https://github.com/Real-Dev-Squad/website-backend/pull/1451
Proposed solution:
First solution :
- Create one route to update the single user in the database with github_created_at key in the Backend
- After that for updating the user we call this route in three places (most visited user page)
- Intro page section before loading and that time updating user DB and add a new button for refreshing after getting updated
- Task page in both sections (status and my site). when the user loads a page
- www website home page
These only apply to that user who visits that page after changing that github_created_at key not present their DB
Second solution
- updating all users at once
Challenges in this solution:
- We can't get all users at once.
- GitHub API limit can't perform all users at once, and calling API for every user throws a limit error.
- This will be too large to update the user API call and get a limit exceed error.
Third solution
- Cron job
Basically, we write one API that runs in a batch, and inside that, we get users from DB ( 100 users at once who do not have that key) and will call GitHub API after some period of time, and this circular will continue till we get o uses.
I have not worked on anything like this before so I don't have a proper approach to solve this after suggestion. so please comment best approach you like
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the linked pull request #1451 and the three proposed approaches in this RFC. No implementation files or tests are named, so first identify the user data update path and existing scheduled-job conventions. Done means agreeing on an approach that updates existing users in bounded batches while respecting database size and GitHub API rate limits.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- github, javascript, nodejs
- Domain
- api, backend, database
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100