RealDevSquad / RealDevSquad/website-backend

[RFC] Update github_created_at key for existing user

Open
#1,453 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
JavaScript
Stars
74
Forks
276
Avg merge
1d 26m
Merged PRs (30d)
14

Description

Purpose:

Currently, we are storing the github_created_at key only for new users but we have to update the previous users also.

Limitation

Due to the large DB, we can not get all users at once. Also if we are trying to update data by calling GitHub API, we also get a rate limit error updating all users at once so I have to keep in mind that also.
GitHub rate limit:

Proposed solution:

First solution :
  • Create one route to update the single user in the database with github_created_at key in the Backend
  • After that for updating the user we call this route in three places (most visited user page)
    • Intro page section before loading and that time updating user DB and add a new button for refreshing after getting updated
    • Task page in both sections (status and my site). when the user loads a page
    • www website home page
      These only apply to that user who visits that page after changing that github_created_at key not present their DB
Second solution
  • updating all users at once
Challenges in this solution:
  • We can't get all users at once.
  • GitHub API limit can't perform all users at once, and calling API for every user throws a limit error.
  • This will be too large to update the user API call and get a limit exceed error.
Third solution
  • Cron job
    Basically, we write one API that runs in a batch, and inside that, we get users from DB ( 100 users at once who do not have that key) and will call GitHub API after some period of time, and this circular will continue till we get o uses.

I have not worked on anything like this before so I don't have a proper approach to solve this after suggestion. so please comment best approach you like

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the linked pull request #1451 and the three proposed approaches in this RFC. No implementation files or tests are named, so first identify the user data update path and existing scheduled-job conventions. Done means agreeing on an approach that updates existing users in bounded batches while respecting database size and GitHub API rate limits.

Written by the indexing model from the issue text.

Assessment

Tech stack
github, javascript, nodejs
Domain
api, backend, database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.