RealDevSquad / RealDevSquad/website-backend
Deliverables, ETAs, Milestones - Rate Limting Middleware | Parent Ticket |
Open
@bajajcodes is already working on this.
Since May 9, 2023.
documentation
- Dominant language
- JavaScript
- Stars
- 74
- Forks
- 276
- Avg merge
- 1d 26m
- Merged PRs (30d)
- 14
Description
Problem
- The Express.js backend API code is missing a rate-limiting feature, which can lead to server overload, slow response times, and poor user experience.
- The absence of rate-limiting can cause issues such as excessive requests, automated bots abuse, and security threats like brute-force attacks.
- The consequences of missing rate-limiting are numerous, including system vulnerability to DDoS attacks, poor API performance, and reduced user functionality and trust.
Solution
- Our proposed solution is to implement a custom rate-limiting middleware using the "node-rate-limiter-flexible" package, which can be easily integrated with the existing Express.js backend.
- The middleware will enforce rate limits on all incoming requests to the API, preventing abuse and improving overall performance.
- We will use a Redis Hash database to store rate-limiting rules and deploy the application on the "railway.app" platform to ensure scalability and reliability.
- Different rate-limiting rules will be set for users with different roles, and IP address resolution will be implemented to resolve public IP and proxy IP issues. (and more)
Info:
- For More Check the Design Document, referenced below.
Deliverables
| Feature Name | Feature ETA | Feature Status | Feature Description | Feature PR/ Discussion | Prod Ready |
|---|---|---|---|---|---|
| Rate Limiting Rules: Research, Analysis, and Identifying Acceptable Rates | 14/05/2023 | IN_PROGRESS | This feature involves researching, identifying, and coming up with rate-limiting rules for common rate limiters and a few routes for testing and debugging while developing. Already build solutions references will be taken for identifying how they came up with the solution. | Proposed: Using newrelic to come up with rate limit parameters and their values |
N |
| Route-Based Rate Limiting: Common Rate Limiter Middleware | 18/05/2023 | IN_PROGRESS | This feature involves: setting up the node-rate-limiter-flexible package, using in-process memory store(temp), getting rate limiting rules from variables stored in process memory (temp). |
PR: Setting Up Rate Limitier | N |
| Route-Based Rate Limiting: Route Specific Rate Limiter Middleware | 24/05/2023 | IDLE | This involves implementing the solution for multiple route-based rate limiters, switching between rate limiters based on the request and again using in-process memory store and environment variables temporarily. | -- | N |
| Redis: Database for Storing Rate Limiting Rules | 30/05/2023 | IDLE | Setting up Redis hash database for storing rate limiting rules, config helper functions/classes for creating rules following the template. | -- | N |
| Redis: Notifying Backend and Rate Limiter Middleware Of Changes | 03/06/2023 | IDLE | Setting up Redis pub/sub for notifying backend and rate-limiter middleware on create/update/delete of rate limiting rules and redeploying the route-based rate-limiters. | -- | N |
| Redis: Cache Service for Storing Rate Limiter Middleware Store(s) | 11/06/2023 | IDLE | Setting up Redis cache service as a memory store for rate limiters stores(route-based rate limiting) for maintaining consistency across different instances if in case auto-horizontal-scaled by deployment-service(railway.app) provider. | -- | N |
| Client Unique Identifier and Builder: IP Address Resolution | 16/06/2023 | IDLE | Helper functions/class to identify and resolve client-unique IP addresses behind a public IP. | -- | N |
| Client Unique Identifier and Builder: IP Address Resolution | 22/06/2023 | IDLE | Helper functions/class to identify and resolve client-unique IP addresses behind proxy IP. Using headers x-forwarded-for, port, and subnetting. |
-- | N |
| Client Unique Identifier and Builder: UserId Resolution | 26/06/2023 | IDLE | Helper functions/class to identify and resolve client-unique userId. | -- | N |
| Client Unique Identifier and Builder: Miscellaneous | 30/06/2023 | IDLE | Helper function/class to identify and resolve client-unique-identifier(cuid or client-uid) based on a parameter named type (ipaddress, userId, both) passed. |
-- | Y |
| Role-Based Rate Limiting: Identifying Acceptable Rate | 05/07/2023 | IDLE | Defining and Identifying, different rate-limiting rules for users with different roles, such as app_owner, super_user, member, and user. |
-- | N |
| Role-Based Rate Limiting: Implementation | 08/07/2023 | IDLE | Implementing Different rate-limiting rules for users with different roles, such as app_owner, super_user, member, and user. |
-- | Y |
| Dashboard Site: Design For Configuring Rate Limiting Rules | To Be Calculated |
IDLE | To Be Added |
-- | N |
| Dashboard Site: Design/UI Implementation | To Be Calculated |
IDLE | To Be Added |
-- | N |
| Dashboard Site: CRUD operations Implementation and Integrating Redis Service, Backend and Rate Limiting Middleware | To Be Calculated |
IDLE | To Be Added |
-- | Y |
[Note:]
- Feature ETAs are based on the date to push PR for review.
- Time To Review, Change and Approve is considered 2-3 in the best situation. Hence, the next ETAs are calculated and may be subject to change.
[Info]:
- Prod Ready Meaning: Usable current and previous features in production.
References
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.