freeCodeCamp / freeCodeCamp/scripts
[feat]: ultimate normalization script
- Dominant language
- TypeScript
- Stars
- 17
- Forks
- 16
- PR merge metrics
- No merged PRs in 30d
Description
Going forward, there are likely many times we may want to normalize records. As such, we need a smart script:
**Musts**
- [ ] not bring down production to run script
- it should be a script we can tune to **not** hammer the db, and quietly run in the background
- [ ] easy to define/add transformations
- [ ] fault tolerant
- [ ] keep track of versioning
- must not repeat work - the script should be stopped/started without hesitation, and start off where it left off
- [ ] fearlessly concurrent
- the api will write a `lastUpdated` field to the user record which should allow all ops to be done without losing data
**Niceties**
- [ ] dynamically asks database how loaded it is, and accordingly adjusts ops/rps
**Notes**
Currently, the idea is to add a `schemaVersion` field to the `user` record.
I would prefer to write this in Rust:
1) Ease of distribution
2) Strict typing
That said, this might go against _"easy to define/add transformations"_.
Contributor guide
Research direction
No files, tests, or entry points are named. Start by clarifying the database and API interfaces, the proposed user.schemaVersion behavior, and how transformations are defined and versioned. Done should include resumable, fault-tolerant, concurrency-safe execution that can be throttled without repeated work or production impact.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend, databases, tooling
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 20/100