freeCodeCamp / freeCodeCamp/scripts

[feat]: ultimate normalization script

Open
#222 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
17
Forks
16
PR merge metrics
No merged PRs in 30d

Description

Going forward, there are likely many times we may want to normalize records. As such, we need a smart script:

**Musts**

- [ ] not bring down production to run script
- it should be a script we can tune to **not** hammer the db, and quietly run in the background
- [ ] easy to define/add transformations
- [ ] fault tolerant
- [ ] keep track of versioning
- must not repeat work - the script should be stopped/started without hesitation, and start off where it left off
- [ ] fearlessly concurrent
- the api will write a `lastUpdated` field to the user record which should allow all ops to be done without losing data

**Niceties**

- [ ] dynamically asks database how loaded it is, and accordingly adjusts ops/rps

**Notes**

Currently, the idea is to add a `schemaVersion` field to the `user` record.

I would prefer to write this in Rust:
1) Ease of distribution
2) Strict typing

That said, this might go against _"easy to define/add transformations"_.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.