ApplETS / ApplETS/planifETS-backend
Refactor jobs pipeline to hydrate data from snapshots
Open
enhancement
high
- Dominant language
- TypeScript
- Stars
- 2
- Forks
- 1
- Avg merge
- 2h 30m
- Merged PRs (30d)
- 1
Description
### Current
Data hydration works as follows:
1. Fetch data from source inputs (PDFs, cheminements.txt, APIs)
2. Parse the data
3. Save it to database
### Goal
Change this pipeline so hydration happens from snapshots instead of directly from source inputs:
1. Save data as a snapshot
2. hydrate the backend from that snapshot only
This should make the data aggregation pipeline easier to maintain.
(Attributes `planificationPdfJson` and `horaireCoursPdfJson` was a good start but never fully implemented, remove it and create table.s containing snapshots)
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.