ApplETS / ApplETS/planifETS-backend

Refactor jobs pipeline to hydrate data from snapshots

Open
#80 0 comments 0 reactions 0 assignees View on GitHub
enhancement high
Dominant language
TypeScript
Stars
2
Forks
1
Avg merge
2h 30m
Merged PRs (30d)
1

Description

### Current
Data hydration works as follows:
1. Fetch data from source inputs (PDFs, cheminements.txt, APIs)
2. Parse the data
3. Save it to database

### Goal
Change this pipeline so hydration happens from snapshots instead of directly from source inputs:
1. Save data as a snapshot
2. hydrate the backend from that snapshot only

This should make the data aggregation pipeline easier to maintain.

(Attributes `planificationPdfJson` and `horaireCoursPdfJson` was a good start but never fully implemented, remove it and create table.s containing snapshots)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.