Add a mode for initial ingestion of event data
Offen
DE
enhancement
Pramen-Scala
- Vorherrschende Sprache
- Scala
- Sterne
- 31
- Forks
- 4
- Ø Merge
- 1 T. 10 Min.
- Gemergte PRs (30 T.)
- 4
Beschreibung
## Background
When an event table is ingested initially, the history can be quite long.
Executing each event date one by one can have a big overhead and execute many queries against the source database.
## Feature
Add a mode for initial ingestion of event data.
## Example
TBD
## Proposed Solution
What we can do is load full table in one go to a temporary directory (cache), and then run pipeline tasks for each date found in the database.
Beitragsleitfaden
Für dieses Repository ist kein Beitragsleitfaden indexiert
Bewertung
Dieses Issue wurde noch nicht bewertet.