bcgov / bcgov/Mines-Data-Platform
D3: Develop Production Ingestion Pipelines with Medallion Architecture
- Dominant language
- HCL
- Stars
- 0
- Forks
- 1
- Avg merge
- 24m
- Merged PRs (30d)
- 1
Description
**What**
Build fully functional Medallion Architecture pipelines (Bronze/Silver/Gold) to ingest core mines data into Fabric. Ensure data shareability for external stakeholders outside the BC government.
**Why**
This will enable efficient data processing, reconciliation, and controlled sharing, ensuring compliance and enhancing data usability for decision-making.
## Acceptance Criteria
Ensure successful data reconciliation and version control across all pipeline stages.
Implement automated schedules for data ingestion and processing.
Develop robust error handling and logging mechanisms.
Integrate on-premise databases into the pipeline architecture.
Validate data shareability functionality for external stakeholders.
**Story Points:** 13
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.