bcgov / bcgov/Mines-Data-Platform

D3: Develop Production Ingestion Pipelines with Medallion Architecture

Open
#415 6 comments 0 reactions 1 assignee Claimed by @deepaktibhe-byte View on GitHub
Dominant language
HCL
Stars
0
Forks
1
Avg merge
24m
Merged PRs (30d)
1

Description

**What**
Build fully functional Medallion Architecture pipelines (Bronze/Silver/Gold) to ingest core mines data into Fabric. Ensure data shareability for external stakeholders outside the BC government.

**Why**
This will enable efficient data processing, reconciliation, and controlled sharing, ensuring compliance and enhancing data usability for decision-making.

## Acceptance Criteria

Ensure successful data reconciliation and version control across all pipeline stages.
Implement automated schedules for data ingestion and processing.
Develop robust error handling and logging mechanisms.
Integrate on-premise databases into the pipeline architecture.
Validate data shareability functionality for external stakeholders.

**Story Points:** 13

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.