bcgov / bcgov/nr-rfc-admin

Create GHA to injest the ZXS data

Open
#40 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Pull this data from the federal government datamart and push it up to object store. For location in object store, try to emulate the existing directory structure used on prem.

The trigger for this will be a cron based trigger, using the same timing as the existing job. Subsequent tasks will transition it to an event based trigger using the same patter created for the CMC pipeline

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the existing on-prem job to confirm its schedule and directory structure, then review the CMC pipeline pattern for the intended GitHub Actions approach. Done means a cron-triggered workflow pulls ZXS data from the federal government datamart and writes it to object storage using the matching structure.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions
Domain
cloud, data-engineering, devops
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.