bcgov / bcgov/nr-rfc-admin

Documentation of Status / Architecture

Open
#37 0 comments 0 reactions 0 assignees View on GitHub
documentation
Dominant language
No language data
Stars
0
Forks
0
PR merge metrics
No merged PRs in 30d

Description

Since starting with the RFC have accomplished the following:
* Snowpack processing data pipeline
* Snowpack frontend
* Datamart AMQP Listener
* CMC data pipeline

Current release / mid term priority is:
* identification of inputs into COFFEE model
* Automation of all aspects of data input to COFFEE in a datapipeline
* Research / Implementation of orchestration tool - more below on this
* Implementation of push based automated COFFEE model data pipeline - Triggered based on Feds AMQP when data becomes available. Each data pipeline event is then run through orchestration. Events that are dependent on more than one dataset are triggered automatically when both data sets (or more) become available.

This ticket describes work invovled with creating diagrams and some documentation about:
* current status of transformation
* Plan for COFFEE pipeline
* Architecture Diagram / Docs

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by reviewing the RFC and the listed Snowpack, Datamart AMQP, and CMC pipeline components. Document the current transformation status, the planned COFFEE data pipeline, and the architecture, including diagrams that reflect the described orchestration and data dependencies.

Written by the indexing model from the issue text.

Assessment

Domain
data-engineering, distributed-systems, documentation
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.