bcgov / bcgov/entity

Update pipeline to support Pension Fund Society Amalgamation Application Data

Open
#25,761 0 comments 0 reactions 0 assignees View on GitHub
COBRS
Dominant language
JavaScript
Stars
23
Forks
62
Avg merge
24m
Merged PRs (30d)
1

Description

This ticket is to update the pipeline to support Pension Fund Society Amalgamation Application Data.

This will include:
* Updating raw sql queries to pull in required data from COBRS db extract
* Updating code to generating filing json for given filing
* Update custom filer code to process generated filing json
* Testing pipeline against a decent sample size to see what errors come up

When updating the pipeline to support a given filing, the initial testing will be done locally against a COBRS db extract.

In our overall migration testing, all migration of prod COBRS corp data will be tested. Under this comprehensive testing, various data issues and edge cases will be discovered for various filings. We will cut tickets at this time to address the issues that come up.

NOTE: Before data can be loaded into an environment, there must be a robust facility to delete that data from the environment into which it was loaded.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the pipeline's raw SQL queries, filing JSON generation, and custom filer entry points, then run the local pipeline against a COBRS database extract. Done means the required Pension Fund Society Amalgamation Application Data is retrieved, represented in filing JSON, processed by the custom filer, and tested against a meaningful sample, including data deletion before loading.

Written by the indexing model from the issue text.

Assessment

Tech stack
sql
Domain
backend, data-engineering, databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.