bcgov / bcgov/ols-geocoder

Data prep - Steps D3 & D4

Open
#344 0 comments 0 reactions 0 assignees View on GitHub
data prep scripts geocoder
Dominant language
Java
Stars
17
Forks
7
Avg merge
10h 2m
Merged PRs (30d)
15

Description

Update BC Address Geocoder Data Processing to eliminate step D3 [CPF batch GSR unit recovery] and decouple step D4 [FME site prep] as shown in the [data flow 2 diagram](https://github.com/bcgov/ols-geocoder/blob/gh-pages/address-data-pipeline.md).

1) Test the elimination of step D3 from the Geocoder data prep process to confirm that the impact on the overall occupants list is minimal. For example, if a unit only record is provided, but we have a parent address, will it still be imported?

2) If impact will be minimal, eliminate step D3 from BC Address Geocoder Data Processing.

3) Once step D3 is eliminated, step D4 can be decoupled and simplify the process. Since this step would no longer require pre-geocoding, it could be run independently and only needed when new site data is provided (don’t need to run after ITN updates).

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the BC Address Geocoder Data Processing flow and the linked data flow 2 diagram, focusing on steps D3 and D4. Test whether removing D3 changes the occupants list, including the unit-only record case with a parent address. Done means D3 is eliminated if impact is minimal and D4 runs independently only when new site data is provided.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
data-engineering
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.