Build the OSM Great Britain addresses dataset

Type

Task

Scope

Addresses

Skillset

engineering

Description

This work is part of our MVP extensions project. It's mean to show users what the process for a supplemental dataset looks like as distinct from the schema extension process.

Steps

• One-time pull, no pipeline for the MVP
• Pull GB addresses from Overpass or a planet file and save as GeoJSON
• Convert to the Overture address schema using the existing code in the data platform repo
• Run through staging to assign IDs
• Write to Parquet and publish

Decisions for supplemental data creator

• Partitioning. You do not have to match Overture's existing partitioning. Pick what makes sense and say why.
• Whether fork of the staging code is needed to run this one dataset in isolation.
• Open question to record rather than solve: how does the addresses pipeline partition today? Worth documenting alongside guidance on what a user SHOULD keep in mind when splitting a dataset across files.