google / google/weather-tools

`weather-mv`: Automatic database migrations

Open
#50 0 comments 0 reactions 0 assignees View on GitHub
P2 weather-mv
Dominant language
Python
Stars
252
Forks
60
Avg merge
8d 4h
Merged PRs (30d)
3

Description

As a user, when I run a newer version of `weather-mv` on a dataset, and the schema of the BigQuery database has changed (i.e. changed wrt to the dataset), I would like `weather-mv` to automatically migrate the table. I need this, as a user, so I don't my data pipelines don't break when new versions of `weather-mv` are released, and so I can get the latest tool features.

Last, as a user, I would like some control over weather migrations are applied, and what migrations to make. Ideally, the CLI should let me opt out of migrations if I know best. Otherwise, I want the tools to default to the recommended case.

Contributor guide

Open the contributing guide

Research direction

Start by locating the `weather-mv` CLI and its BigQuery schema handling. Define how schema changes and migration versions should be detected, how recommended migrations and opt-out controls should work, and how completion will preserve existing data pipelines while enabling newer tool features.

Written by the indexing model from the issue text.

Assessment

Tech stack
google-cloud, python
Domain
cli, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.