MetPX / MetPX/sr_insects

Dynamic flow: should we change to `subtopic *.WXO-DD.#`?

Open
#79 0 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Shell
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

We have https://github.com/MetPX/sr_insects/blob/main/dynamic_flow/config/shovel/no_trouble_f00.inc in dynamic flow that rejects known problematic datasets (e.g. password protected radar data).

Some of this data could be ignored by adding `subtopic *.WXO-DD.#`, now that we have WXO-DD data separated into its own subdirectory.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with dynamic_flow/config/shovel/no_trouble_f00.inc and inspect how the dynamic flow handles WXO-DD data in its own subdirectory. Determine whether adding `subtopic *.WXO-DD.#` safely excludes the intended problematic datasets, then validate the resulting flow behavior or relevant test cases.

Written by the indexing model from the issue text.

Assessment

Tech stack
shell
Domain
stream-processing
Issue type
Feature
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.