Support glob patterns in `open_datatree(group=...)` for selective group loading
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 4.2k
- Forks
- 1.4k
- Avg merge
- 2d 15h
- Merged PRs (30d)
- 14
Description
Description
When working with large hierarchical datasets, users often need only a subset of groups. Currently open_datatree(group=...) accepts a single literal path to re-root the tree. This proposal extends group to accept glob patterns (e.g., */sweep_0), filtering which groups are opened without loading the entire tree first.
Use cases
Radar data (NEXRAD): Volume scan files contain dozens of sweep groups per VCP. To analyze only the lowest elevation scan across all volumes:
dt = xr.open_datatree("radar.nc", group="*/sweep_0")
Climate model output (CMIP): Multi-model archives store data in deeply nested hierarchies like /{model}/{experiment}/{variable}. To load only temperature from all models under a specific experiment:
dt = xr.open_datatree("cmip.zarr", group="*/historical/tas")
Or to compare two specific variables across all models:
dt = xr.open_datatree("cmip.zarr", group="*/historical/ta[su]")
Proposed API
When group contains glob metacharacters (*, ?, [), it switches from root-selection mode to filter mode. Matching uses the same engine as DataTree.match() (PurePosixPath.match). Root (/) and all ancestors of matched nodes are always included to form a valid tree.
Behavior summary
group value |
Behavior |
|---|---|
None |
Load all groups (unchanged) |
"VCP-34" (no glob chars) |
Root selection (unchanged) |
"*/sweep_0" (glob chars) |
Filter mode — only matched groups + ancestors |
| Pattern matches nothing | Root-only tree |
Reference
PR #10742 (async DataTree open) provides the base for this work.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start at the open_datatree API and inspect DataTree.match() for its PurePosixPath.match behavior. Review PR #10742 for the async DataTree open foundation, then verify that glob patterns load matched groups and their ancestors, while literal groups and None retain existing behavior and nonmatching patterns produce a root-only tree.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- backend-api-design, data-engineering
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100