pydata / pydata/xarray

Add an ```only_variables``` or similar option to ```xarray.open_dataset``` and ```xarray.open_mfdataset```

Open
#8,074 7 comments 2 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
Python
Stars
4.2k
Forks
1.4k
Avg merge
2d 15h
Merged PRs (30d)
14

Description

Is your feature request related to a problem?

Sometimes, a variable in a nc file is corrupted or not "xarray friendly" and crashes opening a file (see for example https://github.com/pydata/xarray/issues/8072 ; I solved this on my machine by just drop_variablesing the problematic variables in practice), or reading and parsing the full file or mf-file may be expensive and time consuming, while only a couple of variables are needed.

Describe the solution you'd like

We already can exclude variables with the drop_variables arg to open_dataset (note: this is not present for now in open_mfdataset, should it be added there?), but could we also instead of saying "read all the variables instead of this list", be able to say "read only these variables"? In most case, this would be equivalent of using drop_variables=list(set(all_vars)-set(list_insteresting_vars), but in case some (many vars) may be corrupted, just getting the file opened to list these all_vars may be problematic.

Describe alternatives you've considered

drop_variables=list(set(all_vars)-set(list_insteresting_vars), but this is a lot more verbose.

Additional context

No response

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the xarray.open_dataset and xarray.open_mfdataset entry points and inspect how their existing drop_variables options are handled. Define the behavior of an only-variables option for both APIs, including files containing corrupted or expensive-to-parse variables; done means the requested variables can be opened without requiring the others.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.