Concatenate datasets when some variables are present in one dataset and not present in other dataset
- Dominant language
- Python
- Stars
- 164
- Forks
- 54
- PR merge metrics
- No merged PRs in 30d
Description
Currently, `xarray.concat()` doesn't know how to deal with datasets for which one dataset contains some variables and these same variables are not present in the rest of the datasets to concatenate.
To address this issue, there are two options:
- Add NaNs by determining the set of variables that are not in all datasets but at least in one dataset
and use .full_like(..., fill_value=np.nan) to add the variables not present in the other dataset (
- Dropping the problematic variables
Either option would probably need to be done prior calling `xr.concat()` in `join_new()` and `join_existing()` introduced in #135
Contributor guide
Research direction
Start in join_new() and join_existing(), introduced in #135, and trace how they call xarray.concat(). Reproduce the case where variables exist in only some datasets, then verify the chosen behavior for those variables—adding NaNs or dropping them—is consistent across both entry points.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- data
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100