Error when adding a DataArray to an existing Dataset with a MultiIndex

What is your issue?

This is a mixture between question, bug (potentially) and general issue, so feel free to label it accordingly.

Here is my question: what is the recommended approach to add a xr.DataArray to an existing xr.Dataset with a MultiIndex?

To give some more context, I've a xarray.Dataset called market with several variables and coordinates, one of them, timeslice, a MultiIndex. This is what it looks like:

<xarray.Dataset>
Dimensions:       (region: 1, commodity: 6, timeslice: 6, year: 8)
Coordinates:
  * region        (region) object 'R1'
  * commodity     (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind'
    units_prices  (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt'
  * timeslice     (timeslice) object MultiIndex
  * month         (timeslice) object 'all-year' 'all-year' ... 'all-year'
  * day           (timeslice) object 'all-week' 'all-week' ... 'all-week'
  * hour          (timeslice) object 'night' 'morning' ... 'late-peak' 'evening'
  * year          (year) int64 2020 2025 2030 2035 2040 2045 2050 2055
Data variables:
    prices        (commodity, region, year, timeslice) float64 0.0702 ... 0.0
    exports       (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0
    imports       (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0
    static_trade  (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0

Now, I want to add another variable, called supply, identical to exports but filled with zeros. In a code that was working with xarray==2022.3.0 and pandas==1.4.4, I was simply doing:

market["supply"] = xr.zeros_like(market.exports)

And it worked totally fine. With the newest versions of xarray==2023.5.0 and pandas==2.0.2 under python 3.10, this fails with:

*** DeprecationWarning: Deleting a single level of a MultiIndex is deprecated. Previously, this deleted all levels of a MultiIndex. Please also drop the following variables: {'timeslice'} to avoid an error in the future.

I've tried variants like:

market["supply"] = market.exports * 0
market = market.assign(supply = zeros_like(market.exports))

both failing with the same message.

I totally fail to see how this process is deleting a level of the MultiIndex - or modifying the indexes in any form. Probably it is because I don't understand the inner workings of xarray indexes.

The following works totally fine, but it is rather convoluted having to create a brand new Dataset from scratch manually, in addition to be problematic if you really want to modify the Dataset in place (same problem will have assign).

vars = dict(market.data_vars)
vars["supply"] = xr.zeros_like(market.exports)
market = xr.Dataset(vars)

Resulting in:

<xarray.Dataset>
Dimensions:       (region: 1, commodity: 6, timeslice: 6, year: 8)
Coordinates:
  * region        (region) object 'R1'
  * commodity     (commodity) object 'electricity' 'gas' ... 'CO2f' 'wind'
    units_prices  (commodity) object 'MUS$2010/GWh' ... 'MUS$2010/kt'
  * timeslice     (timeslice) object MultiIndex
  * month         (timeslice) object 'all-year' 'all-year' ... 'all-year'
  * day           (timeslice) object 'all-week' 'all-week' ... 'all-week'
  * hour          (timeslice) object 'night' 'morning' ... 'late-peak' 'evening'
  * year          (year) int64 2020 2025 2030 2035 2040 2045 2050 2055
Data variables:
    prices        (commodity, region, year, timeslice) float64 0.0702 ... 0.0
    exports       (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0
    imports       (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0
    static_trade  (timeslice, commodity, region, year) float64 0.0 0.0 ... 0.0
    supply        (commodity, region, year, timeslice) float64 0.0 0.0 ... 0.0

Many thanks for your support!

Start by reproducing the assignment with xarray.zeros_like on the displayed Dataset under xarray 2023.5.0, pandas 2.0.2, and Python 3.10, then compare it with the older versions and the manual xr.Dataset construction. Trace the assignment and MultiIndex handling involved in market["supply"] and market.assign; done means adding the zero-filled variable no longer emits the deprecation warning or requires rebuilding the Dataset.