cftime default's datetime breaks CFTimeIndex

What happened:
With cftime 1.2.0, one can create datetime object with cftime.datetime(*args, calendar='calendar'), instead of using one of the subclasses (ex cftime.DatetimeNoLeap(*args)). In the latest release (1.4.0, yesterday), the subclasses have been deprecated, but kept as legacy. While all xr code still works (it is using the legacy subclasses), the CFTimeIndex object relies on the type of the datetime object in order to infer the calendar. If the datetime was created outside xarray, using the now default constructor, the returned type is not understood and CFTimeIndexbreaks.

What you expected to happen:
I expected CFTimeIndex to be independent of the way the datetime object is created.

Minimal Complete Verifiable Example:

import cftime
import numpy as np
import xarray as xr

# A datetime array, not created in xarray
time = cftime.num2date(np.arange(365), "days since 2000-01-01", calendar="noleap")
a = xr.DataArray(np.zeros(365), dims=('time',), coords={'time': time})

a.indexes['time']

Fails with :

Traceback (most recent call last):
  File "<stdin>", line 1, in <module>
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/coding/cftimeindex.py", line 342, in __repr__
    attrs_str = format_attrs(self)
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/coding/cftimeindex.py", line 264, in format_attrs
    attrs["freq"] = f"'{index.freq}'" if len(index) >= 3 else None
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/coding/cftimeindex.py", line 692, in freq
    return infer_freq(self)
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/coding/frequencies.py", line 96, in infer_freq
    inferer = _CFTimeFrequencyInferer(index)
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/coding/frequencies.py", line 105, in __init__
    self.values = index.asi8
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/coding/cftimeindex.py", line 673, in asi8
    [
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/coding/cftimeindex.py", line 674, in <listcomp>
    _total_microseconds(exact_cftime_datetime_difference(epoch, date))
  File "/home/phobos/Python/xclim/.tox/py38/lib/python3.8/site-packages/xarray/core/resample_cftime.py", line 370, in exact_cftime_datetime_difference
    seconds = b.replace(microsecond=0) - a.replace(microsecond=0)
  File "src/cftime/_cftime.pyx", line 1153, in cftime._cftime.datetime.__sub__
ValueError: cannot compute the time difference between dates with different calendars

Anything else we need to know?:

Environment:

Reproduce the example with cftime 1.4.0 and inspect xarray/coding/cftimeindex.py, xarray/coding/frequencies.py, and xarray/core/resample_cftime.py at the traceback locations. Check how CFTimeIndex infers calendars for datetime objects created with cftime.datetime, then run the relevant xarray tests and verify the example no longer raises a calendar-mismatch error.