Potential performance issue: Unreliable performance of df.ewm in pandas 1.2.2

**Issue Description:**

Hello.
I have discovered a performance degradation in the df.ewm function of pandas version 1.2.2 when ewm passed with times and it is so slow compared to ewm pased with int. And I notice that `requirements.txt`, shows that it depends on pandas version 1.2.2. I am not sure whether this performance problem in pandas will affect this repository. I found some discussions on GitHub related to this issue, including [#39784](https://github.com/pandas-dev/pandas/issues/39784) and [#40072](https://github.com/pandas-dev/pandas/pull/40072).
I also found that `06_rnns.ipynb` used the influenced api. There may be more files used the influenced api.

**Suggestion**

I would recommend considering an upgrade to a different version of pandas > 1.2.2 or exploring other solutions to optimize the performance.
Any other workarounds or solutions would be greatly appreciated.
Thank you!