Running on .NET Core / .NET 5.0 exhibits huge memory leak when resizing, and uses alot more memory in general

In Avalonia we have had a similar problem, and one of our users tried to compare Avalonia and WPF and found that WPF also has the same/similar issue.

See https://github.com/AvaloniaUI/Avalonia/issues/5646 for details.

* .NET Core Version: 5.0.3
* Windows version: Windows 10
* Does the bug reproduce also in WPF for .NET Framework 4.8?: No
* Is this bug related specifically to tooling in Visual Studio (e.g. XAML Designer, Code editing, etc...)? If yes, please file the issue via the instructions [here](https://docs.microsoft.com/visualstudio/ide/how-to-report-a-problem-with-visual-studio?view=vs-2019). no.
* Security issues and bugs should be reported privately, learn more via our [responsible disclosure guidelines](https://github.com/dotnet/wpf/blob/master/README.md#reporting-security-issues-and-security-bugs).

**Problem description:**
For example, WPF:
**WPF IDLE**

**WPF After Resizing**

**Actual behavior:**

**Expected behavior:**

**Minimal repro:**
[AvaloniaTest.zip](https://github.com/AvaloniaUI/Avalonia/files/6118107/AvaloniaTest.zip) (File says Avalonia, but contains WPF repros also)