dotnet / dotnet/wpf

HwndWrapper leaks class registrations (ATOMs returned by RegisterClassEx)

Open
#9,026 15 comments 1 reaction 0 assignees View on GitHub
Investigate
Dominant language
C#
Stars
7.7k
Forks
1.3k
Avg merge
1d 11h
Merged PRs (30d)
61

Description

### Description

The implementation of `MS.Win32.HwndWrapper` leaks Win32 Windows class registrations.

`MS.Win32.HwndWrapper` in its constructor calls the Win32 API [`RegisterClassEx` function](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-registerclassexa) (and then creates a window with it using [`CreateWindowEx`](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-createwindowexa)).

It holds unmanaged resource, so it also implements the Dispose pattern, and in its `internal Dispose(bool, bool)` it will schedule the corresponding [`UnregisterClass` function](https://learn.microsoft.com/en-us/windows/win32/api/winuser/nf-winuser-unregisterclassa). But it will only do that if the second parameter to `Dispose(bool, bool)`, called `isHwndBeingDestroyed`, is true. And `Dispose(bool, bool)` is called in two locations, the ordinary `Dispose()` method that implements `IDisposable`, and in the finalizer. But in both calls, the second parameter is `false`. `isHwndBeingDestroyed` is **never** true. The class will **never** be unregistered.

As it happens, the ATOM returned by `RegisterClassEx` is is from the [User Atom Table](https://learn.microsoft.com/en-us/windows/win32/dataxchg/about-atom-tables#user-atom-table) for which there is no useful API, so it is not possible to inspect this atom table or maybe search for old class registrations lying around. It also appears to be the case that these class registrations persist even when the process exits. This means that a WPF application that creates a `HwndWrapper` anywhere slowly eats up the available Windows class registrations. And only 16,384 are available in total. If an application unwittingly manages to exhaust them, this renders the operation system unusable. No window can be created anymore, not a Task Manager to shut down the process, not a Shutdown dialog with the option to shut down or restart the machine. This is fatal.

The `User Atom Table` is not really documented (or I haven't found the documentation), so it is not clear to me if it gets cleared when the user logs out, or only when the system restarts. It appears to definitely not get cleared when the process exits, judging from the effect it can have. From the name 'UserAtomTable' it sounds like it should be cleared when logging out.

### Reproduction Steps

**Warning**. If you want to reproduce this, be prepared to restart your operating system afterwards, or at least log off. Have a command window open already to do that. Best try this only in a VM.

`HwndWrapper` is used in various parts of WPF. Here is a code snippet that demonstrates the effect, I have tried it:

```
class MyViewModel
{
private Dispatcher _dispatcher;
public MyViewModel()
{
_dispatcher = Dispatcher.CurrentDispatcher;
Restart();
}

/// .. some properties here to display something while running ....

private void Restart()
{
var thread = new Thread(MakeDispatcher);
thread.Start();
}

private void MakeDispatcher()
{
try
{
var dispatcher = Dispatcher.CurrentDispatcher;

int threadId = dispatcher.Thread.ManagedThreadId;
_dispatcher.Invoke(() =>
{
/// note this happens on the UI thread. This is not relevant to the example, is just here to show how to make this display someting.
// update some properties here
});
Restart();
}
catch (Win32Exception e)
{
_dispatcher.Invoke(() =>
{
Exception = $"{e.Message} ({e.NativeErrorCode})";
});
}
}
}

```
When this comes to its Win32 exception, the system's class atom's are exhausted, and you better restart the machine (or log out). At the very least, close some windows.

You may ask what the real-world scenario is here, so why one should deliberately call `Dispatcher.CurrentDispatcher` on lots of different threads.
* In fact, our production code did not call `Dispatcher.CurrentDispatcher` but `CommandManager.InvalidateRequerySuggested()` which internally calls `Dispatcher.CurrentDispatcher`.
* We were on lots of different threads because some code wanted to update an UI button and forgot it was not on the UI thread but a random thread from a background process that it was being called on.

### Expected behavior

Expected behavior would be to not to leak any class handles. Since class handles are invisible and the User Atom Table cannot be inspected, there is no obvious symptom at all except that the system will be unresponsive all of a sudden:

* If one WPF application that accidentally does this runs for a long time, or
* If the system is not restarted for a long time, and one WPF application that accidentally does this gets used a lot, even if it is closed down in between.

### Actual behavior

The actual behavior is described above. If a WPF application manages to exhaust the available class registrations, it will get the error "Not enough memory resources are available to process this command" (Native error 8). But in addition, this makes the operation system completely unusable. No window can be created anymore. Not even the Task Manager, nor the Performance Monitor, nor the Shutdown Dialog can be created unless some other window is closed. If a user doesn't have the wits to control-tab to some other window, close it, and then start an orderly shutdown or logout, all they can do is hard-reset the system, i.e. use the power button to switch the computer off.

### Regression?

From my code inspections, this is the case in .Net Framework 4.7.2 as well as the current code from dotnet core, and for sure all other versions in between.

### Known Workarounds

If the depletion of windows class handles is very rapid, it will usually be because of an error. Normally it should not be necessary to use `Dispatcher.CurrentDispatcher` on lots of different threads. It should not even be necessary to create many threads (`new Thread()`) in the first place. Even if someone makes a mistake and calls `Dispatcher.CurrentDispatcher` on a background thread, if this thread is from the threadpool (`Task.Run()`), the number of lost class handles will be limited to at most the number of threadpool threads.

But there is no working around that the handles stay leaked even when the process exits, and so the only workaround is to restart the system occasionally.

### Impact

Here is a story of some of the impact this had.

https://stackoverflow.com/questions/78319645/long-running-process-with-wpf-front-end-on-a-windows-ipc-fails-with-not-enough

On an actual machine out there in the real world, this failed a company's production process in progress, made some of their products unusable, and harmed the reputation of the manufacturer of the machine.

### Configuration

As stated above, this code seems to be the same from at least .Net Framework 4.7.2 to now.
Operating system will be Windows, and it is a matter of the Win32 API, which underpins all versions of Windows AFAIK. any architecture. "Not enough memory resources are available to process this command" is an error of the Win32 API when `RegisterClassEx` fails. The problem will be happening until either WPF starts cleaning up the class registrations correctly, or Win 32 will start cleaning them up when a process exits.

### Other information

https://referencesource.microsoft.com/#q=HwndWrapper

https://github.com/dotnet/wpf/blob/main/src/Microsoft.DotNet.Wpf/src/Shared/MS/Win32/HwndWrapper.cs

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.