Performance improvement for os.scandir on Windows

Feature or enhancement

Proposal:

As has been mentioned in numerous issues (See #119169 for example) , there are quite a few performance issues regarding os.scandir and os.walk.


Diving into the implementation - see os_scandir_impl in posixmodule.c - I noticed that we are currently using WinAPI to list a directory. Looking into the relevant WinAPI functions (FindNextFileW, FindFirstFileW) it seems redundant to implement our Python wrapper around these wrappers. I propose to use the native NT functions - for example NtQueryDirectoryFile - directly, as we are already implementing a wrapper ourselves.


After quite a bit of reading MS-Docs and looking at Kernel32.dll & NtDll.dll, I have reached a stable implementation of this proposal. Unfortunately, I could not significantly reduce the amount of syscalls being performed, however I did reduce the amount of memory allocation and copying by a ratio of 1:6.
Additionally, this new implementation should aid in future implementations around Windows file system operations (See #99454 for any easy example).


This improves the performance of both os.scandir and os.walk which is implemented over it.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-122886

Beginne mit os_scandir_impl in Modules/posixmodule.c und prüfe den verknüpften PR gh-122886. Vergleiche die Windows-Implementierung der Verzeichnisauflistung mit den im Issue beschriebenen relevanten WinAPI- und NT-Funktionen und führe anschließend Benchmarks für os.scandir und os.walk durch. Als abgeschlossen gilt die Arbeit, wenn die Windows-Implementierung die Performance bei Allokationen und beim Kopieren verbessert, ohne das Verhalten zu ändern.