Performance improvement for os.scandir on Windows

Feature or enhancement

Proposal:

As has been mentioned in numerous issues (See #119169 for example) , there are quite a few performance issues regarding os.scandir and os.walk.


Diving into the implementation - see os_scandir_impl in posixmodule.c - I noticed that we are currently using WinAPI to list a directory. Looking into the relevant WinAPI functions (FindNextFileW, FindFirstFileW) it seems redundant to implement our Python wrapper around these wrappers. I propose to use the native NT functions - for example NtQueryDirectoryFile - directly, as we are already implementing a wrapper ourselves.


After quite a bit of reading MS-Docs and looking at Kernel32.dll & NtDll.dll, I have reached a stable implementation of this proposal. Unfortunately, I could not significantly reduce the amount of syscalls being performed, however I did reduce the amount of memory allocation and copying by a ratio of 1:6.
Additionally, this new implementation should aid in future implementations around Windows file system operations (See #99454 for any easy example).


This improves the performance of both os.scandir and os.walk which is implemented over it.

Has this already been discussed elsewhere?

This is a minor feature, which does not need previous discussion elsewhere

Links to previous discussion of this feature:

No response

Linked PRs

• gh-122886

Inizia con os_scandir_impl in Modules/posixmodule.c ed esamina la PR collegata gh-122886. Confronta l’implementazione dell’elenco delle directory di Windows con le funzioni pertinenti di WinAPI e NT descritte nell’issue, quindi esegui benchmark di os.scandir e os.walk. Il lavoro è completato quando l’implementazione di Windows migliora le prestazioni di allocazione e copia senza modificare il comportamento.