python-trio / python-trio/trio

Add api for watching for filesystem changes

Open
#1,449 3 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

low-level missing piece
Dominant language
Python
Stars
7.3k
Forks
431
Avg merge
2d 17h
Merged PRs (30d)
6

Description

There was a conversation on gitter about adding a filesystem watching api to trio, specifically in the context of Windows but it's something which would be of general utility.

watchdog was brought up as an example of an existing project implementing this functionality albeit not in a trio-native fashion.

@mozesa also linked to an excellent SO post on the topic:
https://stackoverflow.com/a/49888600

...and @njsmith gave some implementation advice:

if you do want to experiment with integrating it directly with the trio loop, the key functions are trio.hazmat.register_with_iocp(handle) and trio.hazmat.wait_overlapped(handle, lpOverlapped)

Here handle is a Win32 HANDLE object as a python integer (I think for ReadDirectoryChangesW this is the directory handle), and lpOverlapped is the address of an OVERLAPPED object as a python integer

So you'd use CreateFile to open the directory and get a handle object, then you'd pass that handle object to trio.hazmat.register_with_iocp, and then you'd create an OVERLAPPED object and pass it to ReadDirectoryChanges, and then pass the handle+overlapped object to trio.hazmat.wait_overlapped

For win32file, it has ways to create PyHandle and PyOverlapped objects that wrap the C-level HANDLE and OVERLAPPED objects, but what I don't know is whether it gives you a way to "unwrap" those and get the raw integers that Trio needs

I guess it would be reasonable to include a ReadDirectoryChangesW wrapper in trio itself... it's a pretty simple API. Something like async def read_directory_changes(path: str, watch_subtree: bool, notify_filter: IntFlags) -> AsyncIterator[DirectoryChangeEvent]
it would be a low-level platform-specific API, similar to trio.hazmat.readinto_overlapped, but that's OK...

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing trio.hazmat.register_with_iocp, trio.hazmat.wait_overlapped, and the proposed ReadDirectoryChangesW/read_directory_changes interface, then compare the watchdog example and linked Windows guidance. The issue does not name files or tests; completion criteria still need to be defined for a Trio-native filesystem-watching API and its platform scope.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, operating-systems
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.