enthought / enthought/comtypes

Challenges with the STA server message loop and firing events

Open
#565 3 comments 0 reactions 0 assignees View on GitHub
enhancement help wanted
Dominant language
Python
Stars
345
Forks
105
PR merge metrics
No merged PRs in 30d

Description

I use comtypes for making several local servers to provide plugins for another application. I have a lot of challenges working with the MessageLoop class (https://github.com/enthought/comtypes/blob/76c35421c3d8f41e97db861209b4f640b0061b0b/comtypes/messageloop.py#L18) for single threaded apartments, specifically for firing events.

The MessageLoop basically blocks on `GetMessage()` all the time, until a message comes in, which then gets handled, and we go back to blocking on `GetMessage()` again. This means that it is basically impossible to do anything on the main thread, except inside the handler for a COM method. My server design requires the client calling methods which initiate long running calculations on the server but return immediately to avoid the client blocking on the method call.

It is possible to partially relieve this problem by adding a second thread in the server - the main thread gets the method from COM, puts it in a queue for the second thread to pick up when ready, then returns immediately. The issue then comes when the second thread wants to signal something back to the client via an event. Because of the STA model, only the main thread is able to call `Fire_Event()` to deliver the event to the client, but because that is blocked by the call to `GetMessage()`, we cannot actually fire the event (unless we wait for an incoming method call from the client and use that to fire any pending events).

Running in MTA mode allows firing the event from the second thread but this has other consequences for thread safety and I would rather avoid it if possible. Therefore I am looking for suggestions on how we might make firing events from servers a more straightforward task. At the moment my solution is to have an intermediate "event-firing" server, which provides an interface of methods which can be called which immediately synchronously fire the corresponding event. An instance of this server is created by the client (which connects to its events) and passed to the primary server, which marshals it over to the secondary thread (via the GIT) which is then able to call its methods and fire events on the client. This works, but is obviously a sub-optimal design.

What I (currently) think would be nicest would be some way of getting in to the MessageLoop internally without a COM method call coming in. I think it would be really cool if we could interface the loop with cooperative multitasking like `asyncio`, then it would be easy to run other tasks on the main thread interspersed with handling COM method calls. A secondary option might be to decouple Fire_Event so that it can be called in a thread-safe manner - when a secondary thread calls Fire_Event it could put the event in a queue, then post a message in the message loop that would trigger a handler on the main thread to actually fire queued events.

I would also be very happy to hear any other suggestions or alternative approaches to solving this problem. Thanks for your input.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.