agronholm / agronholm/anyio

Logging to stdout from anyio worker processes

Open
#516 5 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
2.5k
Forks
260
Avg merge
1d 8h
Merged PRs (30d)
17

Description

We wanted to use `anyio.to_process.run_sync` in order to execute CPU intensive code in worker processes, and we want to emit logs from the code executed by the worker processes. AnyIO uses the stdout/stderr of the worker processes for internal purposes to communicate with the parent process, which means we need another way to write to main process stdout from the worker process without race conditions, or an alternative way to write logs from inside Docker.

**Alternatives examined without changes to AnyIO**

1. os.dup the parent process stdout file descriptor and pass that file descriptor to the worker process. This will not work as expected since there can be race conditions if both the parent process and worker process write logs at the same time to the parent's stdout even via different file descriptors.
2. Write to Docker logs without using writing to stdout/stderr - we didn't find a way to do it, if you know any that will be appreciated.

**Change proposal to AnyIO**
Instead of hard-coding stdout/stdin as the communication mechanism between parent process and child process, allow AnyIO users to opt into another piping mechanism so stdout can still be used in the child process for e.g. logging purposes inside Docker.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.