Logging to stdout from anyio worker processes
- Dominant language
- Python
- Stars
- 2.5k
- Forks
- 260
- Avg merge
- 1d 8h
- Merged PRs (30d)
- 17
Description
We wanted to use `anyio.to_process.run_sync` in order to execute CPU intensive code in worker processes, and we want to emit logs from the code executed by the worker processes. AnyIO uses the stdout/stderr of the worker processes for internal purposes to communicate with the parent process, which means we need another way to write to main process stdout from the worker process without race conditions, or an alternative way to write logs from inside Docker.
**Alternatives examined without changes to AnyIO**
1. os.dup the parent process stdout file descriptor and pass that file descriptor to the worker process. This will not work as expected since there can be race conditions if both the parent process and worker process write logs at the same time to the parent's stdout even via different file descriptors.
2. Write to Docker logs without using writing to stdout/stderr - we didn't find a way to do it, if you know any that will be appreciated.
**Change proposal to AnyIO**
Instead of hard-coding stdout/stdin as the communication mechanism between parent process and child process, allow AnyIO users to opt into another piping mechanism so stdout can still be used in the child process for e.g. logging purposes inside Docker.
Contributor guide
Assessment
This issue has not been assessed yet.