apple / apple/foundationdb

Python: fdb network thread blocks main thread when gevent is monkey patched

Open
#228 2 comments 6 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
16.7k
Forks
1.6k
Avg merge
1d 20h
Merged PRs (30d)
126

Description

Hiya,

We make use of gevent's `monkey.patch_all` functionality in order to make all of the pure-Python libraries play nicely with gevent. This is pretty successful, but it currently causes the fdb network thread to block the main application thread.

Demo: https://gist.github.com/zorkian/824e86d0d7f001965f62dc8ae7ad28db

This is tested in gevent 1.1.1 as well as the latest 1.3b1.

You can alleviate the blocking by using `gevent.monkey.patch_all(thread=False)`, but this currently doesn't work in our environment due to certain modules we use that aren't implemented to use gevent and require threading.

I've been playing around with the fdb library to see if I can make it work, but haven't succeeded as yet. Figured I'd file the issue just to put it on the radar.

Contributor guide

Open the contributing guide

Research direction

Start with the linked demo and reproduce the blocking after gevent.monkey.patch_all. Trace the fdb network thread interaction with the main application thread, then verify that gevent compatibility works without disabling threading and that the main thread no longer blocks.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.