Hangs with pytest with 0.2.3 (Thread pb ?)

Switching from 0.2.2 to 0.2.3 makes mongo client hanging for closing.
Time go from 1s to 32 s for one test

**To Reproduce**

Run following test with 0.2.2 then 0.2.3 (pytest)

```
def test_mongo():

with MongoClient() as client:
# Given
db = client['toolbox']
tm : Collection = db.tm

# When
oid = tm.insert_one({'name': 'foo', 'kind': 'bar'}).inserted_id

# Then
assert oid is not None
retrieved = tm.find_one({'_id': oid})
assert retrieved is not None
assert retrieved['name'] == 'foo'

# When
tm.delete_one({'_id': oid})
assert tm.find_one({'_id': oid}) is None
```

**Expected behavior**
Avoid hanging

**Logs**
0.2.2

```
time pytest tests/unit/persistance/test_inmemory_mongo.py
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/cram/Workdir/IAS/MAJIS/majis_tm_toolbox, configfile: pytest.ini
plugins: mock-3.6.1, integration-0.2.2
collected 1 item

tests/unit/persistance/test_inmemory_mongo.py::test_mongo
-------------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------------
WARNING PYMONGOIM_DOWNLOADER:__init__.py:214 Starting from MongoDB 4.0.23 there isn't a generic Linux version of MongoDB
PASSED [100%]

================================================================================================ warnings summary =================================================================================================
tests/unit/persistance/test_inmemory_mongo.py::test_mongo
/home/cram/Workdir/IAS/MAJIS/majis_tm_toolbox/.venv/lib/python3.8/site-packages/pymongo_inmemory/downloader/__init__.py:214: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn((

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================================== 1 passed, 1 warning in 0.44s ===========================================================================================

real 0m1,460s
user 0m1,004s
sys 0m0,634s
(majis_tm_toolbox) cram@cram-Precision-M6800:~/Workdir/IAS/MAJIS/majis_tm_toolbox$
```

0.2.3

```
time pytest tests/unit/persistance/test_inmemory_mongo.py
=============================================================================================== test session starts ===============================================================================================
platform linux -- Python 3.8.10, pytest-6.2.4, py-1.10.0, pluggy-0.13.1
rootdir: /home/cram/Workdir/IAS/MAJIS/majis_tm_toolbox, configfile: pytest.ini
plugins: mock-3.6.1, integration-0.2.2
collected 1 item

tests/unit/persistance/test_inmemory_mongo.py::test_mongo
-------------------------------------------------------------------------------------------------- live log call --------------------------------------------------------------------------------------------------
WARNING PYMONGOIM_DOWNLOADER:__init__.py:191 Starting from MongoDB 4.0.23 there isn't a generic Linux version of MongoDB
PASSED [100%]

================================================================================================ warnings summary =================================================================================================
tests/unit/persistance/test_inmemory_mongo.py::test_mongo
/home/cram/Workdir/IAS/MAJIS/majis_tm_toolbox/.venv/lib/python3.8/site-packages/pymongo_inmemory/downloader/__init__.py:191: DeprecationWarning: The 'warn' method is deprecated, use 'warning' instead
logger.warn((

-- Docs: https://docs.pytest.org/en/stable/warnings.html
========================================================================================== 1 passed, 1 warning in 31.84s ==========================================================================================

real 0m32,818s
user 0m1,541s
sys 0m0,499s

```

**Screenshots**
Profiles generated with pycharm profilings tools

**With 0_2_2**

**With 0_2_3**

**Context:**
- OS: Linux Ubuntu
- Version of pymongo_inmemory 0.2.3
- Version of mongo you are downloading 4.0.3
- Any other additional information about context