AgnostiqHQ / AgnostiqHQ/covalent-awslambda-plugin

Not working with current stable Covalent

Open
#92 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
8
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Covalent version: 0.220.0.post2
Python: 3.8

Following error is thrown:

```python
[ERROR] OSError: [Errno 38] Function not implemented
Traceback (most recent call last):
File "/var/task/exec.py", line 58, in handler
raise ex
File "/var/task/exec.py", line 44, in handler
function, args, kwargs = pickle.load(f)
File "/var/task/covalent/__init__.py", line 22, in
from . import executor, leptons # nopycln: import
File "/var/task/covalent/executor/__init__.py", line 34, in
from ..quantum import QCluster, Simulator
File "/var/task/covalent/quantum/__init__.py", line 17, in
from .qcluster import QCluster, Simulator
File "/var/task/covalent/quantum/qcluster/__init__.py", line 21, in
from .clusters import QCluster
File "/var/task/covalent/quantum/qcluster/clusters.py", line 21, in
from .base import AsyncBaseQCluster, BaseQExecutor
File "/var/task/covalent/quantum/qcluster/base.py", line 22, in
from mpire.async_result import AsyncResult
File "/var/task/mpire/__init__.py", line 3, in
from mpire.pool import WorkerPool
File "/var/task/mpire/pool.py", line 17, in
from mpire.async_result import (AsyncResult, AsyncResultType, AsyncResultWithExceptionGetter,
File "/var/task/mpire/async_result.py", line 7, in
from mpire.comms import EXIT_FUNC, INIT_FUNC
File "/var/task/mpire/comms.py", line 10, in
from mpire.params import WorkerMapParams
File "/var/task/mpire/params.py", line 13, in
from mpire.tqdm_utils import get_tqdm
File "/var/task/mpire/tqdm_utils.py", line 102, in
class TqdmManager:
File "/var/task/mpire/tqdm_utils.py", line 104, in TqdmManager
LOCK = Lock()
File "/usr/local/lib/python3.8/multiprocessing/context.py", line 68, in Lock
return Lock(ctx=self.get_context())
File "/usr/local/lib/python3.8/multiprocessing/synchronize.py", line 162, in __init__
SemLock.__init__(self, SEMAPHORE, 1, 1, ctx=ctx)
File "/usr/local/lib/python3.8/multiprocessing/synchronize.py", line 57, in __init__
sl = self._semlock = _multiprocessing.SemLock(
```

This is because `mpire` does this at import time. Thus even patching or overloading the `sys.modules` doesn't work. Ideally, it would be better if we don't need to update the core covalent because of this AWS Lambda specific error and only modifying this plugin is sufficient somehow.

Contributor guide

Open the contributing guide

Research direction

No repository file or test is named. Reproduce the failure with Covalent 0.220.0.post2 on Python 3.8 in AWS Lambda, then trace the plugin's import path through covalent and mpire, especially the multiprocessing Lock creation shown in the traceback. Done means the plugin can be imported and run in Lambda without the OSError or requiring a core Covalent change.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, python
Domain
cloud
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.