mars-project / mars-project/mars

[BUG] ConnectionRefusedError: [Errno 111] Connection refused

Open
#2,766 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
2.7k
Forks
325
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
When using mars web client, if we cancel a task, then executing a new task ,we got `Connection refused.`
**To Reproduce**
To help us reproducing this bug, please provide information below:
1. Your Python version: python3.7.9
2. The version of Mars you use: mars 0.8.1
3. Versions of crucial packages, such as numpy, scipy and pandas: numpy==1.17.3 scipy==1.5.0 pands==1.3.0
4. Full stack of the error.
```
---------------------------------------------------------------------------
ConnectionRefusedError Traceback (most recent call last)
/tmp/ipykernel_14654/104550561.py in
1 df = df.astype({'idx': 'string','trd_latitude':'float','trd_longitude':'float'})
----> 2 df.execute()

~/miniconda3/lib/python3.7/site-packages/mars/core/entity/tileables.py in execute(self, session, **kw)
460
461 def execute(self, session=None, **kw):
--> 462 result = self.data.execute(session=session, **kw)
463 if isinstance(result, TILEABLE_TYPE):
464 return self

~/miniconda3/lib/python3.7/site-packages/mars/core/entity/executable.py in execute(self, session, **kw)
96
97 session = _get_session(self, session)
---> 98 return execute(self, session=session, **kw)
99
100 def _check_session(self, session: SessionType, action: str):

~/miniconda3/lib/python3.7/site-packages/mars/deploy/oscar/session.py in execute(tileable, session, wait, new_session_kwargs, show_progress, progress_update_interval, *tileables, **kwargs)
1796 show_progress=show_progress,
1797 progress_update_interval=progress_update_interval,
-> 1798 **kwargs,
1799 )
1800

~/miniconda3/lib/python3.7/site-packages/mars/deploy/oscar/session.py in execute(self, tileable, show_progress, *tileables, **kwargs)
1589 try:
1590 execution_info: ExecutionInfo = fut.result(
-> 1591 timeout=self._isolated_session.timeout
1592 )
1593 except KeyboardInterrupt: # pragma: no cover

~/miniconda3/lib/python3.7/concurrent/futures/_base.py in result(self, timeout)
433 raise CancelledError()
434 elif self._state == FINISHED:
--> 435 return self.__get_result()
436 else:
437 raise TimeoutError()

~/miniconda3/lib/python3.7/concurrent/futures/_base.py in __get_result(self)
382 def __get_result(self):
383 if self._exception:
--> 384 raise self._exception
385 else:
386 return self._result

~/miniconda3/lib/python3.7/site-packages/mars/deploy/oscar/session.py in _execute(session, wait, show_progress, progress_update_interval, cancelled, *tileables, **kwargs)
1750 t._attach_session(session)
1751
-> 1752 execution_info = await session.execute(*tileables, **kwargs)
1753 execution_info.add_done_callback(_attach_session)
1754 cancelled = cancelled or asyncio.Event()

~/miniconda3/lib/python3.7/site-packages/mars/deploy/oscar/session.py in execute(self, *tileables, **kwargs)
952 task_name=task_name,
953 fuse_enabled=fuse_enabled,
--> 954 extra_config=extra_config,
955 )
956

~/miniconda3/lib/python3.7/site-packages/mars/services/task/api/web.py in submit_tileable_graph(self, graph, task_name, fuse_enabled, extra_config)
224 method="POST",
225 headers={"Content-Type": "application/octet-stream"},
--> 226 data=body,
227 )
228 return res.body.decode()

~/miniconda3/lib/python3.7/site-packages/mars/services/web/core.py in _request_url(self, method, path, wrap_timeout_exception, **kwargs)
194 try:
195 res = await self._client.fetch(
--> 196 path, method=method, raise_error=False, **kwargs
197 )
198 except HTTPTimeoutError as ex:

ConnectionRefusedError: [Errno 111] Connection refused
```

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reproducing the cancel-then-execute sequence through deploy/oscar/session.py, then trace the request from services/task/api/web.py into services/web/core.py using the reported stack. Done means a task can be cancelled and a subsequent task executes without ConnectionRefusedError.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
api, backend, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.