mars-project / mars-project/mars

Warnings when running mars on ray container

Open
#2,709 8 comments 0 reactions 0 assignees View on GitHub
question
Dominant language
Python
Stars
2.7k
Forks
325
PR merge metrics
No merged PRs in 30d

Description

**Describe the bug**
There is a lot of warning when running mars on ray container with python 3.7.7

**To Reproduce**
1. Your Python version

python 3.7.7 in the ray docker.
The docker is created with the cmd below:
`docker run -it --shm-size=4.65gb -v /home/yizhengjiao/ray:/home/ray/ray --rm rayproject/ray:1.9.0`

2. The version of Mars you use
pymars-0.8.1

mars is installed with `pip install pymars`

3. Versions of crucial packages, such as numpy, scipy and pandas

Other packages are install with:
`pip install xgboost`
`pip install "xgboost_ray"`
`pip install lightgbm`

4. Full stack of the error.

base) ray@803dc5f4489d:~/ray/mars$ python boston_xgb.py
Web service started at http://0.0.0.0:41970
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100.0/100 [00:00<00:00, 584.26it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100.0/100 [00:00<00:00, 803.99it/s]
100%|██████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████████| 100.0/100 [00:00<00:00, 387.23it/s]
0%| | 0/100 [00:00

Contributor guide

Open the contributing guide

Research direction

Reproduce boston_xgb.py in the rayproject/ray:1.9.0 container with Python 3.7.7, then trace the shared_memory/resource_tracker warnings shown in the output. The issue does not define whether done means eliminating the warnings or documenting them as harmless, so that outcome needs clarification before implementation.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops, distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.