dmlc / dmlc/dmlc-core

How to make sure the number of workes per core are being used?

Open
#581 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
878
Forks
526
Avg merge
12d 19h
Merged PRs (30d)
1

Description

I am using _dmlc-submit_ to submit the job like this:

_./dmlc-core/tracker/dmlc-submit --cluster=local **--num-workers=X --worker-cores=Y** python myscript.py_

It seems the parameters **--num-workers** and **--worker-cores** are not working.
When I launch my script with **--num-workers=1**, the %CPU seen through the Linux _top_ command is much bigger than 100%.
When I change only the _Y_ variable, the runtime difference does not change as I expected. Also I didn't find under dmlc-core code where **--worker-cores** is being used. Any idea?

Also, given I am using a python script, how do you track the details of a distributed script (e.g., number of tasks per workers, etc.)? I know that by using _top_ command I can see the number of workers but I don't know how to get extra information.

Thanks!

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.