mlcommons / mlcommons/mlcube

GPUs parameter doesn't seem to be working

Open
#307 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
160
Forks
31
PR merge metrics
No merged PRs in 30d

Description

When I try to run a basic MLCube passing the --gpus parameter, I get an error mlcube.errors.ConfigurationError: Unknown keys: ['--gpus']. Namespace = runner.

I've tried different ways of passing the parameter and value: --gpus="all", --gpus=all, --gpus all, --gpus=1, --gpus="1" just in case that would affect, but I get the same error regardless.

I also tried specifying the accelerator_count to different values but nothing changed.

Full traceback:

HelloWorld/data_preparator/mlcube % mlcube run --task=prepare --gpus="all" -Pdocker.build_strategy=always
Traceback (most recent call last):
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/bin/mlcube", line 8, in <module>
    sys.exit(cli())
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/click/core.py", line 1130, in __call__
    return self.main(*args, **kwargs)
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/click/core.py", line 1055, in main
    rv = self.invoke(ctx)
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/click/core.py", line 1657, in invoke
    return _process_result(sub_ctx.command.invoke(sub_ctx))
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/click/core.py", line 1404, in invoke
    return ctx.invoke(self.callback, **ctx.params)
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/click/core.py", line 760, in invoke
    return __callback(*args, **kwargs)
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/click/decorators.py", line 26, in new_func
    return f(get_current_context(), *args, **kwargs)
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/mlcube/__main__.py", line 272, in run
    runner_cls, mlcube_config = parse_cli_args(
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/mlcube/cli.py", line 60, in parse_cli_args
    mlcube_config = MLCubeConfig.create_mlcube_config(
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/mlcube/config.py", line 158, in create_mlcube_config
    runner_cls.CONFIG.validate(mlcube_config)
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/mlcube_docker/docker_run.py", line 100, in validate
    _ = validator.check_unknown_keys(Config.DEFAULT.keys())\
  File "/Users/alejandroaristizabal/opt/anaconda3/envs/medperf/lib/python3.9/site-packages/mlcube/validate.py", line 78, in check_unknown_keys
    raise ConfigurationError(f"Unknown keys: {unknown_keys}.{self._namespace_msg()}")
mlcube.errors.ConfigurationError: Unknown keys: ['--gpus']. Namespace = runner.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Reproduce the command from the report, then start in mlcube/cli.py at parse_cli_args and follow configuration creation into mlcube_docker/docker_run.py, where validation rejects the key. Done means the documented --gpus usage is accepted without the Unknown keys error and GPU configuration reaches the runner as intended.

Written by the indexing model from the issue text.

Assessment

Tech stack
docker, python
Domain
devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.