aws / aws/sagemaker-rl-container

does ray torch version image not supported with python3.7?

Open
#76 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
82
Forks
44
PR merge metrics
No merged PRs in 30d

Description

conda env:
(python3) [ec2-user@ip-172-16-92-179 sagemaker-rl-container]$ python3
Python 3.10.14 | packaged by conda-forge | (main, Mar 20 2024, 12:45:18) [GCC 12.3.0] on linux
Type "help", "copyright", "credits" or "license" for more information.

docker build cli:
docker build -t torch-ray:cpu-py3 -f ray/docker/0.8.5/Dockerfile.torch --build-arg processor=cpu .

base iamge:
FROM 763104351884.dkr.ecr.us-west-2.amazonaws.com/pytorch-training:1.5.0-cpu-py36

build error with dm-tree install:
`Attempting uninstall: protobuf
Found existing installation: protobuf 3.15.6
Uninstalling protobuf-3.15.6:
Successfully uninstalled protobuf-3.15.6
Running setup.py install for dm-tree: started
Running setup.py install for dm-tree: finished with status 'error'
ERROR: Command errored out with exit status 1:
command: /opt/conda/bin/python3.6 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ioqppij4/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.6m/dm-tree
cwd: /tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/
Complete output (55 lines):
running install
running build
running build_py
creating build
creating build/lib.linux-x86_64-3.6
creating build/lib.linux-x86_64-3.6/tree
copying tree/__init__.py -> build/lib.linux-x86_64-3.6/tree
copying tree/sequence.py -> build/lib.linux-x86_64-3.6/tree
copying tree/tree_benchmark.py -> build/lib.linux-x86_64-3.6/tree
copying tree/tree_test.py -> build/lib.linux-x86_64-3.6/tree
warning: build_py: byte-compiling is disabled, skipping.

running build_ext
cmake version 3.5.1

CMake suite maintained and supported by Kitware (kitware.com/cmake).
Found CMake
CMake Error at CMakeLists.txt:3 (cmake_minimum_required):
CMake 3.12 or higher is required. You are running version 3.5.1


-- Configuring incomplete, errors occurred!
Traceback (most recent call last):
File "", line 1, in
File "/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py", line 155, in
keywords='tree nest flatten',
File "/opt/conda/lib/python3.6/site-packages/setuptools/__init__.py", line 163, in setup
return distutils.core.setup(**attrs)
File "/opt/conda/lib/python3.6/distutils/core.py", line 148, in setup
dist.run_commands()
File "/opt/conda/lib/python3.6/distutils/dist.py", line 955, in run_commands
self.run_command(cmd)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/site-packages/setuptools/command/install.py", line 61, in run
return orig.install.run(self)
File "/opt/conda/lib/python3.6/distutils/command/install.py", line 545, in run
self.run_command('build')
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/opt/conda/lib/python3.6/distutils/command/build.py", line 135, in run
self.run_command(cmd_name)
File "/opt/conda/lib/python3.6/distutils/cmd.py", line 313, in run_command
self.distribution.run_command(command)
File "/opt/conda/lib/python3.6/distutils/dist.py", line 974, in run_command
cmd_obj.run()
File "/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py", line 72, in run
self.build_extension(ext)
File "/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py", line 109, in build_extension
['cmake', ext.source_dir] + cmake_args, cwd=self.build_temp)
File "/opt/conda/lib/python3.6/subprocess.py", line 311, in check_call
raise CalledProcessError(retcode, cmd)
subprocess.CalledProcessError: Command '['cmake', '/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/tree', '-DPython3_ROOT_DIR=/opt/conda', '-DPython3_EXECUTABLE=/opt/conda/bin/python3.6', '-DCMAKE_LIBRARY_OUTPUT_DIRECTORY=/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/build/lib.linux-x86_64-3.6', '-DCMAKE_BUILD_TYPE=Release', '-DPython3_LIBRARY=/opt/conda/lib/python3.6', '-DPython3_INCLUDE_DIR=/opt/conda/include/python3.6m']' returned non-zero exit status 1.
----------------------------------------
ERROR: Command errored out with exit status 1: /opt/conda/bin/python3.6 -u -c 'import io, os, sys, setuptools, tokenize; sys.argv[0] = '"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"'; __file__='"'"'/tmp/pip-install-phvm04kk/dm-tree_95eb886d3d904877b744f174998bf67d/setup.py'"'"';f = getattr(tokenize, '"'"'open'"'"', open)(__file__) if os.path.exists(__file__) else io.StringIO('"'"'from setuptools import setup; setup()'"'"');code = f.read().replace('"'"'\r\n'"'"', '"'"'\n'"'"');f.close();exec(compile(code, __file__, '"'"'exec'"'"'))' install --record /tmp/pip-record-ioqppij4/install-record.txt --single-version-externally-managed --compile --install-headers /opt/conda/include/python3.6m/dm-tree Check the logs for full command output.
The command '/bin/sh -c pip install --no-cache-dir Cython==0.29.7 tabulate tensorboardX gputil gym==0.12.1 lz4 opencv-python-headless==4.1.0.25 PyOpenGL==3.1.0 pyyaml redis==3.3.2 ray==0.8.5 ray[tune]==0.8.5 ray[rllib]==0.8.5 scipy psutil setproctitle dm-tree tensorflow-probability' returned a non-zero code: 1`

Contributor guide

Open the contributing guide

Research direction

Start with ray/docker/0.8.5/Dockerfile.torch and the pytorch-training:1.5.0-cpu-py36 base image, then reproduce the listed docker build command. Investigate the dm-tree failure where CMake 3.5.1 does not meet the required 3.12 minimum. Done means the image builds successfully with the documented dependency installation.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, docker, python, pytorch
Domain
build-system, devops
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.