aws / aws/sagemaker-python-sdk

pip install from requirements.txt on SageMaker training jobs does not install from github repo

未关闭
#4,474 2 条评论 0 个 reaction 已指派 1 人 已被 @rsareddy0329 认领 在 GitHub 查看
component: pysdk-team component: training type: bug
主要语言
Python
星标
2.3k
派生
1.3k
平均合并
1 天 22 小时
30 天内合并 PR
35

描述

**Describe the bug**
I have put my requirements.txt file in my code folder, and have specified github repository as installation mode for one of the library. However the job does not get executed successfully in Sagemaker, and gives error saying RemoteNotFoundError.

**To reproduce**
Just try to install a package using git like this by modifying your requirements.txt to include this line:

`-e git+https://github.com/huggingface/diffusers.git@main#egg=diffusers`

**Expected behavior**
Training should start

**Screenshots or logs**
logs:
```
/opt/conda/bin/python3.10 -m pip install -r requirements.txt

-- Obtaining diffusers from git+https://github.com/huggingface/diffusers.git@main#egg=diffusers (from -r requirements.txt (line 8))
ERROR: Exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/vcs/git.py", line 367, in get_remote_url
found_remote = remotes[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
return func(self, options, args)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 377, in run
requirement_set = resolver.resolve(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 534, in collect_root_requirements
reqs = list(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 490, in _make_requirements_from_install_req
cand = self._make_base_candidate_from_link(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 207, in _make_base_candidate_from_link
self._editable_candidate_cache[link] = EditableCandidate(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 318, in __init__
super().__init__(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 328, in _prepare_distribution
return self._factory.preparer.prepare_editable_requirement(self._ireq)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 692, in prepare_editable_requirement
req.update_editable()
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 699, in update_editable
vcs_backend.obtain(self.source_dir, url=hidden_url, verbosity=0)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/vcs/versioncontrol.py", line 526, in obtain
existing_url = self.get_remote_url(dest)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/vcs/git.py", line 369, in get_remote_url
raise RemoteNotFoundError

pip._internal.vcs.versioncontrol.RemoteNotFoundError
```

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: '2.210.0'
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: HuggingFace (Pytorch)
- **Framework version**: Pytorch 2.1.0
- **Python version**: 3.10
- **CPU or GPU**: GPU
- **Custom Docker image (Y/N)**: N

贡献指南

打开贡献指南

评估

这个 Issue 还没有评估数据。

把新 issue 发到你的邮箱

精选适合新手参与的 GitHub issue 摘要。