aws / aws/sagemaker-python-sdk

pip install from requirements.txt on SageMaker training jobs does not install from github repo

Ouverte
#4,474 2 commentaires 0 réactions 1 personne assignée Réclamée par @rsareddy0329 Voir sur GitHub
component: pysdk-team component: training type: bug
Langage dominant
Python
Étoiles
2.3k
Forks
1.3k
Merge moyen
1 j 22 h
PR mergées (30 j)
35

Description

**Describe the bug**
I have put my requirements.txt file in my code folder, and have specified github repository as installation mode for one of the library. However the job does not get executed successfully in Sagemaker, and gives error saying RemoteNotFoundError.

**To reproduce**
Just try to install a package using git like this by modifying your requirements.txt to include this line:

`-e git+https://github.com/huggingface/diffusers.git@main#egg=diffusers`

**Expected behavior**
Training should start

**Screenshots or logs**
logs:
```
/opt/conda/bin/python3.10 -m pip install -r requirements.txt

-- Obtaining diffusers from git+https://github.com/huggingface/diffusers.git@main#egg=diffusers (from -r requirements.txt (line 8))
ERROR: Exception:
Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/vcs/git.py", line 367, in get_remote_url
found_remote = remotes[0]
IndexError: list index out of range

During handling of the above exception, another exception occurred:

Traceback (most recent call last):
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/cli/base_command.py", line 180, in exc_logging_wrapper
status = run_func(*args)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/cli/req_command.py", line 245, in wrapper
return func(self, options, args)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/commands/install.py", line 377, in run
requirement_set = resolver.resolve(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/resolver.py", line 76, in resolve
collected = self.factory.collect_root_requirements(root_reqs)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 534, in collect_root_requirements
reqs = list(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 490, in _make_requirements_from_install_req
cand = self._make_base_candidate_from_link(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/factory.py", line 207, in _make_base_candidate_from_link
self._editable_candidate_cache[link] = EditableCandidate(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 318, in __init__
super().__init__(
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 156, in __init__
self.dist = self._prepare()
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 225, in _prepare
dist = self._prepare_distribution()
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/resolution/resolvelib/candidates.py", line 328, in _prepare_distribution
return self._factory.preparer.prepare_editable_requirement(self._ireq)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/operations/prepare.py", line 692, in prepare_editable_requirement
req.update_editable()
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/req/req_install.py", line 699, in update_editable
vcs_backend.obtain(self.source_dir, url=hidden_url, verbosity=0)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/vcs/versioncontrol.py", line 526, in obtain
existing_url = self.get_remote_url(dest)
File "/opt/conda/lib/python3.10/site-packages/pip/_internal/vcs/git.py", line 369, in get_remote_url
raise RemoteNotFoundError

pip._internal.vcs.versioncontrol.RemoteNotFoundError
```

**System information**
A description of your system. Please provide:
- **SageMaker Python SDK version**: '2.210.0'
- **Framework name (eg. PyTorch) or algorithm (eg. KMeans)**: HuggingFace (Pytorch)
- **Framework version**: Pytorch 2.1.0
- **Python version**: 3.10
- **CPU or GPU**: GPU
- **Custom Docker image (Y/N)**: N

Guide de contribution

Ouvrir le guide de contribution

Évaluation

Cette issue n'a pas encore été évaluée.

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.