aws / aws/sagemaker-python-sdk

Something like a .sagemakerignore file option to allow for skipping of certain directories when creating source tar

Ouverte
#4,187 4 commentaires 10 réactions 0 personnes assignées Voir sur GitHub
component: pysdk-team type: feature request
Langage dominant
Python
Étoiles
2.3k
Forks
1.3k
Merge moyen
1 j 22 h
PR mergées (30 j)
35

Description

**Describe the feature you'd like**
Similar to how in Docker there is a .dockerignore file which works like a .gitignore file, It would be great if there was a .sagemakerignore file so that any directories listed in that file would be skipped when making the tar file. I believe it would be a simple change here: https://github.com/aws/sagemaker-python-sdk/blob/744724b01bcdc3f451c9b73c34c947e12d6a8e2a/src/sagemaker/fw_utils.py#L486
Basically adding a few lines that check for the existence of a .sagemakerignore file in the directory of the `directory` variable, if it exists, and then skipping anything listed out in that file.

**How would this feature be used? Please describe.**
This would be useful to reduce the size of the source tar file which would allow a person to speed up sagemaker build creation without the need to restructure their code, if they have big files in the source directory that they don't commit to other things like git and docker, so they don't want it put into the sagemaker tarball

**Describe alternatives you've considered**
An alternative is not doing it

**Additional context**
I don't mind writing the code to do this but wanted to check in that this was something that would be accepted if I make a PR for it?

Guide de contribution

Ouvrir le guide de contribution

Piste de recherche

Commencez dans src/sagemaker/fw_utils.py vers la ligne 486, où le tar source est créé, et examinez comment le contenu du répertoire est sélectionné. Confirmez le comportement attendu de .sagemakerignore d’après l’issue, puis vérifiez que les répertoires listés sont exclus du tar résultant tandis que les autres fichiers source restent inclus.

Rédigé par le modèle d'indexation à partir du texte de l'issue.

Évaluation

Stack technique
python
Domaine
build-system, machine-learning
Type d'issue
Fonctionnalité
Difficulté
3/5
Temps estimé
1-2 jours
Activité
Calme
Clarté
Plutôt claire
Accessibilité débutants
62/100

Recevez les nouvelles issues par e-mail

Un résumé court des issues GitHub adaptées aux débutants.