aws / aws/sagemaker-python-sdk

Something like a .sagemakerignore file option to allow for skipping of certain directories when creating source tar

Aperta
#4,187 4 commenti 10 reazioni 0 assegnatari Vedi su GitHub
component: pysdk-team type: feature request
Lingua principale
Python
Stelle
2.3k
Fork
1.3k
Merge medio
1g 22h
PR unite (30g)
35

Descrizione

**Describe the feature you'd like**
Similar to how in Docker there is a .dockerignore file which works like a .gitignore file, It would be great if there was a .sagemakerignore file so that any directories listed in that file would be skipped when making the tar file. I believe it would be a simple change here: https://github.com/aws/sagemaker-python-sdk/blob/744724b01bcdc3f451c9b73c34c947e12d6a8e2a/src/sagemaker/fw_utils.py#L486
Basically adding a few lines that check for the existence of a .sagemakerignore file in the directory of the `directory` variable, if it exists, and then skipping anything listed out in that file.

**How would this feature be used? Please describe.**
This would be useful to reduce the size of the source tar file which would allow a person to speed up sagemaker build creation without the need to restructure their code, if they have big files in the source directory that they don't commit to other things like git and docker, so they don't want it put into the sagemaker tarball

**Describe alternatives you've considered**
An alternative is not doing it

**Additional context**
I don't mind writing the code to do this but wanted to check in that this was something that would be accepted if I make a PR for it?

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia in src/sagemaker/fw_utils.py intorno alla riga 486, dove viene creato il tar sorgente, e analizza come vengono selezionati i contenuti della directory. Conferma il comportamento previsto di .sagemakerignore indicato nell’issue, quindi verifica che le directory elencate siano escluse dal tar risultante mentre gli altri file sorgente rimangano inclusi.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
python
Ambito
build-system, machine-learning
Tipo di issue
Funzionalità
Difficoltà
3/5
Tempo stimato
1-2 giorni
Stato di attività
Tranquilla
Chiarezza
Abbastanza chiara
Idoneità per principianti
62/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.