aws / aws/sagemaker-python-sdk

Doesn't allow us to specify a directory path as an argument to submit_py_files parameter in spark_processor.run method

Open
#2,447 1 comment 1 reaction 1 assignee Claimed by @nargokul View on GitHub
component: processing contributions welcome type: bug
Dominant language
Python
Stars
2.3k
Forks
1.3k
Avg merge
2d 2h
Merged PRs (30d)
32

Description

As per https://github.com/aws/sagemaker-python-sdk/blob/master/src/sagemaker/spark/processing.py#L76 the comments local directory paths should be allowed.

However, when we try to pass a directory path we get the following error:

`submit_deps path file:///home/ec2-user/SageMaker/code/ is not a valid local file. Please specify a list of one or more S3 URIs, local file paths, and/or local directory paths`

Most likely this spawns from the check below:

https://github.com/aws/sagemaker-python-sdk/blob/12a69180e2e88739f4fd28ed36fe88da40139f54/src/sagemaker/spark/processing.py#L441

`exists` check instead of isfile here might help resolve the issue.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.