aws / aws/amazon-sagemaker-examples
PySparkProcessor - Unable to locate credentials for boto3 call in AppMaster
- Dominant language
- Jupyter Notebook
- Stars
- 11k
- Forks
- 7k
- Avg merge
- 8h 29m
- Merged PRs (30d)
- 8
Description
I am curious if there are issues calling boto3 client from AppMaster when the network is isolated. Or, maybe my network config is off... not sure. I am running pyspark script using the PySparkProcessor (injected into the interface). That script needs to pull objects from s3 as it runs in the AppMaster\Client before working with spark session across the slave workers. I have network isolation turned on, and security groups set.. I have allowed traffic to s3.
When I turn network isolation I get the following.
"raise NoCredentialsError
botocore.exceptions.NoCredentialsError: Unable to locate credentials"
This traces back to this line of code in my pyspark script I inject.
s3 = boto3.client('s3')
I am using the same role my sagemaker notebook is running in.. that allows me to make these calls to boto3. I simply pass that role to.
networkConfig = NetworkConfig(enable_network_isolation=True,
security_group_ids=[sg_s3_access,
sg_master,
sg_slaves],
subnets=[private_subnet_3])
role = sagemaker.get_execution_role()
spark_processor = PySparkProcessor(base_job_name="some-job",
role=role,
instance_count=2,
instance_type="ml.m5.4xlarge",
max_runtime_in_seconds=2400,
network_config=networkConfig,
image_uri="............dkr.ecr.us-east-1.amazonaws.com/sagemaker-spark-processing:2.4-cpu-py37-v1.0")
All works ok when "enable_network_isolation=False" and I still pass in my networkconfig. Is there a defect in calling boto3 from PySpark script from AppMaster when network isolation is turned on? Or, should I look at my network config again?
Contributor guide
Assessment
This issue has not been assessed yet.