AWS EMR bootstrap script fails
- Dominant language
- Python
- Stars
- 69
- Forks
- 40
- PR merge metrics
- No merged PRs in 30d
Description
**What happened**:
I tried to instantiate an EMR cluster with the AWS EMR bootstrap script. The provisioning of the cluster failed saying "bootstrap action 1 returned a non-zero return code". Extracting the logs, the stdout of the master arrives to call the conda pack command:
```
...
Installing base packages
Packaging environment
```
while the stderr shows this:
```
...
CondaPackError:
Files managed by conda were found to have been deleted/overwritten in the
following packages:
- python='3.8.5'
This is usually due to `pip` uninstalling or clobbering conda managed files,
resulting in an inconsistent environment. Please check your environment for
conda/pip conflicts using `conda list`, and fix the environment by ensuring
only one version of each package is installed (conda preferred).
```
**What you expected to happen**:
The EMR cluster provisions correctly.
**Minimal Complete Verifiable Example**:
I first uploaded the bootstrap script to s3://my-bucket/dask-emr/bootstrap.sh than tried to provision the EMR cluster with this AWS CLI command:
```sh
aws emr create-cluster --name "Janos Dask Trial Cluster" \
--release-label emr-5.30.1 \
--applications Name=HADOOP \
--log-uri s3://my-bucket/dask-emr/logs/ \
--ec2-attributes KeyName=my-key \
--instance-type m5.xlarge \
--instance-count 3 \
--bootstrap-actions Path=s3://my-bucket/dask-emr/bootstrap.sh,Args="[--conda-packages,bokeh,fastparquet,python-snappy,snappy]" \
--use-default-roles
```
Contributor guide
Assessment
This issue has not been assessed yet.