aws / aws/sagemaker-spark-container

awscli and dnf broken due to /usr/bin/python3 symlink change to Python 3.12

Open Beginner friendly
#190 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
43
Forks
78
PR merge metrics
No merged PRs in 30d

Description

Reporting this bug after attempting to upgrade to the latest version in our custom image that builds on top of `sagemaker-spark-container`.

The change made in this prior PR https://github.com/aws/sagemaker-spark-container/pull/185 (affecting versions `sagemaker-spark-processing:3.5-cpu-py312-v1.3` and above) where the `/usr/bin/python3` symlink was pointed to Python 3.12 instead of Python 3.9, breaks `awscli` and `dnf` with the following errors:

```
docker run -it --entrypoint /bin/bash 153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.5

bash-5.2# aws
Traceback (most recent call last):
File "/usr/bin/aws", line 19, in
import awscli.clidriver
ModuleNotFoundError: No module named 'awscli'
bash-5.2# dnf
Traceback (most recent call last):
File "/usr/bin/dnf", line 61, in
from dnf.cli import main
ModuleNotFoundError: No module named 'dnf'
```

If you revert by pointing back to Python 3.9 `ln -sf /usr/bin/python3.9 /usr/bin/python3`, it fixes the issue. It should be noted that the AL2023 image comes installed with Python 3.9 and the [docs](https://docs.aws.amazon.com/linux/al2023/ug/python.html) state that the symlink shouldn't be changed.

Contributor guide

Open the contributing guide

Research direction

Start by comparing the change in prior PR #185 with the /usr/bin/python3 symlink in the affected SageMaker Spark container versions. Reproduce the aws and dnf entry-point failures in the provided container, then verify that keeping the symlink pointed to /usr/bin/python3.9 restores both commands without breaking the image build.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, docker, python
Domain
cloud, infrastructure
Issue type
Bug
Difficulty
2/5
Estimated time
1-3 hours
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
72/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.