aws / aws/sagemaker-spark-container
awscli and dnf broken due to /usr/bin/python3 symlink change to Python 3.12
- Dominant language
- Python
- Stars
- 43
- Forks
- 78
- PR merge metrics
- No merged PRs in 30d
Description
Reporting this bug after attempting to upgrade to the latest version in our custom image that builds on top of `sagemaker-spark-container`.
The change made in this prior PR https://github.com/aws/sagemaker-spark-container/pull/185 (affecting versions `sagemaker-spark-processing:3.5-cpu-py312-v1.3` and above) where the `/usr/bin/python3` symlink was pointed to Python 3.12 instead of Python 3.9, breaks `awscli` and `dnf` with the following errors:
```
docker run -it --entrypoint /bin/bash 153931337802.dkr.ecr.us-west-2.amazonaws.com/sagemaker-spark-processing:3.5-cpu-py312-v1.5
bash-5.2# aws
Traceback (most recent call last):
File "/usr/bin/aws", line 19, in
import awscli.clidriver
ModuleNotFoundError: No module named 'awscli'
bash-5.2# dnf
Traceback (most recent call last):
File "/usr/bin/dnf", line 61, in
from dnf.cli import main
ModuleNotFoundError: No module named 'dnf'
```
If you revert by pointing back to Python 3.9 `ln -sf /usr/bin/python3.9 /usr/bin/python3`, it fixes the issue. It should be noted that the AL2023 image comes installed with Python 3.9 and the [docs](https://docs.aws.amazon.com/linux/al2023/ug/python.html) state that the symlink shouldn't be changed.
Contributor guide
Research direction
Start by comparing the change in prior PR #185 with the /usr/bin/python3 symlink in the affected SageMaker Spark container versions. Reproduce the aws and dnf entry-point failures in the provided container, then verify that keeping the symlink pointed to /usr/bin/python3.9 restores both commands without breaking the image build.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, docker, python
- Domain
- cloud, infrastructure
- Issue type
- Bug
- Difficulty
- 2/5
- Estimated time
- 1-3 hours
- Activity status
- Active
- Clarity
- Clearly specified
- Newbie friendliness
- 72/100