Compile option to build a single-file executable
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 4.6k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 13
Description
### Describe the feature
When running the AWS CLI on a slow NFS network, any simple command takes a long time, depending on the NFS load.
A strace execution shows about 4000 files are read, mostly from botocore Python library.
The same AWS CLI installed on a local disk can show same execution in 0.5s, which can be 10x or 100x faster than the execution on a slow NFS disk.
There's already the option `--with-install-type=portable-exe` but this still stores the Python libraries in many small files.
The previous feature request #6388 was approximately similar but was solved differently with a containerization.
What I'm looking for is a new compilation option that would trigger the `--onefile` option of the _pyinstaller_ so that the final "aws" executable would require less NFS accesses.
### Use Case
Many small aws commands launched in a shell script makes the overall script slow if each command takes 5s-10s instead of 0.5s with a local disk installation. On the other hand, it's not practical to run local installations on all the hosts of the cluster.
### Proposed Solution
The AWS CLI container is a workaround, but still there's a small overhead to always run a container. When I launch the same command on a slow NFS disk is takes between 5s and 10s, vs. 1.5s when executed from a container, and 0.5s when executed from a local disk installation.
### Other Information
Peformance measurements:
- on a slow NFS installation:
```
/usr/bin/time -p /aws eks get-token --cluster-name mycluster | grep real
real 6.63
real 7.16
real 7.45
```
- on a local disk, same host:
```
/usr/bin/time -p /tmp/aws-cli/v2/2.22.18/bin/aws eks get-token --cluster-name mycluster | grep real
real 0.55
real 0.54
real 0.54
```
- with the aws cli container:
```
/usr/bin/time -p apptainer exec aws.sif aws eks get-token --cluster-name mycluster | grep real
real 1.56
real 1.46
real 1.48
```
Filesystem accesses:
```
% strace -s 200 -f -t /aws eks get-token --cluster-name mycluster | grep real > & strace.log
% grep "open " strace.log | wc -l
3907
```
### Acknowledgements
- [ ] I may be able to implement this feature request
- [ ] This feature might incur a breaking change
### CLI version used
2.22.18
### Environment details (OS name and version, etc.)
RHEL8
Contributor guide
Assessment
This issue has not been assessed yet.