aws / aws/aws-elastic-beanstalk-cli
EB CLI Doesn't Preserve Executable File Permissions While Zipping Application Source Bundle on Windows
- Dominant language
- Python
- Stars
- 181
- Forks
- 88
- PR merge metrics
- No merged PRs in 30d
Description
### Description
When running `eb create` or `eb deploy` commands on WINDOWS 10, files with executable permissions (ex: chmod +x build.sh) are not zipped correctly when creating the application version archive that gets uploaded.
### Steps to reproduce
1. Create IAM user named eb-full (IAM > Add user > check programmatic access > Attach "AdministratorAccess-AWSElasticBeanstalk")
2. Add "eb-full" user to /.aws credentials and config files.
```
# C:\Users\\.aws\credentials
[eb-cli]
aws_access_key_id = AK...X3
aws_secret_access_key = wt...4Hi
# C:\Users\\.aws\config
[profile eb-cli]
region=us-west-2
output=json
```
3. cd to .ssh folder and generate keys with no passphrase:
```
ssh-keygen -t rsa -b 4096 -f eb-user -C "eb@inspiron"
```
4. Go to EC2 > Network & Security > Key Pairs > Actions > Import key pair > import eb-user.pub. Give it name eb-user.
5. On Windows machine, make project root folder named `awsebcli-deploy-bug`. Ex: `C:\Users\\Downloads\awsebcli-deploy-bug` for this example.
6. From terminal, initialize Elastic Beanstalk: `eb init --profile eb-cli`
```
C:\Users\Jarad\Downloads\awsebcli-deploy-bug>eb init --profile eb-cli
Select a default region
1) us-east-1 : US East (N. Virginia)
2) us-west-1 : US West (N. California)
3) us-west-2 : US West (Oregon)
4) eu-west-1 : EU (Ireland)
5) eu-central-1 : EU (Frankfurt)
6) ap-south-1 : Asia Pacific (Mumbai)
7) ap-southeast-1 : Asia Pacific (Singapore)
8) ap-southeast-2 : Asia Pacific (Sydney)
9) ap-northeast-1 : Asia Pacific (Tokyo)
10) ap-northeast-2 : Asia Pacific (Seoul)
11) sa-east-1 : South America (Sao Paulo)
12) cn-north-1 : China (Beijing)
13) cn-northwest-1 : China (Ningxia)
14) us-east-2 : US East (Ohio)
15) ca-central-1 : Canada (Central)
16) eu-west-2 : EU (London)
17) eu-west-3 : EU (Paris)
18) eu-north-1 : EU (Stockholm)
19) eu-south-1 : EU (Milano)
20) ap-east-1 : Asia Pacific (Hong Kong)
21) me-south-1 : Middle East (Bahrain)
22) af-south-1 : Africa (Cape Town)
(default is 3):
Enter Application Name
(default is "awsebcli-deploy-bug"):
Application awsebcli-deploy-bug has been created.
Select a platform.
1) .NET Core on Linux
2) .NET on Windows Server
3) Docker
4) GlassFish
5) Go
6) Java
7) Node.js
8) PHP
9) Packer
10) Python
11) Ruby
12) Tomcat
(make a selection): 3
Select a platform branch.
1) Docker running on 64bit Amazon Linux 2
2) Multi-container Docker running on 64bit Amazon Linux
3) Docker running on 64bit Amazon Linux
(default is 1): 1
Cannot setup CodeCommit because there is no Source Control setup, continuing with initialization
Do you want to set up SSH for your instances?
(Y/n): y
Select a keypair.
1) eb-user
2) xxxxxxxxx
3) [ Create new KeyPair ]
(default is 2): 1
```
7. Add these two files to our project:
```
###.platform/hooks/prebuild/build.sh
#!/bin/bash
mv docker-compose.prod.yml docker-compose.yml
###docker-compose.prod.yml
version: "3"
services:
client:
image: nginx
ports:
- 80:80
```
8. From terminal, run `eb create production`
9. It fails. Check **eb-engine.log**. It shows:
```
2021/06/24 05:12:15.881815 [INFO] app source bundle is zip file ...
2021/06/24 05:12:15.881820 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/
2021/06/24 05:12:15.881830 [INFO] Running command /bin/sh -c /usr/bin/unzip -q -o /opt/elasticbeanstalk/deployment/app_source_bundle -d /var/app/staging/
2021/06/24 05:12:15.885707 [INFO] finished extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/ successfully
2021/06/24 05:12:15.885720 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2021/06/24 05:12:15.885762 [INFO] Executing platform hooks in .platform/hooks/prebuild/
2021/06/24 05:12:15.885801 [INFO] Following platform hooks will be executed in order: [build.sh]
2021/06/24 05:12:15.885806 [INFO] Running platform hook: .platform/hooks/prebuild/build.sh
2021/06/24 05:12:15.885993 [ERROR] An error occurred during execution of command [app-deploy] - [RunAppDeployPreBuildHooks]. Stop running the command. Error: Command .platform/hooks/prebuild/build.sh failed with error fork/exec .platform/hooks/prebuild/build.sh: permission denied
```
10. On Windows, this is tricky? I use **git bash for windows** which uses MINGW64 for me.
```
# listing permissions out
$ ls -la
total 518
drwxr-xr-x 1 Jarad 197121 0 Jun 23 22:09 ./
drwxr-xr-x 1 Jarad 197121 0 Jun 23 22:14 ../
drwxr-xr-x 1 Jarad 197121 0 Jun 23 22:10 .elasticbeanstalk/
-rw-r--r-- 1 Jarad 197121 113 Jun 23 14:59 .gitignore
drwxr-xr-x 1 Jarad 197121 0 Jun 23 15:13 .platform/
-rw-r--r-- 1 Jarad 197121 92 Jun 23 15:37 docker-compose.prod.yml
$ ls .platform/hooks/prebuild/ -la
total 1
drwxr-xr-x 1 Jarad 197121 0 Jun 23 15:13 ./
drwxr-xr-x 1 Jarad 197121 0 Jun 23 15:13 ../
-rwxr-xr-x 1 Jarad 197121 55 Jun 23 16:09 build.sh*
```
It shows build.sh has executable permissions seen by -rwxr-xr-x.
But I'll run chmod anyways:
```
chmod +x .platform/hooks/prebuild/build.sh
```
Permissions still the same.
11. Run `eb deploy` again. Same exact error as above.
12. Run `eb ssh`,
```
[ec2-user@ip-172-31-0-100 staging]$ cd /opt/elasticbeanstalk/deployment
[ec2-user@ip-172-31-0-100 deployment]$ zipinfo -l app_source_bundle
Archive: app_source_bundle
Zip file size: 606 bytes, number of entries: 4
drwxrwxrwx 2.0 fat 0 b- 0 stor 21-Jun-23 22:51 ./
-rw-rw-rw- 2.0 fat 92 b- 69 defN 21-Jun-23 22:53 docker-compose.prod.yml
drwxrwxrwx 2.0 fat 0 b- 0 stor 21-Jun-23 22:54 .platform/hooks/prebuild/
-rw-rw-rw- 2.0 fat 58 b- 45 defN 21-Jun-23 22:54 .platform/hooks/prebuild/build.sh
4 files, 150 bytes uncompressed, 114 bytes compressed: 24.0%
```
# Notice that it indicates **-rw-rw-rw- for build.sh**. I think this is a bug caused by EB CLI while zipping on Windows.
13. If I zip the same files, like so:
```
zip -r upload . -x ".elasticbeanstalk/*"
```
and upload to the "production" environment manually in the online interface... **it totally works!** I visit the domain and see the default nginx success page.
```
2021/06/24 06:14:50.628106 [INFO] app source bundle is zip file ...
2021/06/24 06:14:50.628111 [INFO] extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/
2021/06/24 06:14:50.628120 [INFO] Running command /bin/sh -c /usr/bin/unzip -q -o /opt/elasticbeanstalk/deployment/app_source_bundle -d /var/app/staging/
2021/06/24 06:14:50.630505 [INFO] finished extracting /opt/elasticbeanstalk/deployment/app_source_bundle to /var/app/staging/ successfully
2021/06/24 06:14:50.630518 [INFO] Executing instruction: RunAppDeployPreBuildHooks
2021/06/24 06:14:50.630543 [INFO] Executing platform hooks in .platform/hooks/prebuild/
2021/06/24 06:14:50.630594 [INFO] Following platform hooks will be executed in order: [build.sh]
2021/06/24 06:14:50.630600 [INFO] Running platform hook: .platform/hooks/prebuild/build.sh
2021/06/24 06:14:50.633811 [INFO] Finished running the platform hooks in .platform/hooks/prebuild/
2021/06/24 06:14:50.633825 [INFO] Executing instruction: Generate environment file
2021/06/24 06:14:50.633898 [INFO] Executing instruction: Docker Specific Build Application
2021/06/24 06:14:50.633904 [INFO] start build docker app
2021/06/24 06:14:50.633908 [INFO] detected new app as docker compose app
2021/06/24 06:14:50.633922 [INFO] Running command /bin/sh -c docker-compose config
2021/06/24 06:14:53.754789 [INFO] Running command /bin/sh -c docker-compose config --services
2021/06/24 06:14:54.145881 [INFO] client
2021/06/24 06:14:54.146000 [INFO] Running command /bin/sh -c docker-compose pull
...
```
I ssh into the working running EC2 instance with `eb ssh` and check the permissions of build.sh. It shows executable, when I manually uploaded the package.
```
[ec2-user@ip-172-31-0-100 deployment]$ zipinfo -l app_source_bundle
Archive: app_source_bundle
Zip file size: 1804 bytes, number of entries: 8
...
-rw-r--r-- 3.0 unx 113 tx 63 defN 21-Jun-23 21:59 .gitignore
drwxr-xr-x 3.0 unx 0 bx 0 stor 21-Jun-23 22:13 .platform/
drwxr-xr-x 3.0 unx 0 bx 0 stor 21-Jun-23 22:13 .platform/hooks/
drwxr-xr-x 3.0 unx 0 bx 0 stor 21-Jun-24 05:54 .platform/hooks/prebuild/
-rwxr-xr-x 3.0 unx 58 tx 45 defN 21-Jun-24 05:54 .platform/hooks/prebuild/build.sh <----- look, executable permissions when it was manually uploaded
-rw-r--r-- 3.0 unx 92 tx 69 defN 21-Jun-24 05:53 docker-compose.prod.yml
8 files, 722 bytes uncompressed, 432 bytes compressed: 40.2%
```
### Expected result
I don't want to manually zip and upload a package; I want to use EB CLI 100% of the time when deploying a change. I expect when doing `eb deploy`, that it zips the package and maintains the correct executable file permissions for files that need to be executable.
### Additional environment details (Ex: Windows, Mac, Amazon Linux etc)
1. OS: Windows 10 Pro version 20H2
2. EBCLI version: EB CLI 3.20.0 (Python 3.7.3)
### Additional Context
1. I know line endings can cause problems (CRLF vs LF). This problem is not that. My build.sh file has line ending as LF.
2. I posted a [StackOverflow question about this as well](https://stackoverflow.com/questions/67879603/define-specific-docker-compose-file-to-use-for-aws-elastic-beanstalk-deployment).
3. I FIRMLY want to name my production compose file `docker-compose.prod.yml` for workflow reasons and efficiency. That's why I have the build.sh file to begin with. Elastic Beanstalk doesn't seem to have a key that let's me define the name of the docker-compose file I want to use. If I were able to set that as a key somewhere, that would be best-case.
Contributor guide
Research direction
Start by tracing the archive creation performed by `eb create` or `eb deploy` on Windows, then inspect the generated `app_source_bundle` with `zipinfo -l` and compare it with the manually created archive. Done means executable files such as `.platform/hooks/prebuild/build.sh` retain executable permissions after EB CLI packaging.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- cli, devops
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100