aws cloudformation deploy fails although deploy continues to run on AWS
- Dominant language
- Python
- Stars
- 17.3k
- Forks
- 4.6k
- Avg merge
- 1d 2h
- Merged PRs (30d)
- 13
Description
We have embedded stack which takes a while to create. Comand "aws cloudformation deploy" fails with error code 255 and "Max attempts exceeded", although the stack continues to be created on AWS and eventually succeeds.
Version:
```
aws --version
aws-cli/1.11.148 Python/2.7.12+ Linux/4.10.0-35-generic botocore/1.7.6
```
Last lines of debug output are:
```
2017-09-14 11:03:04,807 - MainThread - botocore.hooks - DEBUG - Event needs-retry.cloudformation.DescribeStacks: calling handler
2017-09-14 11:03:04,808 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2017-09-14 11:03:04,809 - MainThread - awscli.customizations.cloudformation.deployer - DEBUG - Execute changeset waiter exception
Traceback (most recent call last):
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deployer.py", line 184, in wait_for_execute
waiter.wait(StackName=stack_name)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/botocore/waiter.py", line 53, in wait
Waiter.wait(self, **kwargs)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/botocore/waiter.py", line 329, in wait
last_response=response
WaiterError: Waiter StackCreateComplete failed: Max attempts exceeded
2017-09-14 11:03:04,811 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 200, in main
return command_table[parsed_args.command](remaining, parsed_args)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 338, in __call__
return command_table[parsed_args.operation](remaining, parsed_globals)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/commands.py", line 187, in __call__
return self._run_main(parsed_args, parsed_globals)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deploy.py", line 184, in _run_main
parsed_args.notification_arns)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deploy.py", line 199, in deploy
deployer.wait_for_execute(stack_name, result.changeset_type)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deployer.py", line 188, in wait_for_execute
raise exceptions.DeployFailedError(stack_name=stack_name)
DeployFailedError: Failed to create/update the stack. Run the following command
to fetch the list of events leading up to the failure
aws cloudformation describe-stack-events --stack-name op-smooth-operator-development
2017-09-14 11:03:04,814 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255
```
I tried to look if it would be simple to increase maxAttempts in botocore, but that seemed to be pretty complicated.
Workaround for this is to check with `aws cloudformation describe-stack` if the stack really failed.
Contributor guide
Assessment
This issue has not been assessed yet.