aws / aws/aws-cli

aws cloudformation deploy fails although deploy continues to run on AWS

Open
#2,831 9 comments 1 reaction 0 assignees View on GitHub
cloudformation package-deploy customization feature-request p3
Dominant language
Python
Stars
17.3k
Forks
4.6k
Avg merge
1d 2h
Merged PRs (30d)
13

Description

We have embedded stack which takes a while to create. Comand "aws cloudformation deploy" fails with error code 255 and "Max attempts exceeded", although the stack continues to be created on AWS and eventually succeeds.

Version:
```
aws --version
aws-cli/1.11.148 Python/2.7.12+ Linux/4.10.0-35-generic botocore/1.7.6
```

Last lines of debug output are:
```
2017-09-14 11:03:04,807 - MainThread - botocore.hooks - DEBUG - Event needs-retry.cloudformation.DescribeStacks: calling handler
2017-09-14 11:03:04,808 - MainThread - botocore.retryhandler - DEBUG - No retry needed.
2017-09-14 11:03:04,809 - MainThread - awscli.customizations.cloudformation.deployer - DEBUG - Execute changeset waiter exception
Traceback (most recent call last):
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deployer.py", line 184, in wait_for_execute
waiter.wait(StackName=stack_name)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/botocore/waiter.py", line 53, in wait
Waiter.wait(self, **kwargs)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/botocore/waiter.py", line 329, in wait
last_response=response
WaiterError: Waiter StackCreateComplete failed: Max attempts exceeded
2017-09-14 11:03:04,811 - MainThread - awscli.clidriver - DEBUG - Exception caught in main()
Traceback (most recent call last):
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 200, in main
return command_table[parsed_args.command](remaining, parsed_args)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/clidriver.py", line 338, in __call__
return command_table[parsed_args.operation](remaining, parsed_globals)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/commands.py", line 187, in __call__
return self._run_main(parsed_args, parsed_globals)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deploy.py", line 184, in _run_main
parsed_args.notification_arns)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deploy.py", line 199, in deploy
deployer.wait_for_execute(stack_name, result.changeset_type)
File "/home/jyrki/.virtualenvs/op-aws/local/lib/python2.7/site-packages/awscli/customizations/cloudformation/deployer.py", line 188, in wait_for_execute
raise exceptions.DeployFailedError(stack_name=stack_name)
DeployFailedError: Failed to create/update the stack. Run the following command
to fetch the list of events leading up to the failure
aws cloudformation describe-stack-events --stack-name op-smooth-operator-development
2017-09-14 11:03:04,814 - MainThread - awscli.clidriver - DEBUG - Exiting with rc 255
```
I tried to look if it would be simple to increase maxAttempts in botocore, but that seemed to be pretty complicated.

Workaround for this is to check with `aws cloudformation describe-stack` if the stack really failed.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.