hashicorp / hashicorp/packer-plugin-amazon
amazon-ebs timeout - Error waiting for instance to stop reoccur when building on newly available mac instances
- Dominant language
- Go
- Stars
- 91
- Forks
- 141
- Avg merge
- 2d 18h
- Merged PRs (30d)
- 3
Description
_This issue was originally opened by @OliverKoo as hashicorp/packer#10688. It was migrated here as a result of the [Packer plugin split](https://github.com/hashicorp/packer/issues/8610#issuecomment-770034737). The original body of the issue is below._
When building on the newly released mac1.metal instances that runs on dedicated host, the timeout occurs when waiting for instance to stop
```
==> amazon-ebs: Waiting for the instance to stop...
==> amazon-ebs: Error waiting for instance to stop: ResourceNotReady: exceeded wait attempts
==> amazon-ebs: Pausing before cleanup of step 'StepCleanupTempKeys'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepProvision'. Press enter to continue.
==> amazon-ebs: Provisioning step had errors: Running the cleanup provisioner, if present...
==> amazon-ebs: Pausing before cleanup of step 'StepSetGeneratedData'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepConnect'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepCreateSSMTunnel'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepGetPassword'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepRunSourceInstance'. Press enter to continue.
==> amazon-ebs: Terminating the source AWS instance...
==> amazon-ebs: Pausing before cleanup of step 'StepCleanupVolumes'. Press enter to continue.
==> amazon-ebs: Cleaning up any extra volumes...
==> amazon-ebs: No volumes to clean up, skipping
==> amazon-ebs: Pausing before cleanup of step 'StepIamInstanceProfile'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepSecurityGroup'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepKeyPair'. Press enter to continue.
==> amazon-ebs: Deleting temporary keypair...
==> amazon-ebs: Pausing before cleanup of step 'StepNetworkInfo'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepSourceAMIInfo'. Press enter to continue.
==> amazon-ebs: Pausing before cleanup of step 'StepPreValidate'. Press enter to continue.
2021/02/22 21:33:00 [INFO] (telemetry) ending amazon-ebs
Build 'amazon-ebs' errored after 1 hour 28 minutes: Error waiting for instance to stop: ResourceNotReady: exceeded wait attempts
```
Above are the logs I get by building on a mac1.metal instances. I didn't provision anything on top of the source AMI. the AMI I use is the latest Catalina AMI provided by amasion (ami-07f480f3fa002bc15)
I was able to get around this by setting AWS_MAX_ATTEMPTS=300 and AWS_POLL_DELAY_SECONDS=30
Similar issues [#6526](https://github.com/hashicorp/packer/issues/6526) and [#6536](https://github.com/hashicorp/packer/issues/6536) happened before but was fixed with the raised timeout at [1.3.0 ](https://github.com/hashicorp/packer/pull/6601)
These mac ec2 instances are fairly new to the AWS world, they are made available just short of 3 months ago 11/30/2020. Because they run on a dedicated host (mac mini), when terminating the instances there is a very lengthy scrubbing process to ensure the next user does not get any info from disk, memory, nor NVRAM. AWS doesn't release info about how long this process takes but is about 1h30m. I got this number from my own experiments and other user's report (https://dev.to/svasylenko/mac1-metal-ec2-instance-user-experience-j08), see Destroying the Instance section.
The default limit should probably be raised when building on mac1.metal instances.
Contributor guide
Assessment
This issue has not been assessed yet.