aws / aws/aws-cdk

aws_ec2: Cannot replace an instance when a private IP is set

Open
#31,635 4 comments 0 reactions 1 assignee Assigned to @pahud View on GitHub
@aws-cdk/aws-ec2 bug p2
Dominant language
TypeScript
Stars
12.9k
Forks
4.6k
Avg merge
2d 3h
Merged PRs (30d)
83

Description

### Describe the bug

It is possible to deploy an AWS EC2 Instance with a private IPv4 address (for example: `10.0.0.5`). When we try to update the stack however, and the instance needs to be replaced, we find that the deployment fails with the error:
```
Resource handler returned message: "Address is in use. ...
```

### Regression Issue

- [ ] Select this option if this issue appears to be a regression.

### Last Known Working CDK Version

_No response_

### Expected Behavior

The old instance is destroyed, the private IP is relinquished, and a new instance takes its place with the same IP address.

### Current Behavior

The first `cdk deploy` command succeeds, but subsequent `deploy` commands fail with the following error. In this example, we were using `10.0.0.5` as the static private IP.
```
11:00:24 a.m. | CREATE_FAILED | AWS::EC2::Instance | testinstanceinstan...61da7d69c9172494c5
Resource handler returned message: "Address 10.0.0.5 is in use. (Service: Ec2, Status Code: 400, Request ID: 7dc158b7-794c-4fad-a61a-e8f2d33b0e13)" (RequestToken: f3080a75-38cb-e40f-acfa-4d4c390
2a5be, HandlerErrorCode: InvalidRequest)

❌ ec2-instance failed: Error: The stack named ec2-instance failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Address 10.0.0.5 is in use. (Service: Ec2, Status Code: 400, Request ID: 7dc158b7-794c-4fad-a61a-e8f2d33b0e13)" (RequestToken: f3080a75-38cb-e40f-acfa-4d4c3902a5be, HandlerErrorCode: InvalidRequest)
at FullCloudFormationDeployment.monitorDeployment (/nix/store/qkdadnxhfbx9np4rlx2rd10k722dqbln-aws-cdk-2.159.1/lib/node_modules/aws-cdk/lib/index.js:463:10567)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/nix/store/qkdadnxhfbx9np4rlx2rd10k722dqbln-aws-cdk-2.159.1/lib/node_modules/aws-cdk/lib/index.js:466:200334)
at async /nix/store/qkdadnxhfbx9np4rlx2rd10k722dqbln-aws-cdk-2.159.1/lib/node_modules/aws-cdk/lib/index.js:466:181756

❌ Deployment failed: Error: The stack named ec2-instance failed to deploy: UPDATE_ROLLBACK_COMPLETE: Resource handler returned message: "Address 10.0.0.5 is in use. (Service: Ec2, Status Code: 400, Request ID: 7dc158b7-794c-4fad-a61a-e8f2d33b0e13)" (RequestToken: f3080a75-38cb-e40f-acfa-4d4c3902a5be, HandlerErrorCode: InvalidRequest)
at FullCloudFormationDeployment.monitorDeployment (/nix/store/qkdadnxhfbx9np4rlx2rd10k722dqbln-aws-cdk-2.159.1/lib/node_modules/aws-cdk/lib/index.js:463:10567)
at process.processTicksAndRejections (node:internal/process/task_queues:95:5)
at async Object.deployStack2 [as deployStack] (/nix/store/qkdadnxhfbx9np4rlx2rd10k722dqbln-aws-cdk-2.159.1/lib/node_modules/aws-cdk/lib/index.js:466:200334)
at async /nix/store/qkdadnxhfbx9np4rlx2rd10k722dqbln-aws-cdk-2.159.1/lib/node_modules/aws-cdk/lib/index.js:466:181756

```

### Reproduction Steps

# Initialize a python3 cdk app
```
cdk init app --language python
```
# Add reproduction code
Populate the `app.py` file with the following code, it is esentially a simplified version of the [ec2/instances](https://github.com/aws-samples/aws-cdk-examples/tree/main/python/ec2/instance) example.
```
#!/usr/bin/env python3
import os.path
import datetime

from aws_cdk import aws_ec2 as ec2, App, Stack
from constructs import Construct

dirname = os.path.dirname(__file__)

class EC2InstanceStack(Stack):
def __init__(self, scope: Construct, id: str, **kwargs) -> None:
super().__init__(scope, id, **kwargs)

# VPC
vpc = ec2.Vpc(
self,
"VPC",
enable_dns_hostnames=True,
enable_dns_support=True,
subnet_configuration=[
ec2.SubnetConfiguration(
cidr_mask=22,
name="public",
subnet_type=ec2.SubnetType.PUBLIC
),
],
)

# Instances
test_instance = ec2.Instance(
self,
"test-instance-instance",
instance_name="test-instance",
instance_type=ec2.InstanceType.of(
ec2.InstanceClass.T3, ec2.InstanceSize.NANO
),
machine_image=ec2.MachineImage.latest_amazon_linux2(),
vpc=vpc,
vpc_subnets=ec2.SubnetSelection(subnet_type=ec2.SubnetType.PUBLIC),
# Set a private IP and enforce that it gets replaced on every deploy
private_ip_address="10.0.0.5",
user_data_causes_replacement=True,
)

# Create a instance user script that changes on every deploy
current_time = str(datetime.datetime.now())

user_data_string = f"""
echo {current_time}
"""
test_instance.add_user_data(user_data_string)

app = App()
EC2InstanceStack(app, "ec2-instance")

app.synth()
```
# Deploy once
- Run the necessary steps to initialize the python3 stack (e.g. installing the venv etc etc)
- Run `cdk deploy` this should succeed.

# Deploy again
- Run deploy again, changing the private ip.
- This should fail when the cdk attempts to replace the instance.
- There may need to be a cdk diff step done here, I'm unsure if that is important to force the cdk to destroy and create a new instance.

### Possible Solution

_No response_

### Additional Information/Context

_No response_

### CDK CLI Version

2.159.1 (build c66f4e3)

### Framework Version

_No response_

### Node.js Version

v20.17.0

### OS

Ubuntu 22.04.5 LTS

### Language

Python

### Language Version

Python 3.12.5

### Other information

Here's my `shell.nix` file if it is helpful, along with the `requirements.txt` file created by the `cdk` CLI.

`shell.nix`
```nix
{pkgs ? import {}}:
let
pythonEnv = pkgs.python3.withPackages(ps: []);
in
pkgs.mkShell {
buildInputs = with pkgs; [nodePackages.aws-cdk nodejs];
packages = [pythonEnv];
}
```

`requirements.txt`
```
aws-cdk-lib==2.159.0
constructs>=10.0.0,<11.0.0
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.