canonical / canonical/cloud-init
Unexpected routing policy added in EC2 instance with multiple interfaces
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 1.1k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 18
Description
# Bug report
With the recent change in https://github.com/canonical/cloud-init/pull/4799 we will now add additional `ip rule` for each interfaces with the default gateway in VPC network. However this literally break at least two of our production services. Take one of the services for example:
```
--- Instance1 ---> Instance2 ---> Internet
|
X
|
VPC GW
```
In the above topo, both `instance1` and `instance2` are AWS EC2 instances, and both have multiple NICs. Both instances are using cloud-init and netplan for network configuration.
We installed some static routes on instance1, so that any traffic initiated from instance1 is supposed to be routed via instance2 (for inspection) regardless of its source IP. However since we have the policy routing rules which has a higher priority than main table lookup, it will be routed via VPC default gateway directly instead.
In my opinion this type of service chaining is very common in the cloud environment, and cloud-init and netplan are widely used across various cloud vendors. The routing between these different services should be defined by the service owner, rather than by the automatically generated PBR rules.
## Steps to reproduce the problem
As described in the above section, the problem can be easily produced. This was working with 23.4.4 cloud-init package but recently it is upgraded to 24.1.3 in Ubuntu 20.04.
## Environment details
- Cloud-init version: 24.1.3
- Operating System Distribution: Ubuntu 20.04
- Cloud provider, platform or installer type: AWS EC2
## cloud-init logs
Since the issue has been illustrated and it is easy to reproduce, I assume no logs are needed. But please let me know in case any additional information is needed.
Contributor guide
Assessment
This issue has not been assessed yet.