cloud-init hard codes MTU configuration at initial deploy time

This bug was originally filed in Launchpad as [LP: #1899487](https://bugs.launchpad.net/cloud-init/+bug/1899487)

Launchpad details

affected_projects = ['neutron', 'netplan.io (Ubuntu)']

assignee = None

assignee_name = None

date_closed = None

date_created = 2020-10-12T13:37:40.543454+00:00

date_fix_committed = 2021-11-23T15:20:25.220656+00:00

date_fix_released = 2021-11-23T15:20:25.220656+00:00

id = 1899487

importance = undecided

is_complete = False

lp_url = https://bugs.launchpad.net/cloud-init/+bug/1899487

milestone = None

owner = fnordahl

owner_name = Frode Nordahl

private = False

status = incomplete

submitter = fnordahl

submitter_name = Frode Nordahl

tags = []

duplicates = []

_Launchpad user **Frode Nordahl(fnordahl)** wrote on 2020-10-12T13:37:40.543454+00:00_

When using OpenStack cloud provider `cloud-init` will write out /etc/netplan/50-cloud-init.yaml at initial instance boot and not update it on subsequent boots of the instance.

The OpenStack metadata service provides information about MTU for the network [0] and cloud-init takes this value and writes it into the netplan configuration [1].

A side effect of configuring the MTU through netplan is that the `systemd-networkd` [Link] section [2] gets the MTUBytes value filled and this in turn makes `systemd-networkd` ignore the MTU value provided by DHCP [3][4].

During the lifetime of a cloud events occur that will force a operator to reduce the MTU available to instances attached to its overlay networks. This may happen because of software imposed change of tunnel type (GRE -> VXLAN, VXLAN -> GENEVE) or change of topology or encapsulation in the physical network equipment.

To maximize performance these clouds have configured their instances to use the maximum available MTU without leaving any headroom to account for such changes and the only way to move forward is to reduce the available MTU on the instances. We are facing a concrete challenge with this now where we have users wanting to migrate from VXLAN tunnels to GENEVE tunnels with 38 byte header size.

0: # curl http://169.254.169.254/openstack/2018-08-27/network_data.json
{"links": [{"id": "tapa035fb68-01", "vif_id": "a035fb68-010c-42e3-8da7-ea3c36a0d607", "type": "ovs", "mtu": 8942, "ethernet_mac_address": "fa:16:3e:31:26:f7"}], "networks": [{"id": "network0", "type": "ipv4_dhcp", "link": "tapa035fb68-01", "network_id": "b4ef84c0-1235-48a8-aaf7-03fab7ef5367"}], "services": []}

1: # cat /etc/netplan/50-cloud-init.yaml
# This file is generated from information provided by the datasource. Changes
# to it will not persist across an instance reboot. To disable cloud-init's
# network configuration capabilities, write a file
# /etc/cloud/cloud.cfg.d/99-disable-network-config.cfg with the following:
# network: {config: disabled}
network:
version: 2
ethernets:
ens2:
dhcp4: true
match:
macaddress: fa:16:3e:31:26:f7
mtu: 8950
set-name: ens2

2: # cat /run/systemd/network/10-netplan-ens2.link
[Match]
MACAddress=fa:16:3e:31:26:f7

[Link]
Name=ens2
WakeOnLan=off
MTUBytes=8950

3: # cat /run/systemd/network/10-netplan-ens2.network
[Match]
MACAddress=fa:16:3e:31:26:f7
Name=ens2

[Link]
MTUBytes=8950

[Network]
DHCP=ipv4
LinkLocalAddressing=ipv6

[DHCP]
RouteMetric=100
UseMTU=true

4: Oct 12 13:30:18 canary-3 systemd-networkd[24084]: /run/systemd/network/10-netplan-ens2.network: MTUBytes= in [Link] section and UseMTU= in [DHCP] section are set. Disabling UseMTU=.