canonical / canonical/cloud-init

rename attempt of already renamed device

Open
#5,075 11 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
3.8k
Forks
1.1k
Avg merge
3d 8h
Merged PRs (30d)
22

Description

# Bug report
When cloud-init is instructed to rename an interface it should check if the name is already being used or not before trying to attempt to rename a network device.

We also have to decide what the behavior should be when the name is detected to exist.

## Steps to reproduce the problem
There is an inherent race condition between cloud-init and udev that occurs when the system is being rebooted before cloud-init is completely finished. In a case where a network interface gets assigned a name by the user through the network configuration cloud-init will write this information into the udev rules file (persistent-net-cloud-init.rules) generated during the init-local phase. If after cloud-init local is complete the system gets rebooted and the same network config is provided by the user cloud-init will start again, as usual, but on this boot the udev rules file also exists and as such udev will run and process the rules in the previously generated persistent-net-cloud-init.rules file. While the file itself will get overwritten during the new run udev will have already read the old file and process it. At this point it is just a matter of time that udev will create the renamed device setup and by the time cloud-init comes around the rename will have already happened and cloud-init will generate an error:

"""
Command: ['ip', 'link', 'set', 'enp55s1f0', 'name', 'ccd_ban']
Exit code: 2
Reason: -
Stdout:
Stderr: RTNETLINK answers: File exists
"""

As a result the interface remains down.

## Environment details
- Cloud-init version: 23.3
- Operating System Distribution: SLES
- Cloud provider, platform or installer type:

## cloud-init logs
cloud-init starts at 2024-02-08 05:54:17,618 then the network devices are enumerated starting at 2024-02-08 05:54:17,790 and ending at 2024-02-08 05:54:17,829

Then network devices are enumerated again starting at 2024-02-08 05:54:17,838 and ending at 2024-02-08 05:54:17,843. This is pretty much at the end of the clod-init-local phase. After all this we write out /etc/udev/rules.d/85-persistent-net-cloud-init.rules

The udev rules are written at

2024-02-08 05:54:17,846

Then at 2024-02-08 05:55:00,230 cloud-init starts it's "init" phase with net device configuration starting again at 2024-02-08 05:55:00,276 and that ends at 2024-02-08 05:55:00,292

Then at 2024-02-08 05:55:03,848 the final phase of cloud-init starts and at 2024-02-08 05:56:08,100 cloud-init gets killed. -> Reboot is triggered

After the system reboot cloud-init starts again at 2024-02-08 05:58:36,710 and the network device enumeration starts at 2024-02-08 05:58:36,950 and ends at 2024-02-08 05:58:37,009 close to the end of the second local phase. THe second "init" phase starts at 2024-02-08 05:58:45,541 with the network device enumertaion in init starting at 2024-02-08 05:58:45,589, only now udev has caught up and during this enumeration cycle we see

2024-02-08 05:58:45,590 - util.py[DEBUG]: Reading from /sys/class/net/ccd_ban/address

Contributor guide

Open the contributing guide

Research direction

Trace the network-device rename path and its interaction with persistent-net-cloud-init.rules, using the reported ip link set failure as the starting point. Reproduce or model the udev/cloud-init race, then clarify with maintainers what should happen when the target name already exists. Done means the agreed behavior is implemented and covered for this failure case.

Written by the indexing model from the issue text.

Assessment

Tech stack
linux, python
Domain
networking, operating-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.