canonical / canonical/cloud-init

Azure - FreeBSD - Failure to setup network interface

Open
#6,657 8 comments 0 reactions 0 assignees View on GitHub
bug incomplete
Dominant language
Python
Stars
3.8k
Forks
1.1k
Avg merge
2d 23h
Merged PRs (30d)
18

Description

# Bug report
When running cloud-init on FreeBDS 15 (only version i tried) on Azure, the network interface configuration fails.

Using the out-of-date cloud-init in the FreeBSD Ports results the same. I have tried the specific commit to include the #6556 in hopes of that fixing the issue, but it didn't.

It seems that cloud-init chose interface `mce0` for dhcp/networking. When starting FreeBSD instance without cloud-init, or just waiting for cloud-init to give up, the dhcp/networking is then configured on `hn0` instead (by the regular FreeBSD boot process?).

Why did cloud-init chose `mce0` instead of `hn0`? Is there a way for me to "force" it to use `hn0`?

cc @cadejacobson as you worked on the mentioned #6556 issue/pr, if you have any ideas.

## Steps to reproduce the problem
A bit convoluted, but:
- Use [Packer](https://developer.hashicorp.com/packer) (or some other tool) to create new FreeBSD image with cloud-init installed.
- Deploy the image that you have created with packer.
- Check "boot diagnostics" to see the serial output.

## Environment details
- Cloud-init version: Commit 974646e6fe25d1dc8d9a6b96f6990ecc5df30c21 or 25.2
- Operating System Distribution: FreeBDS 15.0
- Cloud provider, platform or installer type:
```
cloud: Azure
image_publisher = "freebsd"
image_offer = "freebsd-15_0"
image_sku = "15_0-release-amd64-gen2-zfs"
```

## cloud-init logs
```
2026-01-08 13:44:31,952 - handlers.py[DEBUG]: start: azure-ds/_setup_ephemeral_networking: _setup_ephemeral_networking
2026-01-08 13:44:31,955 - subp.py[DEBUG]: Running command ['ifconfig', '-a'] with allowed return codes [0, 1] (shell=False, capture=True)
2026-01-08 13:44:32,040 - performance.py[DEBUG]: Running ['ifconfig', '-a'] took 0.083 seconds
2026-01-08 13:44:32,041 - handlers.py[DEBUG]: start: azure-ds/obtain-dhcp-lease: obtain dhcp lease
2026-01-08 13:44:32,044 - subp.py[DEBUG]: Running command ['ifconfig', '-l', '-u', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:32,051 - subp.py[DEBUG]: Running command ['ifconfig', '-a', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:32,055 - subp.py[DEBUG]: Running command ['ifconfig', '-a', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:32,059 - util.py[DEBUG]: Reading from /sys/class/net/mce0/uevent (quiet=False)
2026-01-08 13:44:32,060 - util.py[DEBUG]: Reading from /sys/class/net/mce0/addr_assign_type (quiet=False)
2026-01-08 13:44:32,063 - util.py[DEBUG]: Reading from /sys/class/net/mce0/address (quiet=False)
2026-01-08 13:44:32,067 - subp.py[DEBUG]: Running command ['ifconfig', '-a', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:32,073 - util.py[DEBUG]: Reading from /sys/class/net/mce0/uevent (quiet=False)
2026-01-08 13:44:32,074 - util.py[DEBUG]: Reading from /sys/class/net/mce0/addr_assign_type (quiet=False)
2026-01-08 13:44:32,077 - util.py[DEBUG]: Reading from /sys/class/net/mce0/address (quiet=False)
2026-01-08 13:44:32,080 - azure.py[DEBUG]: Bringing up ephemeral networking with iface=mce0 mac=None driver=None: []
2026-01-08 13:44:32,084 - distros[DEBUG]: DHCP client not found: dhcpcd
2026-01-08 13:44:32,087 - distros[DEBUG]: DHCP client selected: dhclient
2026-01-08 13:44:32,090 - dhcp.py[DEBUG]: Performing a dhcp discovery on mce0
2026-01-08 13:44:32,092 - subp.py[DEBUG]: Running command ['ip', 'link', 'set', 'dev', 'mce0', 'up'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:32,097 - azure.py[ERROR]: Command failed: cmd=['ip', 'link', 'set', 'dev', 'mce0', 'up'] stderr='-' stdout='-' exit_code=-
2026-01-08 13:44:33,099 - subp.py[DEBUG]: Running command ['ifconfig', '-l', '-u', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:33,104 - subp.py[DEBUG]: Running command ['ifconfig', '-a', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:33,109 - subp.py[DEBUG]: Running command ['ifconfig', '-a', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:33,114 - util.py[DEBUG]: Reading from /sys/class/net/mce0/uevent (quiet=False)
2026-01-08 13:44:33,115 - util.py[DEBUG]: Reading from /sys/class/net/mce0/addr_assign_type (quiet=False)
2026-01-08 13:44:33,118 - util.py[DEBUG]: Reading from /sys/class/net/mce0/address (quiet=False)
2026-01-08 13:44:33,121 - subp.py[DEBUG]: Running command ['ifconfig', '-a', 'ether'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:33,128 - util.py[DEBUG]: Reading from /sys/class/net/mce0/uevent (quiet=False)
2026-01-08 13:44:33,129 - util.py[DEBUG]: Reading from /sys/class/net/mce0/addr_assign_type (quiet=False)
2026-01-08 13:44:33,133 - util.py[DEBUG]: Reading from /sys/class/net/mce0/address (quiet=False)
2026-01-08 13:44:33,136 - azure.py[DEBUG]: Bringing up ephemeral networking with iface=mce0 mac=None driver=None: []
2026-01-08 13:44:33,139 - dhcp.py[DEBUG]: Performing a dhcp discovery on mce0
2026-01-08 13:44:33,142 - subp.py[DEBUG]: Running command ['ip', 'link', 'set', 'dev', 'mce0', 'up'] with allowed return codes [0] (shell=False, capture=True)
2026-01-08 13:44:33,147 - azure.py[ERROR]: Command failed: cmd=['ip', 'link', 'set', 'dev', 'mce0', 'up'] stderr='-' stdout='-' exit_code=-
```

Cloud init retries the last messages for a while until:
```
2026-01-08 14:04:31,685 - handlers.py[DEBUG]: finish: azure-ds/obtain-dhcp-lease: FAIL: obtain dhcp lease (duration: 1199.643s)
2026-01-08 14:04:31,689 - handlers.py[DEBUG]: finish: azure-ds/_setup_ephemeral_networking: FAIL: _setup_ephemeral_networking (duration: 1199.737s)
```

After that cloud-init seems to still do some things, but without networking set up, once it finishes, the FreeBSD boot takes over and does:
```
lo0: link state changed to UP
mce0: link state changed to UP
hn0: disable IPV6 mbuf hash delivery
hn0: link state changed to UP
Starting dhclient.
DHCPREQUEST on hn0 to 255.255.255.255 port 67
DHCPNAK from 168.63.129.16
DHCPDISCOVER on hn0 to 255.255.255.255 port 67 interval 3
DHCPOFFER from 168.63.129.16
unknown dhcp option value 0xf5
DHCPREQUEST on hn0 to 255.255.255.255 port 67
DHCPACK from 168.63.129.16
unknown dhcp option value 0xf5
bound to 10.42.12.134 -- renewal in 2147483646 seconds.
Starting Network: lo0 hn0 mce0.
lo0: flags=1008049 metric 0 mtu 16384
options=680003
inet 127.0.0.1 netmask 0xff000000
inet6 ::1 prefixlen 128
inet6 fe80::1%lo0 prefixlen 64 scopeid 0x1
groups: lo
nd6 options=21
hn0: flags=1008843 metric 0 mtu 1500
options=66ef07bb
ether 7c:ed:8d:83:96:7c
inet 10.42.12.134 netmask 0xffffffe0 broadcast 10.42.12.159
media: Ethernet 100GBase-CR4
status: active
nd6 options=29
mce0: flags=1008a43 metric 0 mtu 1500
options=66ef07bb
ether 7c:ed:8d:83:96:7c
media: Ethernet 100GBase-CR4
status: active
nd6 options=29
Starting devd.
Starting event2 moused.
Starting event3 moused.
route: message indicates error: File exists
add host 127.0.0.1: gateway lo0 fib 0: route already in table
route: message indicates error: File exists
add host ::1: gateway lo0 fib 0: route already in table
add net fe80::: gateway ::1
add net ff02::: gateway ::1
add net ::ffff:0.0.0.0: gateway ::1
add net ::0.0.0.0: gateway ::1
/usr/local/bin/cloud-init startingWARN: no logging configured! (tried 0 configs)
Setting up basic logging...
Cloud-init v. 25.3 running 'init' at Thu, 08 Jan 2026 14:04:34 +0000. Up 1223.0178318023682 seconds.
2026-01-08 14:04:34,537 - log_util.py[DEBUG]: Cloud-init v. 25.3 running 'init' at Thu, 08 Jan 2026 14:04:34 +0000. Up 1223.0178318023682 seconds.
2026-01-08 14:04:34,540 - main.py[INFO]: PID [7026] started cloud-init 'init'.
2026-01-08 14:04:34,545 - main.py[DEBUG]: No kernel command line url found.
2026-01-08 14:04:34,547 - main.py[DEBUG]: Not closing stdin, stdin is a tty.
2026-01-08 14:04:34,551 - util.py[DEBUG]: Writing to /var/log/cloud-init.log - ab: [640] 0 bytes
2026-01-08 14:04:34,553 - util.py[DEBUG]: Changing the ownership of /var/log/cloud-init.log to 0:0
2026-01-08 14:04:34,555 - util.py[DEBUG]: Reading from /var/lib/cloud/data/python-version (quiet=False)
2026-01-08 14:04:34,558 - util.py[DEBUG]: Reading 4 bytes from /var/lib/cloud/data/python-version
2026-01-08 14:04:34,562 - main.py[DEBUG]: Will wait for network connectivity before continuing
2026-01-08 14:04:34,566 - stages.py[DEBUG]: Using distro class
2026-01-08 14:04:34,568 - activators.py[DEBUG]: Using selected activator: from priority: ['eni', 'netplan', 'network-manager', 'networkd', 'ifconfig']
2026-01-08 14:04:34,571 - ubuntu.py[ERROR]: Failed to wait for network:
2026-01-08 14:04:34,577 - subp.py[DEBUG]: Running command ['ifconfig', '-a'] with allowed return codes [0, 1] (shell=False, capture=True)
2026-01-08 14:04:34,600 - subp.py[DEBUG]: Running command ['netstat', '--route', '--numeric', '--extend'] with allowed return codes [0, 1, 64] (shell=False, capture=True)
2026-01-08 14:04:34,684 - performance.py[DEBUG]: Running ['netstat', '--route', '--numeric', '--extend'] took 0.083 seconds
2026-01-08 14:04:34,685 - subp.py[DEBUG]: Running command ['netstat', '-A', 'inet6', '--route', '--numeric'] with allowed return codes [0, 1] (shell=False, capture=True)
ci-info: ++++++++++++++++++++++++++++++Net device info+++++++++++++++++++++++++++++++
ci-info: +--------+-------+----------------+------------+-------+-------------------+
ci-info: | Device | Up | Address | Mask | Scope | Hw-Address |
ci-info: +--------+-------+----------------+------------+-------+-------------------+
ci-info: | hn0 | True | 10.42.12.134 | 0xffffffe0 | . | 7c:ed:8d:83:96:7c |
ci-info: | lo0 | True | 127.0.0.1 | 0xff000000 | . | . |
ci-info: | lo0 | True | ::1/128 | . | . | . |
ci-info: | lo0 | True | fe80::1%lo0/64 | . | 0x1 | . |
ci-info: | mce0 | False | . | . | . | 7c:ed:8d:83:96:7c |
ci-info: +--------+-------+----------------+------------+-------+-------------------+
ci-info:
```

## freebsd boot logs before cloud-init
```
2J|Loading /boot/loader.conf.local
/-\|/c-\Loading kernel...
|/-\/boot/kernel/kernel text=0x1865e8 |text=0xdbf5d4 /-\text=0x4474db |data=0x180+0xe80 data=0x1972e0+0x468d20 /0x8+0x1949a0+0x8+0x1b90eb
Loading configured modules...
-\/etc/hostid size=0x25
|/boot/entropy size=0x1000
/-\|/boot/kernel/zfs.ko /size 0x620c10 at 0x2150000
-\|/boot/kernel/mlx4en.ko size 0x25568 at 0x2771000
loading required module 'mlx4'
/-/boot/kernel/mlx4.ko size 0x61870 at 0x2797000

Booting [/boot/kernel/kernel]...
staging 0x36800000 (not copying) tramp 0x366ce000 PT4 0x366c5000
Start @ 0xffffffff80387000 ...
\|Loading splash ok
EFI framebuffer information:
addr, size 0x40000000, 0x800000
dimensions 1024 x 768
stride 1024
masks 0x00ff0000, 0x0000ff00, 0x000000ff, 0xff000000
---<>---
Copyright (c) 1992-2025 The FreeBSD Project.
Copyright (c) 1979, 1980, 1983, 1986, 1988, 1989, 1991, 1992, 1993, 1994
The Regents of the University of California. All rights reserved.
FreeBSD is a registered trademark of The FreeBSD Foundation.
FreeBSD 15.0-RELEASE-p1 releng/15.0-n280999-7bceec30b351 GENERIC amd64
FreeBSD clang version 19.1.7 (https://github.com/llvm/llvm-project.git llvmorg-19.1.7-0-gcd708029e0b2)
SRAT: Ignoring memory at addr 0x2c0000000
SRAT: Ignoring memory at addr 0x1000000000
SRAT: Ignoring memory at addr 0x10000000000
SRAT: Ignoring memory at addr 0x20000000000
SRAT: Ignoring memory at addr 0x40000000000
SRAT: Ignoring memory at addr 0x80000000000
SRAT: Ignoring memory at addr 0x100000000000
SRAT: Ignoring memory at addr 0x200000000000
VT(efifb): resolution 1024x768
Hyper-V Version: 10.0.26102 [SP1]
Features=0xae7f
PM Features=0x20 [C2]
Features3=0xc0bed7b2
Timecounter "Hyper-V" frequency 10000000 Hz quality 2000
CPU: Intel(R) Xeon(R) Platinum 8370C CPU @ 2.80GHz (2793.44-MHz K8-class CPU)
Origin="GenuineIntel" Id=0x606a6 Family=0x6 Model=0x6a Stepping=6
Features=0x1f83fbff
Features2=0xfffa3223
AMD Features=0x2c100800
AMD Features2=0x121
Structured Extended Features=0xf1bf2fbb
Structured Extended Features2=0x415f46
Structured Extended Features3=0x20000010
XSAVE Features=0xf
IA32_ARCH_CAPS=0xc000165
VT-x: PAT,HLT,PAUSE,EPT,UG,VPID
Hypervisor: Origin = "Microsoft Hv"
real memory = 8589934592 (8192 MB)
avail memory = 8256233472 (7873 MB)
Event timer "LAPIC" quality 100
ACPI APIC Table:
FreeBSD/SMP: Multiprocessor System Detected: 2 CPUs
FreeBSD/SMP: 1 package(s) x 1 core(s) x 2 hardware threads
random: registering fast source Intel Secure Key Seed
random: fast provider: "Intel Secure Key Seed"
random: unblocking device.
ioapic0 irqs 0-23
Launching APs: 1
random: entropy device external interface
Timecounter "Hyper-V-TSC" frequency 10000000 Hz quality 3000
kbd0 at kbdmux0
efirtc0:
efirtc0: registered as a ime-of-day clock, resolution 1.000000s
smbios0: at iomem 0x3ff86000-0x3ff86017
smbios0: Entry point: v3 (64-bit), Version: 3.1
aesni0:
acpi0:
atrtc0: port 0x70-0x71 irq 8 on acpi0
atrtc0: registered as a time-of-day clock, resolution 1.000000s
Event timer "RTC" frequency 32768 Hz quality 0
Timecounter "ACPI-fast" frequency 3579545 Hz quality 900
acpi_timer0: <32-bit timer at 3.579545MHz> port 0x408-0x40b on acpi0
cpu0: on acpi0
acpi_syscontainer0: on acpi0
vmbus0: on acpi_syscontainer0
uart0: <16550 or compatible> port 0x3f8-0x3ff irq 4 flags 0x10 on acpi0
uart0: console (115200,n,8,1)
uart1: <16550 or compatible> port 0x2f8-0x2ff irq 3 on acpi0
vmgenc0: on acpi0
random: registering fast source VM Generation ID
vmbus_res0: irq 5 on acpi0
Timecounters tick every 10.000 msec
ZFS filesystem version: 5
ZFS storage pool version: features support (5000)
usb_needs_explore_all: no devclass
vmbus0: version 4.0
hvet0: on vmbus0
Event timer "Hyper-V" frequency 10000000 Hz quality 1000
hvkbd0: on vmbus0
kbd1 at hvkbd0
hvheartbeat0: on vmbus0
hvkvp0: on vmbus0
hvshutdown0: on vmbus0
hvtimesync0: on vmbus0
hvtimesync0: RTT
hn0: on vmbus0
hn0: Ethernet add: 7c:ed:8d:83:96:7c
hn0: link state changed to UP
storvsc0: on vmbus0
storvsc1: on vmbus0
cd0 at storvsc0 bus 0 scbus0 target 0 lun 2
cd0: Removable CD-ROM SPC-3 SCSI device
cd0: 300.000MB/s transfers
cd0: 0MB (320 2048 byte sectors)
Trying to mount root from zfs:zroot/ROOT/default []...
da0 at storvsc0 bus 0 scbus0 target 0 lun 0
da0: Fixed Direct Access SPC-3 SCSI device
da0: 300.000MB/s transfers
da0: Command Queueing enabled
da0: 30754MB (62984192 512 byte sectors)
(da0:storvsc0:0:0:0): CACHE PAGE TOO SHORT data len 15 desc len 0
(da0:storvsc0:0:0:0): Mode page 8 missing, disabling SYNCHRONIZE CACHE
Dual Console: Serial Primary, Video Secondary
hn0: got notify, nvs type 128
pcib0: on vmbus0
pcib0: PCI VMBus using version 0x10004
pci0: on pcib0
mlx5_core0: at device 2.0 on pci0
mlx5: Mellanox Core driver 3.7.1 (November 2021)mlx5_core0: WARN: mlx5_init_once:1003:(pid 0): Unable to find vendor specific capabilities
mce0: Ethernet address: 7c:ed:8d:83:96:7c
mce0: link state changed to DOWN
hn0: link state changed to DOWN
mlx5_core0: WARN: mlx5_fwdump_prep:91:(pid 0): Unable to find vendor-specific capability, error 2
Setting hostuuid: 80962478-c183-4946-9330-6f461f270a6d.
Setting hostid: 0x766a7ed6.
Starting file system checks:
/dev/gpt/efiboot0: FILESYSTEM CLEAN; SKIPPING CHECKS
Mounting local filesystems:.
no pools available to import
Autoloading module: hv_hid
hvhid0: on vmbus0
hidbus0: on hvhid0
hms0: on hidbus0
hms0: 5 buttons and [XYW] coordinates ID=0
/usr/local/lib/cloud-init/ds-identify starting
eval: /usr/local/lib/cloud-init/ds-identify: not found
Setting up harvesting: PURE_VMGENID,PURE_RDSEED,RANDOMDEV,[CALLOUT],[UMA],[FS_ATIME],SWI,INTERRUPT,NET_NG,[NET_ETHER],NET_TUN,MOUSE,KEYBOARD,ATTACH,CACHED
Feeding entropy: .
Setting hostname: pkrvmr2e7i2inv7.
ELF ldconfig path: /lib /usr/lib /usr/local/lib /usr/local/lib/compat/pkg /usr/local/lib/compat/pkg /usr/local/lib/perl5/5.42/mach/CORE
32-bit compatibility ldconfig path: /usr/lib32
/usr/local/bin/cloud-init startingWARN: no logging configured! (tried 0 configs)
Setting up basic logging...
Cloud-init v. 25.3 running 'init-local' at Thu, 08 Jan 2026 13:44:29 +0000. Up 18.450096130371094 seconds.
2026-01-08 13:44:30,084 - log_util.py[DEBUG]: Cloud-init v. 25.3 running 'init-local' at Thu, 08 Jan 2026 13:44:29 +0000. Up 18.450096130371094 seconds.
2026-01-08 13:44:30,087 - main.py[INFO]: PID [298] started cloud-init 'init-local'.
```

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.