canonical / canonical/cloud-init

trying to load metadata from network before configuring the network

Open
#5,771 8 comments 1 reaction 0 assignees View on GitHub
bug incomplete
Dominant language
Python
Stars
3.8k
Forks
1.1k
Avg merge
2d 23h
Merged PRs (30d)
18

Description

# Bug report

During boot cloud-init attempts to contact the metadata and reporting services prior to configuring the network.

During cloud-init of a successfully configured node cloud-init attempts to retrieve metadata from MaaS BEFORE initializing the network.

```
2024-10-02 02:18:39,147 - __init__.py[DEBUG]: Detected platform: DataSourceMAAS [None]. Checking for active instance data
2024-10-02 02:18:39,151 - url_helper.py[DEBUG]: [0/1] open 'http://10.10.10.10:5248/MAAS/metadata/2012-03-01/meta-data/instance-id' with {'url': 'http://10.10.10.10:5248/MAAS/metadata/2012-03-01/meta-data/instance-id', 'stream': False, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0, 'headers': {'User-Agent': 'Cloud-Init/23.4-7.el9_4.6.alma.1', 'Authorization': 'OAuth oauth_nonce="****", oauth_timestamp="1727835519", oauth_version="1.0", oauth_signature_method="PLAINTEXT", oauth_consumer_key="****", oauth_token="****", oauth_signature="****"'}} configuration
2024-10-02 02:18:39,154 - url_helper.py[DEBUG]: Calling 'None' failed [0/120s]: request error [HTTPConnectionPool(host='10.10.10.10', port=5248): Max retries exceeded with url: /MAAS/metadata/2012-03-01/meta-data/instance-id (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))]
```

It also fails to post status messages. In fact, the cloud-init log has dozens and dozens of reports of network failure. After about 10 pages of this and 2 minutes of failures, we get down to:

```
2024-10-02 02:20:45,347 - util.py[DEBUG]: Reading from /sys/class/net/enp129s0f0/address (quiet=False)
2024-10-02 02:20:45,347 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp129s0f0/address
...snip repeat for each interface...

2024-10-02 02:20:45,347 - util.py[DEBUG]: Reading from /sys/class/net/lo/address (quiet=False)
2024-10-02 02:20:45,347 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/lo/address
2024-10-02 02:20:45,348 - util.py[DEBUG]: Reading from /sys/class/net/enp129s0f0/address (quiet=False)
2024-10-02 02:20:45,348 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp129s0f0/address
...snip repeat for each interface...

2024-10-02 02:20:45,348 - util.py[DEBUG]: Reading from /usr/lib/python3.9/site-packages/cloudinit/config/schemas/schema-network-config-v1.json (quiet=False)
```

and then we configure the network. This is clearly out of order, no?

Naive ideas about improvements:

* Configure the network before querying the network for metadata
* Queue all status reports until the network is available

I reported this to the maas project, they claim it's a cloud-init bug. If there’s anything I or maas (via curtin) can do in the configuration to address it, please supply clue-by-four.

## Steps to reproduce the problem

I'll happily supplied the complete cloud.config.d files to someone offline, but I can't post them here. Key points are

```shell
$ cd /etc/cloud/cloud.cfg.d
$ cat 50-cloudconfig-maas-datasource.cfg
datasource_list: [ MAAS ]

$ cat 50-cloudconfig-maas-cloud-config.cfg
#cloud-config
datasource:
MAAS:
metadata_url: http://10.10.10.10:5248/MAAS/metadata/
...keys snipped...

$ cat 50-cloudconfig-maas-reporting.cfg
#cloud-config
reporting:
maas:
metadata_url: http://10.10.10.10:5248/MAAS/metadata/status/*****
type: webhook
...keys snipped...

# cat 50-curtin-networking.cfg
network:
config:
- id: enp129s0f0
...snip...
version: 1
```

## Environment details
- Cloud-init version:
- Operating System Distribution:
- Cloud provider, platform or installer type:

## cloud-init logs

[cloud-init.tar.gz](https://github.com/user-attachments/files/17234732/cloud-init.tar.gz)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.