canonical / canonical/cloud-init
trying to load metadata from network before configuring the network
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 1.1k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 18
Description
# Bug report
During boot cloud-init attempts to contact the metadata and reporting services prior to configuring the network.
During cloud-init of a successfully configured node cloud-init attempts to retrieve metadata from MaaS BEFORE initializing the network.
```
2024-10-02 02:18:39,147 - __init__.py[DEBUG]: Detected platform: DataSourceMAAS [None]. Checking for active instance data
2024-10-02 02:18:39,151 - url_helper.py[DEBUG]: [0/1] open 'http://10.10.10.10:5248/MAAS/metadata/2012-03-01/meta-data/instance-id' with {'url': 'http://10.10.10.10:5248/MAAS/metadata/2012-03-01/meta-data/instance-id', 'stream': False, 'allow_redirects': True, 'method': 'GET', 'timeout': 50.0, 'headers': {'User-Agent': 'Cloud-Init/23.4-7.el9_4.6.alma.1', 'Authorization': 'OAuth oauth_nonce="****", oauth_timestamp="1727835519", oauth_version="1.0", oauth_signature_method="PLAINTEXT", oauth_consumer_key="****", oauth_token="****", oauth_signature="****"'}} configuration
2024-10-02 02:18:39,154 - url_helper.py[DEBUG]: Calling 'None' failed [0/120s]: request error [HTTPConnectionPool(host='10.10.10.10', port=5248): Max retries exceeded with url: /MAAS/metadata/2012-03-01/meta-data/instance-id (Caused by NewConnectionError(': Failed to establish a new connection: [Errno 101] Network is unreachable'))]
```
It also fails to post status messages. In fact, the cloud-init log has dozens and dozens of reports of network failure. After about 10 pages of this and 2 minutes of failures, we get down to:
```
2024-10-02 02:20:45,347 - util.py[DEBUG]: Reading from /sys/class/net/enp129s0f0/address (quiet=False)
2024-10-02 02:20:45,347 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp129s0f0/address
...snip repeat for each interface...
2024-10-02 02:20:45,347 - util.py[DEBUG]: Reading from /sys/class/net/lo/address (quiet=False)
2024-10-02 02:20:45,347 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/lo/address
2024-10-02 02:20:45,348 - util.py[DEBUG]: Reading from /sys/class/net/enp129s0f0/address (quiet=False)
2024-10-02 02:20:45,348 - util.py[DEBUG]: Read 18 bytes from /sys/class/net/enp129s0f0/address
...snip repeat for each interface...
2024-10-02 02:20:45,348 - util.py[DEBUG]: Reading from /usr/lib/python3.9/site-packages/cloudinit/config/schemas/schema-network-config-v1.json (quiet=False)
```
and then we configure the network. This is clearly out of order, no?
Naive ideas about improvements:
* Configure the network before querying the network for metadata
* Queue all status reports until the network is available
I reported this to the maas project, they claim it's a cloud-init bug. If there’s anything I or maas (via curtin) can do in the configuration to address it, please supply clue-by-four.
## Steps to reproduce the problem
I'll happily supplied the complete cloud.config.d files to someone offline, but I can't post them here. Key points are
```shell
$ cd /etc/cloud/cloud.cfg.d
$ cat 50-cloudconfig-maas-datasource.cfg
datasource_list: [ MAAS ]
$ cat 50-cloudconfig-maas-cloud-config.cfg
#cloud-config
datasource:
MAAS:
metadata_url: http://10.10.10.10:5248/MAAS/metadata/
...keys snipped...
$ cat 50-cloudconfig-maas-reporting.cfg
#cloud-config
reporting:
maas:
metadata_url: http://10.10.10.10:5248/MAAS/metadata/status/*****
type: webhook
...keys snipped...
# cat 50-curtin-networking.cfg
network:
config:
- id: enp129s0f0
...snip...
version: 1
```
## Environment details
- Cloud-init version:
- Operating System Distribution:
- Cloud provider, platform or installer type:
## cloud-init logs
[cloud-init.tar.gz](https://github.com/user-attachments/files/17234732/cloud-init.tar.gz)
Contributor guide
Assessment
This issue has not been assessed yet.