canonical / canonical/cloud-init
[docs]: Incomplete and misleading guidance on merging vendor-data
- Dominant language
- Python
- Stars
- 3.8k
- Forks
- 1.1k
- Avg merge
- 2d 23h
- Merged PRs (30d)
- 18
Description
Issue #2979 described a problem where vendor-data and user-data are not merged. I've run into the same confusion, but didn't want to necro a closed, 2 year old issue, plus I have an additional request for clarification that wasn't in that issue.
### Misleading documentation
In the [Vendor-data](https://cloudinit.readthedocs.io/en/latest/explanation/vendordata.html) page, it states:
> User-supplied cloud-config is merged over cloud-config from vendor-data.
To me, the phrase "merged over" here implies a recursive merge, with user-data taking precedence over vendor-data. Eg., given
```yaml
# user-data
map_a:
key_a: scalar from user-data
key_b:
subkey_u: hello user
list_a:
- item from user-data
top_level_scalar: foobar
# vendor-data
map_a:
key_a: scalar from vendor-data
key_b:
subkey_v: hello vendor
list_a:
- item from vendor-data
top_level_scalar: raboof
```
The combined config I expected would be
```yaml
map_a:
key_a: scalar from user-data
key_b:
subkey_u: hello user
subkey_v: hello vendor
list_a:
- item from user-data
- item from vendor-data
top_level_scalar: foobar
```
It would be helpful if the documentation was more explicit that this is not how it works:
> User-supplied cloud-config is merged, non-recursively, over cloud-config from vendor-data. When a key contains a collection (map or list) and is present in both, only the user-data collection is retained.
It might be helpful to further clarify that vendor-data is not suitable for platform-supplied defaults, but only for a platform-defined "bare" setup. The hint, "if the use of vendor-data is required for the instance to run, then vendor-data should not be used," doesn't quite land right if the use-case doesn't involve configuration that would prevent the instance from running if missing, eg. using `write_file` to create an `/etc/motd` with the vendor's ASCII art logo.
I can imagine a good reason for it to work the way that it does so I wouldn't call this a bug, but that only came after losing several hours wondering why it wasn't working the way I expected.
### Incomplete information
While searching on the topic, I came across this post: https://discuss.linuxcontainers.org/t/how-to-merge-profiles-user-vendor-data/774/17:
> Still don’t know in detail how all magic works, but when I add the `merge_how` section (https://cloudinit.readthedocs.io/en/latest/topics/merging.html) it works – without the `merge_how`, one file wins!
Notably, the [Merging page](https://cloudinit.readthedocs.io/en/latest/reference/merging.html) makes no mention at all that `merge_how` acts on vendor-data.
I took a leap of faith that an old comment on an LXD forum might be onto something. So, now I'm testing `merge_how`, following the information in the ["How to activate" section](https://cloudinit.readthedocs.io/en/latest/reference/merging.html#how-to-activate).
What I've observed it that some setting combinations have no effect, and (at least) one actually _inverts_ the behavior of user-data overriding vendor-data, so the final, combined config is only vendor-data. At least that proves that `merge_how` _does_ apply to vendor-data 🤷
This configuration causes the inversion:
```yaml
# user-data
merge_how:
- name: list
settings: [append]
apt: ...
write_files:
- {file_u1: {...}}
- ...
# vendor-data
merge_how:
- name: list
settings: [append]
# There doesn't seem to be any difference if merge_how is given in both, one, or the other.
write_files:
- {file_v1: {...}}
# combined-config
write_files:
- {file_v1: {...}}
# The apt block and all write_files items from user-data are gone
```
Since there's no documentation saying whether `merge_how` applies to vendor-data in the first place, I can't say if this is expected behavior or a bug. It feels like a bug to me, though.
This configuration does absolutely nothing, the end result being identical to not setting `merge_how` at all:
```yaml
# user-data
merge_how:
- name: list
settings: [append, recurse_list, recurse_dict]
- name: dict
settings: [no_replace, recurse_list, recurse_dict]
apt: ...
write_files:
-
- ...
# vendor-data
merge_how:
- name: list
settings: [append, recurse_list, recurse_dict]
- name: dict
settings: [no_replace, recurse_list, recurse_dict]
# Again, it doesn't seem to matter if this is in either one or both
write_files:
-
# combined-config
apt: ...
write_files:
-
- ... (but NOT file_v1)
```
I haven't gone through the whole matrix of possible merge setting combinations, but if I do happen to brute-force one that does what I want (or causes more weird behavior, like inversion) I'll be sure to update. I plan next to try varying the replace/no_replace options, since it's not clear if they mean replacing the entire collection or replacing conflicting items within the collection.
Any tips on getting the `write_files` list to merge would be appreciated! (Especially if they get added to the docs!)
Contributor guide
Assessment
This issue has not been assessed yet.