Automatically create storage config from source dire... | coreos/ignition

Created originally in Butane by @PhrozenByte: https://github.com/coreos/butane/issues/295

/kind enhancement

I'm using Butane to provision some servers with rather complex setups and unfortunately I've to say that Butane's way of handling files, directories and links makes things quite hard right now.

Adding new files, directories and links to a server is probably the most common task when provisioning a server, however, right now one has to add every single file manually to config.bu. The biggest problem are files, as one has to either inline their contents (making the config unreadable pretty fast), or one has to create the source file separately and set a source path in config.bu. Luckily one doesn't have to add full directory trees to config.bu, because Ignition will silently create missing directories anyway - unless we want the directories not to be owned by root. Then we must add the full directory tree, too.

Here's an example of such a config.bu - that needs to be multiplied a few times, once per rootless container (you don't have to actually check the config, it doesn't matter, it's just an example).

variant: fcos
version: 1.4.0
passwd:
  groups:
    - name: acme
      gid: 2000
    - name: acme_croot
      gid: 100000
  users:
    - name: acme
      uid: 2000
      primary_group: acme
      home_dir: /srv/containers/acme
    - name: acme_croot
      uid: 100000
      primary_group: acme_croot
      home_dir: /srv/containers/acme
      no_create_home: true
      shell: /usr/bin/nologin
      system: true
storage:
  directories:
    - path: /srv/containers/acme
      mode: 0700
      user: { name: "acme" }
      group: { name: "acme" }
    - path: /srv/containers/acme/data
      user: { name: "acme" }
      group: { name: "acme" }
    - path: /srv/containers/acme/config
      user: { name: "acme" }
      group: { name: "acme" }
    - path: /srv/containers/acme/.config
      user: { name: "acme" }
      group: { name: "acme" }
    - path: /srv/containers/acme/.config/systemd
      user: { name: "acme" }
      group: { name: "acme" }
    - path: /srv/containers/acme/.config/sxstemd/user
      user: { name: "acme" }
      group: { name: "acme" }
    - path: /srv/containers/acme/.config/systemd/user/multi-user.target.wants
      user: { name: "acme" }
      group: { name: "acme" }
  files:
    - path: /etc/subuid
      append:
        - inline: |
            acme:100000:65536
    - path: /etc/subgid
      append:
        - inline: |
            acme:100000:65536
    - path: /var/lib/systemd/linger/acme
    - path: /srv/containers/acme/.config/systemd/user/container-acme.service
      user: { name: "acme" }
      group: { name: "acme" }
      overwrite: true
      contents:
        inline: |
          [Unit]
          Description=Podman container-acme.service
          Wants=network-online.target
          After=network-online.target
          RequiresMountsFor=%t/containers
          
          [Service]
          Environment=PODMAN_SYSTEMD_UNIT=%n
          Restart=on-failure
          TimeoutStopSec=70
          ExecStartPre=/bin/rm -f %t/%n.ctr-id
          ExecStart=/usr/bin/podman run --cidfile=%t/%n.ctr-id --sdnotify=conmon --cgroups=no-conmon --replace -dt --name acme --pull always --uidmap 0:1:65536 --uidmap 65536:0:1 --gidmap 0:1:65536 --gidmap 65536:0:1 --mount type=bind,src=/srv/containers/acme/data,dst=/var/local/acme --mount type=bind,src=/srv/containers/acme/config,dst=/etc/acme ghcr.io/sgsgermany/acme:latest
          ExecStop=/usr/bin/podman stop --ignore --cidfile=%t/%n.ctr-id
          Type=notify
          NotifyAccess=all
          
          [Install]
          WantedBy=multi-user.target
  links:
    - path: /srv/containers/acme/.config/systemd/user/multi-user.target.wants/container-acme.service
      user: { name: "acme" }
      group: { name: "acme" }
      overwrite: true
      target: ../container-acme.service

This is very repetitive and error-prone (did you see the typo for /srv/containers/acme/.config/systemd/user?).

Thus I'd like to suggest to add a new feature allowing Butane to automatically create storage configs from a source directory. For this we need two additional command line options, --storage-src-dir and --storage-config-file (or similar). --storage-src-dir takes a path to a directory, --storage-config-file a file name (defaults to subconfig.bu, or similar).

Butane iterates all files, directories and links in --storage-src-dir and automatically creates entries for these files in the resulting Ignition config, adding path, contents (for files only) and target (for links only) respectively. File ownership and permissions are ignored. This allows users to add files to their Ignition config by simply creating the necessary files and directories in the --storage-src-dir directory. For all other parameters (like user, group, mode and overwrite), of a directory, file or link, one uses magic files like subconfig.bu, matching the --storage-config-file command line option.

Here's an example of a subconfig.bu stored at …/srv/containers/acme/subconfig.bu

variant: fcos-sub
version: 1.5.0
directories:
  - path: /
    mode: 0700
  - path: "*"
    user: { name: "acme" }
    group: { name: "acme" }
files:
  - path: "*"
    user: { name: "acme" }
    group: { name: "acme" }
    overwrite: true
links:
  - path: "*"
    user: { name: "acme" }
    group: { name: "acme" }
    overwrite: true

The config.bu is now way simpler.

variant: fcos
version: 1.5.0
passwd:
  groups:
    - name: acme
      gid: 2000
    - name: acme_croot
      gid: 100000
  users:
    - name: acme
      uid: 2000
      primary_group: acme
      home_dir: /srv/containers/acme
    - name: acme_croot
      uid: 100000
      primary_group: acme_croot
      home_dir: /srv/containers/acme
      no_create_home: true
      shell: /usr/bin/nologin
      system: true
storage:
  files:
    - path: /etc/subuid
      append:
        - inline: |
            acme:100000:65536
    - path: /etc/subgid
      append:
        - inline: |
            acme:100000:65536

subconfig.bu files are used to merge their config into the specification of files, directories and links matching the path pattern below this point. The path patterns are just usual glob patterns, like e.g. in .gitignore files. So, for example, the path: / directory config adds mode: 0700 to the specification of /srv/containers/acme. Since the directory pattern path: "*" matches everything, it ensures that all directories below /srv/containers/acme (including /srv/containers/acme) are owned by acme. The same for files and links, which additionally set overwrite: true.

I wrote a pre-processor for Butane with Python (unfortunately I don't know Go...) that implements exactly this (it also implements #118; no public release yet). It makes things way easier. What do you guys think about this?

Automatically create storage config from source directory

Description

Contributor guide

First steps

Research direction

Assessment