compression: gzip on inline/local contents is accepted without checking whether the contents are actually gzipped

Created originally in Butane by @w453y: https://github.com/coreos/butane/issues/730

Butane accepts compression: gzip on an inline or local file resource whose contents are not gzipped, and emits the declaration unchanged. The resulting config is valid as far as ignition-validate is concerned, and fails at first boot instead.

Butane already has these contents in memory and inspects them, it compresses them itself when that yields a smaller data URL, and sets compression accordingly. When the user sets the field instead, MakeDataURL takes it as a statement that the contents are already compressed (base/util/url.go, the else branch of util.NilOrEmpty(currentCompression)) and skips its own compression path. A gzip stream is identifiable from its first two bytes (1f 8b, RFC 1952), so the mismatch seems cheaply detectable at translation time.

Severity is low, I want to be upfront about that. The runtime failure is clean: I checked on both Flatcar and Fedora CoreOS and Ignition reports gzip: invalid header, names the file and the stage, fails ignition-files.service, and drops to an emergency shell. So this is about when the user finds out, not whether they can. The expectation stated in coreos/butane#332, that a mismatched declaration produces an Ignition error at runtime, does hold.

This came out of writing a cloud-init to Butane transpiler, where cloud-init's write_files encoding: gzip+b64 has to be turned into either plain contents or data:;base64, plus compression: gzip. Getting that choice backwards produces a config that passes --strict and fails at boot, which is why I went looking.

Reproduction

butane 0.29.0.

variant: flatcar
version: 1.1.0
storage:
  files:
    - path: /etc/t
      mode: 0644
      contents:
        inline: not actually gzipped
        compression: gzip

$ butane --pretty --strict < gz.bu
{
  "ignition": { "version": "3.4.0" },
  "storage": {
    "files": [
      {
        "path": "/etc/t",
        "contents": {
          "compression": "gzip",
          "source": "data:,not%20actually%20gzipped"
        },
        "mode": 420
      }
    ]
  }
}

Same with local:, where Butane reads the file itself. variant: fcos / version: 1.5.0 behaves identically.

For comparison, without the declaration Butane compresses the same content itself when it is worth it, e.g. 5000 bytes of A comes back as data:;base64,H4sIA... with compression: gzip set, so the magic bytes are something Butane produces in its own path a few lines further down.

Suggested fix

For inline and local only, where the contents are available: reject a user-supplied compression: gzip when the content does not start with 1f 8b.

Deliberately not suggesting:

• anything for source resources, since Butane cannot see remote content and cannot second-guess a child config's intent (the reasoning in #332).
• the inverse check (flagging undeclared content that is gzipped), since #332 explicitly excepts users who intend to write gzip-compressed data.

Related: coreos/butane#100, coreos/butane#123, coreos/butane#124, coreos/butane#244, coreos/butane#332, coreos/butane#341. coreos/butane#244 is the closest, the openshift variants already fail when compression is manually configured, though for a different reason.

Happy to send a PR if this seems worth doing.

Start in base/util/url.go, especially the else branch when compression is already declared, and trace how inline and local contents reach MakeDataURL. Use the supplied Butane reproduction to verify that non-gzipped content with compression: gzip is rejected during translation, while valid gzip content and source resources retain their existing behavior.