composefs / composefs/composefs-rs

Fix oci whiteout implementation

Open
#8 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Rust
Stars
93
Forks
30
Avg merge
2d 10h
Merged PRs (30d)
13

Description

I implemented whiteouts in the image-merging code without having read the spec.

https://github.com/opencontainers/image-spec/blob/main/layer.md

There are two things that need fixing:

  • Whiteout files MUST only apply to resources in lower/parent layers.
  • Files that are present in the same layer as a whiteout file can only be hidden by whiteout files in subsequent layers.

When processing the second layer, a/.wh..wh..opq is applied first, before creating the new version of a/b, regardless of the ordering in which the whiteout file was encountered.

Implementations SHOULD generate layers such that the whiteout files appear before sibling directory entries.

If we encounter a whiteout file in a layer tarball after that file was added in the same tarball we're going to delete it again. We could argue that the tarball is not a valid layer because it attempts to do something that the spec says that it shouldn't, but at the very least we should try to detect this situation, and probably we do need to support it (since a SHOULD is only a SHOULD).

This is going to be very difficult. The good news: this isn't a problem as long as we're only dealing with layers produced in the way that they SHOULD have been.

  • In addition to expressing that a single entry should be removed from a lower layer, layers may remove all of the children using an opaque whiteout entry.
  • An opaque whiteout entry is a file with the name .wh..wh..opq indicating that all siblings are hidden in the lower layer.

We just need to add support for this (but need to be careful about the interaction with the above). Assuming we've managed to solve the above issue, this issue will be trivial.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the image-merging code and the OCI image layer specification linked in the issue. Trace how whiteouts are applied within and across layers, including .wh..wh..opq, then verify that same-layer entries are not removed and opaque whiteouts hide lower-layer siblings.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
operating-systems
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.