containerd / containerd/runwasi
Create an API for Pre-processing layers
- Dominant language
- Rust
- Stars
- 1.3k
- Forks
- 131
- PR merge metrics
- No merged PRs in 30d
Description
We've added the ability to pre-compile layers in #405 and followed up with some improvements in #492.
During the iterations it was identified that a similar API could be used to do [other pre-initialization work as well](https://github.com/containerd/runwasi/pull/492#issuecomment-1962103124):
- composing multiple layers into a single layer
- pre-compiling (as is already done but could be done in this api instead along with other processing)
- using tools like [Wizer](https://github.com/bytecodealliance/wizer?tab=readme-ov-file#using-wizer-with-a-custom-linker) to do other pre-initialization optimizations
I would like to propose we create an API similar to the precompile but would replace it with something like `process_layers` or `initialize_layers `. This should also allow the shim to give us [processed layers that are combined](https://github.com/containerd/runwasi/pull/492#discussion_r1506639405) and the id's of ones the want to remain untouched.
One possible way to do this would look something like the precompile layer (open to suggestions):
```rust
struct ProcessedLayers {
layers: Vec
unique_cache_id: Option // return None if nothing to do
}
enum Layer {
Original(digest) // for files we want untouched
Processed(Vec)
}
fn precompile(&self, _layers: &[WasmLayer]) -> Result {
// do work
}
```
On the shim side we could take the layers create a new file that stores these digests and link it to the original image. Upon reloading the image we could look up the file, load those digests.
Questions:
- in https://github.com/containerd/runwasi/pull/492#pullrequestreview-1912475229 we add ability to re-use layers that where already processed. How to accomplish that in this scenario?
- is this an evolution of precompile api, do we add new and depreciate, or keep both?
Contributor guide
Assessment
This issue has not been assessed yet.