varnish / varnish/libvmod-tinykvm

Shared storage new API

Open
#9 2 comments 0 reactions 1 assignee View on GitHub

@fwsGonzo is already working on this.

Since Apr 4, 2025.

feature
Dominant language
C++
Stars
12
Forks
2
Avg merge
1h 9m
Merged PRs (30d)
1

Description

Currently the storage VM is a completely separate VM from the main VM, which runs through main() separately, with a different type argument. Instead of "request" it's "storage".

This is fine, however it takes quite a bit of redundant memory and page tables to have two separate VMs with a large (sometimes gigantic) address space.

Instead, I would like to either directly use the main VM as storage, or make a fork of the main VM and use that as storage. I will most likely implement the fork variant, as that will be the most robust solution. It's also safe to take down storage, re-initialize it if it crashes etc. Can't do that with the main VM.

But, what will the API look like now? All forks must have the same parent VM, and they must all be forked from the same state. So, how do you turn that into a good API? How do you diverge from the main VM into a storage VM in a sane way?

Initially I thought:

if (enable_storage()) {
    // Inside this we have diverged from requests and are now a storage VM
}

... but that's not possible, because forks can not start from a different main VM state. It introduces potential ghosts in the machine and lots of scary behavior.
The simplest way could be to introduce a new callback function, which is simply what happens after storage is forked:

set_storage_initializer(my_storage_function);

Not sure yet.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.