oxidecomputer / oxidecomputer/omicron

Want deterministic VPC Subnet allocation strategy

Open
#685 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

networking
Dominant language
Rust
Stars
572
Forks
97
Avg merge
2d 12h
Merged PRs (30d)
96

Description

As part of #677, we implemented some validation on user-supplied address ranges for VPC Subnets. Specifically, that PR ensures that the provided IPv4 and IPv6 address ranges do not overlap with any existing ranges within the same VPC. In the case when the user does not provide an IPv6 address range, a random subnet from the VPC's prefix is provided. This is reasonable at this point, but could be improved to use a deterministic allocation strategy.

This work was initially deferred because it's not exactly obvious how to implement it in CockroachDB. IPv6 ranges for VPC Subnets are all /64s, specifically the upper 64 bits of the 128-bit IPv6 address stored in CockroachDB. Cockroach implements addition between an INET and an INT, but the integer required would overflow the widest integer type available, which is 64-bit. Addition between two INETs is not supported.

The most plausible strategy then is to keep track of which ranges are free (or allocated) for each VPC in a separate table. This could be implemented as something like (vpc_id, range_start, range_end), where each row gives the first and last available subnet offsets, as a u16, for a VPC. When a new allocation occurs, we'd look up the next available address, and update that record by moving range_start to range_start + 1.

Freeing is a bit more complicated. We'd probably want to coalesce the ranges as much as possible. If the freed subnet has no neighboring range, it's just a new row in the table. If it has a neighboring range above, or below, those are coalesced by updating those record(s) as appropriate. There's a lot of detail here, but it's similar to how a memory allocator might coalesce free blocks.

One difficulty is how to go from these ranges to the actual IPv6 address range. Suppose the VPC has IPv6 prefix fd00::/48, and that we've determined the next available subnet has value 2. How do we get to the fact that this subnet should be fd00:0:0:2::/64? Again, because there's no u128 type in CockroachDB, we might need to do some annoying byte-munging or bit-shifting to hack it together. We should still probably prefer to do these in the database if possible, rather than in application code, either in a transaction or with an update that's conditional on the relevant records not having changed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue names no files or tests. Start by locating the VPC subnet allocation path and its CockroachDB schema, then inspect how IPv6 prefixes and existing ranges are represented. Done means allocations are deterministic and non-overlapping, freed ranges can be reused, and the behavior is covered by tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust, sql
Domain
backend, databases
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.