hashgraph / hashgraph/solo-weaver

test(network/firewall): verify coexistence with a third-party default-drop nftables ruleset on the UTM harness

Open
#1,011 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
3
Forks
0
Avg merge
3d 5h
Merged PRs (30d)
47

Description

## Problem

On hosts where a configuration-management agent already maintains an nftables ruleset, that ruleset
and the solo-provisioner host firewall will both be present during a migration, and possibly for
much longer. Both install a default-drop `input` chain at the same hook priority. We believe the
composition is a logical AND — a packet addressed to the host must be accepted by both chains, the
order between two chains at equal priority is unspecified, and a drop in either is final — and that
pod-bound service traffic never reaches either `input` chain because it is forwarded rather than
delivered locally.

All of that is reasoned from reading rulesets, not observed. It is the assumption the whole migration
plan rests on: that an operator can stand the new firewall up beside the existing one, verify access,
and remove the old rules afterwards, rather than performing a single cutover with no verification
window. If the composition behaves differently, the recommended sequence silently locks operators out
or, worse, leaves a port reachable that both rulesets intend to deny.

Two further behaviours need the same treatment. A ruleset that begins with a global flush destroys
every table on the host when its agent re-applies, not only its own; and the OS-level nftables unit
flushes globally when stopped, regardless of what its configuration contains.

## Proposed verification

Use the existing UTM harness (`task vm:start`, `task vm:ssh`, `task vm:ip`). No production host and
no real address ranges are involved: use the documentation ranges from RFC 5737
(`192.0.2.0/24`, `198.51.100.0/24`, `203.0.113.0/24`) and arbitrary high ports.

**Topology.** One VM as the node under test, plus at least two VMs standing in for distinct source
groups (for example one representing a management/jump source and one representing a monitoring or
build source). A third source with no rule in either ruleset provides the negative case.

**On the node VM**, in this order:

1. Apply a stand-in third-party ruleset written by hand for this test, in the shape such rulesets
commonly take: a global `flush ruleset`, a `table inet filter` with a default-drop `input` chain
at filter priority, named address sets per source group, a small set of accepted ports, and an
`accept`-policy `forward` chain. It must be written from scratch in the test fixtures. Do not copy
one from anywhere, and do not put real addresses or real service ports in it.
2. Stand up the solo-provisioner host firewall with `network firewall create` plus the declare and
populate sequence, granting a deliberately *different* set of ports and sources so the
intersection is observable rather than masked by an identical allowlist.
3. Run listeners with `nc -l ` for each port under test, and use `nc -vz` (or `telnet`) plus
`ping` from each source VM.

**Matrix.** For each source group and port, record reachable or not, under: third-party ruleset only;
host firewall only; both present. Include at least one port allowed by exactly one of the two, in
each direction, and one allowed by neither.

**Expected, to be confirmed or refuted:**

- A port allowed by only one of the two rulesets is *not* reachable. Acceptance by both is required.
- ICMP echo is accepted only from the sources granted it, and denied from the rest.
- A port allowed by both is reachable, and stays reachable across a re-apply of either ruleset.
- Reachability is stable regardless of which ruleset was loaded last, since the two chains share a
hook priority and their relative order is not defined.

**Service traffic.** Verify separately that traffic to a workload service port is unaffected by
either `input` chain, because it is forwarded to the pod rather than delivered to the host. The
assumption worth testing explicitly is the converse: if any workload port is host-networked, it
becomes host-destined and the intersection above applies to it, which would make a mismatch between
the two rulesets an outage rather than a non-event. Establish which is true on a node running the
real workload.

**Destructive paths**, on the same harness:

- Re-apply the stand-in third-party ruleset as its agent would on a convergence cycle. Confirm
whether solo-provisioner's tables survive, and how long the host is left unprotected if they do not.
- `systemctl restart nftables` and `systemctl stop nftables`, then confirm what remains in the kernel
and whether the loader unit re-applies. This overlaps #982 (unit ordering) and #981 (nothing
re-asserts the tables after a third-party flush); the results here should feed both.

## Acceptance

- [ ] The reachability matrix above is recorded, with the stand-in ruleset committed as a test
fixture containing no real addresses or service ports.
- [ ] The intersection behaviour is confirmed or refuted, and the migration guidance updated to match.
- [ ] The behaviour of a forwarded service port under both rulesets is established empirically, along
with whether any workload port is host-networked.
- [ ] The flush and restart paths are characterised, with the exposure window measured rather than
estimated.
- [ ] Findings are linked from #981 and #982.

Contributor guide

Open the contributing guide

Research direction

Start with the existing UTM harness using task vm:start, task vm:ssh, and task vm:ip, then inspect the firewall declare/populate flow and the real workload networking. Add a from-scratch stand-in ruleset fixture and record the source/port reachability matrix for each ruleset combination, forwarded traffic, re-apply, and nftables restart/stop paths. Commit the fixture, measure any exposure window, update migration guidance, and link findings from #981 and #982.

Written by the indexing model from the issue text.

Assessment

Tech stack
kubernetes, linux
Domain
devops, infrastructure, networking, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Clearly specified
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.