26.7.3_8: radix4_lockless FIB rebuild failures cause incorrect IPv4 routing despite correct routing table

No source file or test is named. Start by reproducing the failure with net.route.algo.inet.algo=radix4_lockless and tracing the radix4_lockless setup_fd_instance and rebuild_fd failures, then compare the radix4 path. Done means the IPv4 FIB rebuilds successfully and normal traffic to 10.10.10.2 follows wg0 without changing the route configuration.

Important notices

Before you add a new report, we ask you kindly to acknowledge the following:

• I have read the contributing guide lines at https://github.com/opnsense/src/blob/master/CONTRIBUTING.md
• I am convinced that my issue is new after having checked both open and closed issues at https://github.com/opnsense/src/issues?q=is%3Aissue

Describe the bug

On an OPNsense DEC850v2 running OPNsense 26.7.3_8, the IPv4 radix4_lockless FIB algorithm repeatedly fails to rebuild.

The visible IPv4 routing table and nexthop information remain correct, but actual IPv4 packet routing does not follow those routes.

The problem was discovered because an existing WireGuard road-warrior configuration stopped providing IPv4 connectivity after restoring the OPNsense configuration on a replacement SSD.

The WireGuard tunnel itself remained healthy:

• Handshakes were successful.
• The peer routes were present.
• IPv6 through the same WireGuard tunnel worked normally.
• IPv4 traffic worked when normal FIB lookup was bypassed with ping -r.
• Switching only the IPv4 FIB algorithm from radix4_lockless to radix4 immediately restored normal IPv4 routing and full Internet access through WireGuard.

No WireGuard, NAT, PF or firewall rule changes were required to restore connectivity.

Last known working version: exact version unknown. The same WireGuard configuration had worked for years before the SSD replacement/reinstall. The problem was observed on OPNsense 26.7.3_8.

To Reproduce

1. Run OPNsense 26.7.3_8 with the default IPv4 FIB algorithm:

   net.route.algo.inet.algo: radix4_lockless
   

2. Available IPv4 FIB algorithms on this system:

   net.route.algo.inet.algo_list: bsearch4, radix4_lockless, radix4
   

3. Use a directly connected WireGuard IPv4 network. In this case:

   wg0: 10.10.10.1/24
   peer: 10.10.10.2/32
   

4. Verify that the routing table reports the correct WireGuard route:

   # route -n get 10.10.10.2
   
      route to: 10.10.10.2
   destination: 10.10.10.2
           fib: 0
     interface: wg0
         flags: <UP,HOST,DONE,STATIC>
           mtu: 1420
   

5. Verify the routing table and nexthop:

   # netstat -rWn4
   
   Destination        Gateway      Flags   Nhop#   Mtu    Netif
   10.10.10.0/24      link#11      U          14   1420   wg0
   10.10.10.2         link#11      UHS        16   1420   wg0
   

   The nexthop itself also looks correct:

   # netstat -4onW
   
   Idx  Type        IFA         Gateway      Flags   Mtu   Netif
   16   v4/resolve  10.10.10.1  wg0/resolve  HS      1420  wg0
   

6. With radix4_lockless active, a normal IPv4 ping fails:

   # ping -c 1 10.10.10.2
   
   PING 10.10.10.2 (10.10.10.2): 56 data bytes
   
   --- 10.10.10.2 ping statistics ---
   1 packets transmitted, 0 packets received, 100.0% packet loss
   

7. PF shows the locally generated IPv4 packet incorrectly using the PPPoE/WAN path instead of wg0, despite the routing table and nexthop above:

   all icmp 45.142.146.190 -> 10.10.10.2
   origif: pppoe0
   

8. Bypassing normal route lookup works immediately:

   # ping -r -S 10.10.10.1 -c 1 10.10.10.2
   
   64 bytes from 10.10.10.2: icmp_seq=0 ttl=64 time=36.075 ms
   
   1 packets transmitted, 1 packets received, 0.0% packet loss
   

   PF then shows the expected interface:

   all icmp 10.10.10.1 -> 10.10.10.2
   origif: wg0
   

9. IPv6 to the same WireGuard peer works normally:

   # ping6 -c 1 2a10:3781:315c:2::2
   
   16 bytes from 2a10:3781:315c:2::2, icmp_seq=0 hlim=64 time=29.844 ms
   
   1 packets transmitted, 1 packets received, 0.0% packet loss
   

10. Check the active IPv4 FIB algorithm:

# sysctl net.route.algo.inet.algo

net.route.algo.inet.algo: radix4_lockless

11. The kernel log contains repeated radix4_lockless FIB rebuild failures:

[fib_algo] inet.0 setup_fd_instance: radix4_lockless algo instance setup failed, failures=14
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd_flm: table rebuild failed
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd: sync rebuild failed

These errors continue repeatedly, with the failure counter reaching at least 42.

12. Change only the IPv4 FIB algorithm at runtime:

# sysctl net.route.algo.inet.algo=radix4

net.route.algo.inet.algo: radix4_lockless -> radix4

13. Clear the relevant PF test states and repeat the exact same normal ping:

# ping -c 1 10.10.10.2

64 bytes from 10.10.10.2: icmp_seq=0 ttl=64 time=33.097 ms

1 packets transmitted, 1 packets received, 0.0% packet loss

14. PF now correctly shows:

all icmp 10.10.10.1 -> 10.10.10.2
origif: wg0

15. Full IPv4 Internet connectivity through the WireGuard tunnel also immediately starts working again.

No WireGuard, PF, NAT, firewall or route configuration change was required. Changing only the IPv4 FIB algorithm from radix4_lockless to radix4 restores connectivity.

Expected behavior

radix4_lockless should successfully build and maintain the IPv4 FIB.

Actual IPv4 packet routing should follow the routes and nexthops shown by route and netstat.

A FIB rebuild failure should not leave the system in a state where:

• route -n get reports the correct interface;
• netstat -rWn4 reports the correct nexthop;
• netstat -4onW reports the correct IFA and interface;
• but actual IPv4 packet lookup uses a different interface/path.

For the example above, traffic for 10.10.10.2 should be transmitted through wg0, as shown in the routing table.

Describe alternatives you considered

The following were tested during diagnosis and did NOT resolve the issue:

• Recreating the 10.10.10.2/32 host route through wg0.
• Explicitly setting the route IFA to 10.10.10.1.
• Disabling the OPNsense force-gateway rule.
• Disabling net.pf.share_forward.
• Disabling IPFW/dummynet temporarily.
• Testing WireGuard source NAT rules.
• Testing PF antispoof rules.
• Checking for Suricata divert-to rules on wg0.
• Checking WireGuard AllowedIPs, peer routes and handshakes.
• Checking the IPv4 route and kernel nexthop in detail.

The problem also affects forwarded/NATed IPv4 traffic.

A TCP SYN from the WireGuard client enters wg0 and is correctly source-NATed and transmitted through pppoe0.

The Internet host returns a SYN/ACK and PF correctly reverse-NATs it back to 10.10.10.2, but the resulting IPv4 packet is not transmitted through wg0.

Example capture on pppoe0:

45.142.146.190.53236 > 1.1.1.1.443: SYN
1.1.1.1.443 > 45.142.146.190.53236: SYN,ACK
1.1.1.1.443 > 10.10.10.2.50372: SYN,ACK

The final reverse-NATed SYN/ACK is not seen leaving on wg0.

This was also reproduced with the OPNsense force-gateway route-to rule disabled, so the force-gateway rule is not the cause.

Changing only:

net.route.algo.inet.algo=radix4

immediately restores both local IPv4 routing to the WireGuard peer and full forwarded IPv4 Internet connectivity.

Current workaround:

System > Settings > Tunables

net.route.algo.inet.algo = radix4

Screenshots

If required, I can provide a screenshot of the persistent OPNsense tunable:

net.route.algo.inet.algo = radix4

Relevant log files

The kernel repeatedly reports failures from radix4_lockless.

Example output:

[fib_algo] inet.0 setup_fd_instance: radix4_lockless algo instance setup failed, failures=14
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd_flm: table rebuild failed
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd: sync rebuild failed

[fib_algo] inet.0 setup_fd_instance: radix4_lockless algo instance setup failed, failures=20
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd_flm: table rebuild failed
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd: sync rebuild failed

[fib_algo] inet.0 setup_fd_instance: radix4_lockless algo instance setup failed, failures=30
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd_flm: table rebuild failed
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd: sync rebuild failed

[fib_algo] inet.0 setup_fd_instance: radix4_lockless algo instance setup failed, failures=40
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd_flm: table rebuild failed
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd: sync rebuild failed

[fib_algo] inet.0 setup_fd_instance: radix4_lockless algo instance setup failed, failures=41
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd_flm: table rebuild failed
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd: sync rebuild failed

[fib_algo] inet.0 setup_fd_instance: radix4_lockless algo instance setup failed, failures=42
[fib_algo] inet.0 (radix4_lockless#43) rebuild_fd_flm: table rebuild failed

The failure counter therefore reached at least 42 during testing.

Additional context

The issue is IPv4-specific in this setup.

IPv6 uses:

net.route.algo.inet6.algo: radix6_lockless

and IPv6 connectivity through the same WireGuard peer works correctly.

The decisive A/B test was:

radix4_lockless:
normal IPv4 routing to the peer fails

radix4:
normal IPv4 routing to the peer immediately works

No reboot or other configuration change was required between these tests.

There is also an OPNsense forum report from another user running OPNsense 26.7.3_8 on a DEC750 showing the same radix4_lockless FIB setup/rebuild failures:

https://forum.opnsense.org/index.php?topic=52842.0

In that case the visible symptom was WAN connectivity failure.

My system shows a different visible symptom, but provides a reproducible A/B test demonstrating that changing only the IPv4 FIB algorithm from radix4_lockless to radix4 immediately restores the broken IPv4 datapath.

Environment

OPNsense 26.7.3_8 (amd64)
Hardware: OPNsense DEC850v2

WAN:
PPPoE
Interface: pppoe0
IPv4 address during testing: 45.142.146.190

WireGuard:
Interface: wg0
IPv4: 10.10.10.1/24
Test peer: 10.10.10.2/32
MTU: 1420

IPv4 FIB:
net.route.algo.inet.algo: radix4_lockless

Available IPv4 algorithms:
net.route.algo.inet.algo_list: bsearch4, radix4_lockless, radix4

IPv6 FIB:
net.route.algo.inet6.algo: radix6_lockless

Number of FIBs:
net.fibs: 1

Working workaround:
net.route.algo.inet.algo=radix4