NAT not working under higher load

Debian 12 netavark 1.4.0-3 podman 4.3.1+ds1-8+b1 aardvark 1.4.0-3

I experience issues with NAT not being done, in my case only for one source address. My traffic is 99% IPv6.
I run an rsyslog container. The syslog clients send the traffic to the podman host on port 514 UDP.
With tcpdump I see that the traffic hits the host and is then DNATed to the IP of the container
```
16:15:56.208912 ens18 In IP6 2001:4aaa:28d4:7300::10.47492 > 2001:4aaa:28d4:7300::1f.514: SYSLOG daemon.error, length: 121
16:15:56.208923 podman1 Out IP6 2001:4aaa:28d4:7300::10.47492 > fda9:92d6:18ac:e10e::4ce.514: SYSLOG daemon.error, length: 121
```
One syslog client is much more active than others as it's a firewall that sends all rejected/dropped packet logs to the rsyslog container. Those packets are not NATed thus never arriving at the container.
I'm not certain that it's due to the fact that the traffic from this source is higher but it's my best guess.
```
16:15:51.520750 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:51.850439 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:52.750655 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:52.750656 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:52.860613 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:52.860630 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:53.380422 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:54.200551 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 301
16:15:54.250611 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:54.260591 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:54.600579 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
16:15:54.600806 ens18 In IP6 2001:4aaa:28d4:7300::1.52900 > 2001:4aaa:28d4:7300::1f.514: SYSLOG kernel.warning, length: 294
```

This is the network my guests use:
```json
[
{
"name": "podmannet",
"id": "cbea01b7398ff5fdf95faf1a5275d62d0691f0821a7906f4dd36bfe054956b53",
"driver": "bridge",
"network_interface": "podman1",
"created": "2023-11-01T23:35:21.231236772+01:00",
"subnets": [
{
"subnet": "10.89.0.0/24",
"gateway": "10.89.0.1"
},
{
"subnet": "fda9:92d6:18ac:e10e::/64",
"gateway": "fda9:92d6:18ac:e10e::1"
}
],
"ipv6_enabled": true,
"internal": false,
"dns_enabled": true,
"ipam_options": {
"driver": "host-local"
}
}
]
```
Here's an excerpt of the nft ruleset:
```
chain NETAVARK-DN-AA6364D1A4EDD {
[...]
ip6 saddr fda9:92d6:18ac:e10e::/64 tcp dport 514 counter packets 0 bytes 0 jump NETAVARK-HOSTPORT-SETMARK
ip6 saddr ::1 tcp dport 514 counter packets 0 bytes 0 jump NETAVARK-HOSTPORT-SETMARK
tcp dport 514 counter packets 0 bytes 0
ip6 saddr fda9:92d6:18ac:e10e::/64 udp dport 514 counter packets 0 bytes 0 jump NETAVARK-HOSTPORT-SETMARK
ip6 saddr ::1 udp dport 514 counter packets 0 bytes 0 jump NETAVARK-HOSTPORT-SETMARK
udp dport 514 counter packets 3073 bytes 509720
}
```

No source files, tests, or code entry points are named. Start by reproducing the IPv6 UDP port 514 traffic and inspect the packet captures alongside the shown nft ruleset and counters; done means identifying why traffic from the busy source is not NATed and confirming that affected syslog packets reach the container.

Written by the indexing model from the issue text.