RUSTSEC-2026-0118: NSEC3 closest-encloser proof validation enters unbounded loop on cross-zone responses

NSEC3 closest-encloser proof validation enters unbounded loop on cross-zone responses

The NSEC3 closest-encloser proof validation in hickory-proto's
DnssecDnsHandle walks from the QNAME up to the SOA owner name, building a
list of candidate encloser names. The iterator used assumes the
QNAME is a descendant of the SOA owner, terminating only when the current
candidate equals the SOA name. When the SOA in a response's authority section
is not an ancestor of the QNAME, the loop stalls at the DNS root and never
terminates, repeatedly calling Name::base_name() and pushing newly allocated
Name and hashed-name entries into the candidate Vec.

The bug is reachable by any caller of DnssecDnsHandle — including the
resolver, recursor, and client — when built with the dnssec-ring or
dnssec-aws-lc-rs feature and configured to perform DNSSEC validation. It is
triggered while validating a NoData or NXDomain response whose authority
section contains an SOA record from a zone other than an ancestor of the
QNAME, on a code path that requires NSEC3 closest-encloser proof. In practice
this can be reached through an insecure CNAME chain that crosses zone
boundaries into a DNSSEC-signed zone returning NoData, but the minimum
condition is just a mismatched SOA owner on a response requiring NSEC3
validation.

A debug_assert_ne!(name, Name::root()) guards the loop body, so debug builds
abort with a panic on the first iteration past the root. Release builds
compile the assertion out and run the loop unbounded, allocating until the
process exhausts available memory (OOM). A reachable upstream attacker who
can return such a response can therefore crash a debug-built validator or
exhaust memory on a release-built one.

The affected code was migrated from hickory-proto to hickory-net as part of
the 0.26.0 release. The hickory-proto 0.26.x release no longer offers
DnssecDnsHandle and so we recommend all affected users update to hickory-net
0.26.1 when the implementation of that type is required.

See advisory page for additional details.

The report names hickory-proto's DnssecDnsHandle and the RUSTSEC-2026-0118 advisory, but no rust-libp2p file or test to change. First establish whether this repository uses the affected package and identify the appropriate dependency or implementation entry point; done requires a confirmed fix or upgrade that prevents the unbounded validation loop.