DnsNameResolver clears all DnsCache entries, even if external DnsCache instance is provided to DnsNameResolverBuilder

Expected behavior

If a DNS resolution call is made for a hostname and is cached, clients attempting to resolve the same hostname will fetch the cached value, if the resolution attempt is made during the TTL duration.

Actual behavior

If you create multiple DnsAddressResolverGroup instances, the cache is stored per DnsAddressResolverGroup instance instead of globally. To work this around, in the DnsNameResolverBuilder, an external DnsCache instance was provided, so it can be shared between DnsAddressResolverGroup and respectively clients, via the DnsNameResolverBuilder's resolveCache method.

Nonetheless when the DnsAddressResolverGroup is closed, clients, which have their own DnsAddressResolverGroup instances (which still use the same externally provided DnsCache instance) that are calling the same hostname were resolving the same hostnames, for which the DNS server returned a large TTL (e.g. 3600) instead of relying on the cached entry.

The issue was identified to be in DnsNameResolver's close method - it calls resolveCache.clear(), which clears the externally provided DnsCache instance.

Steps to reproduce:

1. Create a new client, where you explicitly provide a DnsCache instance, when creating a new DnsAddressResolverGroup instance, and connect it
2. Once the connection succeeds, shutdown the client and close its groups, including the DnsAddressResolverGroup
3. Create a new client, where you explicitly provide the same DnsCache instance, when creating a new DnsAddressResolverGroup instance, and connect it
   3.1 Ensure that the second client attempts the connection during the TTL period
4. Observe that both clients will perform a DNS resolution call before connecting

PS: You can set io.netty.resolver.dns loggers to DEBUG to see the two DNS calls

Minimal yet complete reproducer code (or URL to code):

import java.io.IOException;
import java.net.InetAddress;
import java.net.InetSocketAddress;
import java.util.Collections;
import java.util.HashMap;
import java.util.Map;
import java.util.Set;
import java.util.concurrent.TimeUnit;

import org.apache.directory.server.dns.DnsException;
import org.apache.directory.server.dns.DnsServer;
import org.apache.directory.server.dns.messages.QuestionRecord;
import org.apache.directory.server.dns.messages.RecordClass;
import org.apache.directory.server.dns.messages.RecordType;
import org.apache.directory.server.dns.messages.ResourceRecord;
import org.apache.directory.server.dns.messages.ResourceRecordImpl;
import org.apache.directory.server.dns.protocol.DnsProtocolHandler;
import org.apache.directory.server.dns.store.DnsAttribute;
import org.apache.directory.server.dns.store.RecordStore;
import org.apache.directory.server.protocol.shared.transport.UdpTransport;
import org.apache.mina.transport.socket.DatagramAcceptor;
import org.apache.mina.transport.socket.DatagramSessionConfig;

import io.netty.bootstrap.Bootstrap;
import io.netty.bootstrap.ServerBootstrap;
import io.netty.channel.Channel;
import io.netty.channel.ChannelFuture;
import io.netty.channel.ChannelHandlerContext;
import io.netty.channel.ChannelInboundHandlerAdapter;
import io.netty.channel.ChannelInitializer;
import io.netty.channel.ChannelPipeline;
import io.netty.channel.EventLoopGroup;
import io.netty.channel.nio.NioEventLoopGroup;
import io.netty.channel.socket.nio.NioDatagramChannel;
import io.netty.channel.socket.nio.NioServerSocketChannel;
import io.netty.channel.socket.nio.NioSocketChannel;
import io.netty.resolver.ResolvedAddressTypes;
import io.netty.resolver.dns.DefaultDnsCache;
import io.netty.resolver.dns.DnsAddressResolverGroup;
import io.netty.resolver.dns.DnsCache;
import io.netty.resolver.dns.DnsNameResolverBuilder;
import io.netty.resolver.dns.DnsServerAddressStream;
import io.netty.resolver.dns.DnsServerAddressStreamProvider;
import io.netty.resolver.dns.DnsServerAddresses;
import io.netty.util.concurrent.DefaultThreadFactory;

public class DnsProblemReproduction {

	private static final String SERVER_HOSTNAME = "dnsproblem.test";
	private static final int SERVER_PORT = 30000;

	private static final int DNS_SERVER_PORT = 54;

	private static final long TIMEOUT = 5L;

	public static void main(String[] args) throws Exception {
		EventLoopGroup serverBossGroup = createEventLoopGroup("server-boss");
		EventLoopGroup serverWorkerGroup = createEventLoopGroup("server-worker");
		DnsCache dnsCache = new DefaultDnsCache();

		EmbeddedDnsServer dnsServer = null;
		Channel serverChannel = null;
		EventLoopGroup clientWorkerGroup = null;
		try {
			dnsServer = createAndStartDnsServer();

			serverChannel = createAndBindServer(serverBossGroup, serverWorkerGroup);

			clientWorkerGroup = createClientWorkerGroup();
			DnsAddressResolverGroup addressResolverGroup = createAddressResolverGroup(dnsCache);
			ChannelFuture connectFuture = createAndConnectClient(clientWorkerGroup, addressResolverGroup);
			connectFuture.await(TIMEOUT, TimeUnit.SECONDS);

			connectFuture.channel().close().await(TIMEOUT, TimeUnit.SECONDS);
			shutdownEventLoopGroup(clientWorkerGroup);
			addressResolverGroup.close();

			clientWorkerGroup = createClientWorkerGroup();
			addressResolverGroup = createAddressResolverGroup(dnsCache);
			connectFuture = createAndConnectClient(clientWorkerGroup, addressResolverGroup);
			connectFuture.await(TIMEOUT, TimeUnit.SECONDS);

			connectFuture.channel().close().await(TIMEOUT, TimeUnit.SECONDS);
		} finally {
			if (dnsServer != null) {
				dnsServer.stop();
			}

			if (serverChannel != null) {
				serverChannel.close().syncUninterruptibly();
			}

			shutdownEventLoopGroup(serverBossGroup);
			shutdownEventLoopGroup(serverWorkerGroup);
			shutdownEventLoopGroup(clientWorkerGroup);
		}
	}

	private static DnsAddressResolverGroup createAddressResolverGroup(DnsCache dnsCache) {
		DnsNameResolverBuilder dnsNameResolverBuilder = new DnsNameResolverBuilder().datagramChannelType(NioDatagramChannel.class)
				.resolveCache(dnsCache).nameServerProvider(new EmbeddedDnsServerAddressStreamProvider()).optResourceEnabled(false)
				.resolvedAddressTypes(ResolvedAddressTypes.IPV4_ONLY);

		return new DnsAddressResolverGroup(dnsNameResolverBuilder);
	}

	private static EmbeddedDnsServer createAndStartDnsServer() throws Exception {
		EmbeddedDnsServer dnsServer = new EmbeddedDnsServer();
		dnsServer.start();

		return dnsServer;
	}

	private static Channel createAndBindServer(EventLoopGroup serverBossGroup, EventLoopGroup serverWorkerGroup) throws Exception {
		ServerBootstrap serverBootstrap = new ServerBootstrap().group(serverBossGroup, serverWorkerGroup).channel(NioServerSocketChannel.class)
				.childHandler(new ChannelInitializer<Channel>() {

					@Override
					protected void initChannel(Channel ch) throws Exception {
						ChannelPipeline pipeline = ch.pipeline();
						pipeline.addLast(new ChannelInboundHandlerAdapter() {
							@Override
							public void channelActive(ChannelHandlerContext ctx) throws Exception {
								super.channelActive(ctx);

								System.out.println("Client connected");
							}
						});
					}
				});

		Channel serverChannel = serverBootstrap.bind(SERVER_PORT).syncUninterruptibly().channel();

		System.out.println("Local server started");

		return serverChannel;
	}

	private static ChannelFuture createAndConnectClient(EventLoopGroup clientWorkerGroup, DnsAddressResolverGroup addressResolverGroup) {
		Bootstrap bootstrap = new Bootstrap().group(clientWorkerGroup).channel(NioSocketChannel.class).resolver(addressResolverGroup)
				.handler(new ChannelInitializer<Channel>() {

					@Override
					protected void initChannel(Channel ch) throws Exception {
						ChannelPipeline pipeline = ch.pipeline();
						pipeline.addLast(new ChannelInboundHandlerAdapter());
					}
				});

		return bootstrap.connect(InetSocketAddress.createUnresolved(SERVER_HOSTNAME, SERVER_PORT));
	}

	private static EventLoopGroup createClientWorkerGroup() {
		return createEventLoopGroup("client-worker");
	}

	private static EventLoopGroup createEventLoopGroup(String poolName) {
		return new NioEventLoopGroup(1, new DefaultThreadFactory(poolName));
	}

	private static void shutdownEventLoopGroup(EventLoopGroup eventLoopGroup) {
		if (eventLoopGroup != null) {
			eventLoopGroup.shutdownGracefully(0L, 1L, TimeUnit.SECONDS);
		}
	}

	private static final class EmbeddedDnsServerAddressStreamProvider implements DnsServerAddressStreamProvider {

		private final DnsServerAddresses dnsServerAddresses;

		private EmbeddedDnsServerAddressStreamProvider() {
			InetSocketAddress dnsServerAddress = new InetSocketAddress("localhost", DNS_SERVER_PORT);
			this.dnsServerAddresses = DnsServerAddresses.sequential(Collections.singleton(dnsServerAddress));
		}

		@Override
		public DnsServerAddressStream nameServerAddressStream(String hostname) {
			return dnsServerAddresses.stream();
		}
	}

	private static final class EmbeddedDnsServer extends DnsServer {

		private final RecordStore store = new RecordStore() {

			@Override
			public Set<ResourceRecord> getRecords(QuestionRecord questionRecord) throws DnsException {
				if (RecordType.A.equals(questionRecord.getRecordType())) {
					try {
						String domainName = questionRecord.getDomainName();
						if (SERVER_HOSTNAME.equalsIgnoreCase(domainName)) {
							return Collections.singleton(createRecord(domainName, InetAddress.getLocalHost()));
						}
					} catch (Exception e) {
						e.printStackTrace();
					}
				}

				return Collections.emptySet();
			}

			private ResourceRecord createRecord(String domainName, InetAddress address) {
				Map<String, Object> attributes = new HashMap<>();
				attributes.put(DnsAttribute.IP_ADDRESS, address.getHostAddress());

				return new ResourceRecordImpl(domainName, RecordType.A, RecordClass.IN, 3600, attributes);
			}
		};

		@Override
		public void start() throws IOException {
			UdpTransport transport = new UdpTransport(DNS_SERVER_PORT);
			setTransports(transport);

			DatagramAcceptor acceptor = transport.getAcceptor();
			acceptor.setHandler(new DnsProtocolHandler(this, store));
			((DatagramSessionConfig) acceptor.getSessionConfig()).setReuseAddress(true);

			acceptor.bind();

			System.out.println("DNS server started");
		}
	}
}

Netty version

Originally reproduced with 4.1.128.Final, but also reproducible with latest version at the moment 4.1.135.Final

JVM version (e.g. java -version)

1.8.0_491

OS version (e.g. uname -a)

Independent of OS (reproduced both on Windows and MacOS), nonetheless Darwin 25.5.0

Start at DnsNameResolver.close and trace how resolveCache is owned when DnsNameResolverBuilder receives an external DnsCache. Run the supplied DnsProblemReproduction with io.netty.resolver.dns logging at DEBUG; done when closing one DnsAddressResolverGroup leaves the shared cache available to another group during the TTL.