Internal DNS servers return records with a TTL of `0`

Our internal DNS servers currently don't set an explicit TTL, so we get the hickory-proto default of 0. This instructs clients to never cache results, which seems to be in conflict with connection pooling as we've started to integrate qorb. When we construct a qorb DnsResolver, we currently instruct it to ignore those 0-valued TTLs: https://github.com/oxidecomputer/omicron/blob/f3ab3387b640de27875502209a55211087e9b23d/nexus/db-queries/src/db/pool.rs#L63-L66

From this conversation on the Oximeter qorb integration PR:

smklein Oct 2, 2024

Yup! Right now, I think our internal DNS system is using a TTL of "zero", which means we'd see a backend, start to spin up connections for them, and then discard them immediately. Next time we query DNS, we'd start to create connections, then discard them immediately...

This behavior is tested here: https://github.com/oxidecomputer/qorb/blob/7c886a24369f458bdab4b9530e01f498da79e61f/src/resolvers/dns.rs#L712-L746 , but it sucks.

By ignoring TTLs, we still discard backends that are explicitly removed from DNS, but we only remove backends when DNS servers stop telling us about them successfully.

This might all be fine? We could decide qorb ignoring the TTL and doings its own periodic querying is exactly what we want, but I wanted to write this down in case we want to control TTLs from the DNS side (at which point we'd presumably want to stop overriding it in the qorb config(s)).

Start by reading nexus/db-queries/src/db/pool.rs around the DnsResolver configuration and the qorb DNS behavior tested in src/resolvers/dns.rs. Determine whether internal DNS should provide explicit TTLs or qorb should continue ignoring zero-valued TTLs. Done means the chosen behavior is implemented consistently and covered by the relevant tests.