[EKS] [CoreDNS]: Recovered from panic in server: "dns://:53" runtime error: invalid memory address or nil pointer dereference
- Dominant language
- Shell
- Stars
- 5.4k
- Forks
- 334
- PR merge metrics
- No merged PRs in 30d
Description
### Community Note
* Please vote on this issue by adding a 👍 [reaction](https://blog.github.com/2016-03-10-add-reactions-to-pull-requests-issues-and-comments/) to the original issue to help the community and maintainers prioritize this request
* Please do not leave "+1" or "me too" comments, they generate extra noise for issue followers and do not help prioritize the request
* If you are interested in working on this issue or have submitted a pull request, please leave a comment
**Tell us about your request**
CoreDNS 1.9.3 for EKS 1.24 appears to be experiencing potential memory leaks after recovering from a panic.
**Which service(s) is this request for?**
EKS
**Tell us about the problem you're trying to solve. What are you trying to do, and why is it hard?**
When CoreDNS recovers from a panic, it stops responding to requests, and memory starts to steadily increase. I observed a memory increase, and just before this occurred, I received this error:
`[ERROR] Recovered from panic in server: "dns://:53" runtime error: invalid memory address or nil pointer dereference. `
**Are you currently working around this issue?**
Yes, it's a production environment, and I am actively seeking a solution to this error.
**Additional context**
I found some merged PRs in newer CoreDNS versions.
https://github.com/coredns/coredns/pull/6120 (I'm not using DoH, but could be a relation.)
https://github.com/coredns/coredns/pull/6179
Just one plugin has stopped responding to queries.
**Attachments**
Memory increase graph

Panic and coredns_dns_requests_total (Pink represents panic, while other colors represent coredns_dns_requests_total for each type AAAA, CNAME, and A)

Contributor guide
Research direction
Start by comparing the reported CoreDNS 1.9.3 behavior on EKS 1.24 with the newer versions and merged PRs 6120 and 6179 mentioned in the issue. Use the panic message, memory graph, and coredns_dns_requests_total observations to investigate; done means a recovered panic does not leave the plugin unresponsive or cause steadily increasing memory.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, kubernetes
- Domain
- cloud, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100