Kuadrant / Kuadrant/dns-operator
Large memory usage on Primary clusters
- Dominant language
- Go
- Stars
- 12
- Forks
- 23
- Avg merge
- 1d 4h
- Merged PRs (30d)
- 14
Description
**Describe the bug**
With the multicluster-runtime, the events for all the dnsrecords are watch by a primary cluster in a multi cluster set up.
These are cached, causing the memory usage to increase.
Records from the secondary that are not delegated will also be cached.
This cause a high usage requirement on the primaries.
**To Reproduce**
Steps to reproduce the behavior:
1. Deploy a multi cluster set up. `make multicuster-local-setup`
2. Set up coredns secret as the default secret on the primary cluster.
3. Add dnsrecords to the secondary with `delegation=true`.
4. Monitor the memory usage of both primary, and secondary controllers.
5. Expect to see the primary use about double the memory.
**Expected behavior**
Reduce the memory over head on the primary.
**Scenarios**
Some scenarios that should to be investigated
- Multi secondary clusters with a spread of dnsrecords, compared to one secondary with the same number of dnsrecords.
- Multi primary clusters with dnsrcords on the primaries
- What affect does adding secondaries have to the primary if there is no dnsrecords. Is there an affect on the base level resources required per connection.
- What affect does a mixture of delegated, and non delegated have.
- Can a running primary with a high memory usage be restarted. What would the memory usage do during upgrade.
- With a large number of records on secondaries. How long does a reconcile take compared to non delegated records.
**Expected Outcomes**
- Issue to document the findings
- Possible follow up issue with improvement.
Contributor guide
Research direction
Start by reproducing the multi-cluster setup with `make multicuster-local-setup`, then monitor primary and secondary controller memory while varying delegated records, secondary clusters, and record counts. Document the observed memory and reconcile-time findings, including restart behavior, and use them to identify a possible follow-up improvement issue.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- infrastructure, networking
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100