[Question] CPU and Memory Utilization too High for Manager/startup module [Distributed Architecture startup , collector , warehouse]
- Dominant language
- Java
- Stars
- 7.4k
- Forks
- 1.3k
- Avg merge
- 1d 20h
- Merged PRs (30d)
- 32
Description
### Question
1. Currently Hertzbeat uses apache arrow and hertzbeat has both in-memory heap and external memory usage .
It uses apache arrow to write in external memory .
**Our scenario :** we have distributed actictecture where we spin multi-collectors and warehouses . data get converted to arrow object in collector after collection and is passed to manager via netty here in manager data gets deserialized and stored into memory and suspect manager memory is increased due to this and later the data send to warehouse for persistence via kafka topic .
if we add more monitors the memory grows drastically -> need to solve this
here manager memory is increased and not decreased later .
tried to clear the objects from memory before sending to kafka topic but it didnot work .
here is sample code from manager where it pushes the data to kafka .
attached the ss for Heap and Code as well
Contributor guide
Research direction
Start by reproducing the distributed setup with multiple collectors and warehouses, then inspect the manager's Netty deserialization and Kafka publishing path using the provided heap and code captures. Compare manager memory before and after Arrow data is forwarded, and identify whether retained objects or external memory remain after publishing; done means memory no longer grows drastically as monitors are added.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, kafka
- Domain
- backend, distributed-systems, performance
- Issue type
- Bug
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Active
- Clarity
- Needs clarification
- Newbie friendliness
- 35/100