Feature: Add Node Collection Information Monitoring for Log Agent
- Dominant language
- Java
- Stars
- 122
- Forks
- 46
- Avg merge
- 11h 56m
- Merged PRs (30d)
- 9
Description
**Labels:** `enhancement`, `logging`, `monitoring`
## 📋 Feature Description
This PR introduces a new node collection information monitoring feature for the OzHera log agent system. The feature allows real-time monitoring and collection of log processing status across different nodes in the distributed logging infrastructure.
## 🎯 What's New
### New Components Added:
1. **NodeCollInfoProcessor** - RPC processor for handling node collection information requests
2. **NodeCollInfo** model - Data structure for node collection information
3. **MachineCollInfo** model - Machine-specific collection information (replaced by NodeCollInfo)
### Key Features:
- **Real-time Node Monitoring**: Monitor log collection progress across different nodes
- **Tail Collection Status**: Track individual log tail collection information including:
- Tail ID and name
- File collection progress
- Current and maximum file pointers
- Collection timestamps
- **Host Information**: Capture host IP and hostname for each node
- **RPC Communication**: Efficient communication between log agent server and agents
## 🔧 Technical Implementation
### Core Classes:
- `NodeCollInfoProcessor`: Handles RPC requests for node collection info
- `NodeCollInfo`: Main data model containing host info and tail collection details
- `ChannelEngine.getNodeCollInfo()`: Collects and aggregates node information
- `DefaultLogProcessCollector.getNodeCollInfo()`: Server-side collection method
### Data Structure:
```java
NodeCollInfo {
String hostIp;
String hostName;
List tailCollInfos;
TailCollInfo {
Long tailId;
String tailName;
List collInfos;
}
CollInfo {
String fileName;
String fileNode;
String collProgress;
Long maxPointer;
Long currentPointer;
Long currentNumber;
Long collTime;
}
}
Contributor guide
Research direction
Start with the named NodeCollInfoProcessor, NodeCollInfo and MachineCollInfo components, then trace ChannelEngine.getNodeCollInfo() and DefaultLogProcessCollector.getNodeCollInfo(). Confirm how node and tail collection data move through the RPC path; done means the monitoring information is collected and exposed across the log agent system.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, distributed-systems, observability-sre
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100