apache / apache/ozhera

Feature: Add Node Collection Information Monitoring for Log Agent

Open
#588 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
122
Forks
46
Avg merge
11h 56m
Merged PRs (30d)
9

Description

**Labels:** `enhancement`, `logging`, `monitoring`

## 📋 Feature Description

This PR introduces a new node collection information monitoring feature for the OzHera log agent system. The feature allows real-time monitoring and collection of log processing status across different nodes in the distributed logging infrastructure.

## 🎯 What's New

### New Components Added:
1. **NodeCollInfoProcessor** - RPC processor for handling node collection information requests
2. **NodeCollInfo** model - Data structure for node collection information
3. **MachineCollInfo** model - Machine-specific collection information (replaced by NodeCollInfo)

### Key Features:
- **Real-time Node Monitoring**: Monitor log collection progress across different nodes
- **Tail Collection Status**: Track individual log tail collection information including:
- Tail ID and name
- File collection progress
- Current and maximum file pointers
- Collection timestamps
- **Host Information**: Capture host IP and hostname for each node
- **RPC Communication**: Efficient communication between log agent server and agents

## 🔧 Technical Implementation

### Core Classes:
- `NodeCollInfoProcessor`: Handles RPC requests for node collection info
- `NodeCollInfo`: Main data model containing host info and tail collection details
- `ChannelEngine.getNodeCollInfo()`: Collects and aggregates node information
- `DefaultLogProcessCollector.getNodeCollInfo()`: Server-side collection method

### Data Structure:
```java
NodeCollInfo {
String hostIp;
String hostName;
List tailCollInfos;

TailCollInfo {
Long tailId;
String tailName;
List collInfos;
}

CollInfo {
String fileName;
String fileNode;
String collProgress;
Long maxPointer;
Long currentPointer;
Long currentNumber;
Long collTime;
}
}

Contributor guide

Open the contributing guide

Research direction

Start with the named NodeCollInfoProcessor, NodeCollInfo and MachineCollInfo components, then trace ChannelEngine.getNodeCollInfo() and DefaultLogProcessCollector.getNodeCollInfo(). Confirm how node and tail collection data move through the RPC path; done means the monitoring information is collected and exposed across the log agent system.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
backend, distributed-systems, observability-sre
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.