lablup / lablup/backend.ai

Implement exporter registration system in watcher

Open
#6,023 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Story

As a system administrator, I want to dynamically register and manage metrics exporters through the watcher system, so that I can collect node-level and GPU-specific metrics without manual configuration.

## Description

Implement a flexible registration system that allows the watcher to register various metrics exporters (DCGM exporter, node exporter, etc.) using a fixture/plugin-based approach.

## Acceptance Criteria

- [ ] Watcher can register multiple types of exporters (DCGM, node exporter, etc.)
- [ ] Support fixture-based injection for exporter configuration
- [ ] Implement registry pattern for managing multiple exporters
- [ ] Each exporter can be uniquely identified and tracked
- [ ] Support for adding custom exporters through plugin system
- [ ] Proper error handling for registration failures

## Technical Details

### Exporter Interface

```python
class MetricsExporter(Protocol):
def register(self) -> None: ...
def start(self) -> None: ...
def stop(self) -> None: ...
def get_metrics_endpoint(self) -> str: ...
def health_check(self) -> bool: ...
```

### Registry Implementation

- Create `ExporterRegistry` class to manage all registered exporters
- Support concurrent registration and deregistration
- Thread-safe operations for multi-exporter scenarios

### Supported Exporters (Initial)

1. **DCGM Exporter** - NVIDIA GPU metrics
1. **Node Exporter** - System-level metrics
1. **Custom Exporter** - Base class for user-defined exporters

## Dependencies

- Watcher system must be running
- Prometheus client libraries
- Target exporter binaries/containers

## Testing Requirements

- Unit tests for registry operations
- Integration tests with mock exporters
- Test fixture injection mechanism
- Test concurrent registration scenarios

## Documentation

- API documentation for exporter interface
- Example implementation for custom exporter
- Configuration guide for common exporters

JIRA Issue: BA-2500

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.