Add configuration management for exporter selection in watcher
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
## Story
As a system administrator, I want to configure which metrics exporters are enabled through configuration files, so that I can easily manage and customize metrics collection per environment.
## Description
Implement a configuration system that allows users to specify which exporters should be activated when the watcher starts, including their specific settings and parameters.
## Acceptance Criteria
- [ ] Configuration schema supports multiple exporter definitions
- [ ] Each exporter can be enabled/disabled via configuration
- [ ] Support for exporter-specific parameters (port, interval, etc.)
- [ ] Configuration validation on startup
- [ ] Support for runtime configuration updates (hot-reload)
- [ ] Default configurations for common exporters
- [ ] Environment variable overrides for configuration values
## Technical Details
### Configuration Schema
```yaml
watcher:
exporters:
dcgm:
enabled: true
port: 9400
health_check_interval: 30s
gpu_devices: "all" # or specific GPU IDs
metrics_groups:
- "GPU_UTIL"
- "MEMORY"
- "POWER"
node:
enabled: true
port: 9100
health_check_interval: 30s
collectors:
- "cpu"
- "memory"
- "disk"
- "network"
custom:
- name: "app_metrics"
enabled: false
module: "ai.backend.watcher.exporters.custom.app"
config:
key: "value"
```
### Configuration Loading
- Use Pydantic models for configuration validation
- Support YAML and environment variable inputs
- Implement configuration merging (defaults -> file -> env)
- Validate exporter availability before enabling
### Hot-reload Support
- File watcher for configuration changes
- Graceful reload without dropping metrics
- Validation before applying new configuration
- Rollback on invalid configuration
## Dependencies
- Pydantic for configuration models
- YAML parser
- File system watcher for hot-reload
## Testing Requirements
- Unit tests for configuration parsing
- Validation tests for invalid configurations
- Integration tests for configuration updates
- Test environment variable overrides
- Test hot-reload functionality
## Documentation
- Configuration reference guide
- Example configurations for different scenarios
- Environment variable documentation
- Migration guide from static to dynamic configuration
JIRA Issue: BA-2501
Contributor guide
Assessment
This issue has not been assessed yet.