Feature Request: Pluggable Collector Architecture for Custom Data Ingestion
- Dominant language
- Go
- Stars
- 3.2k
- Forks
- 304
- Avg merge
- 1d 19h
- Merged PRs (30d)
- 78
Description
## Motivation
In enterprise and multi-cloud environments, there is often a need to enrich flow data or forward telemetry to custom systems such as internal threat intelligence platforms, legacy SIEMs, or proprietary analytics engines. Currently, extending Retina’s collection or enrichment logic requires modifying the core codebase, which introduces maintenance challenges and limits safe integration paths.
## Proposal
Introduce a pluggable collector interface that allows external developers to define, load, and run custom collectors or enrichers without modifying Retina's core.
- Example interface in Go:
```Go
type CollectorPlugin interface {
Init(config map[string]string) error
Start(ctx context.Context, emitter EventEmitter) error
Stop() error
Metadata() PluginMetadata
}
```
- Collectors could be configured dynamically:
```yaml
customCollectors:
- name: "custom-enricher"
type: "native"
path: "/opt/retina/plugins/custom_enricher.so"
```
Optional support for Wasm-based plugins could provide security and isolation for untrusted modules.
## Benefits
- Enables domain-specific enrichment and data forwarding pipelines
- Reduces friction in adopting Retina across varied environments
- Decouples internal logic from enterprise-specific extensions
- Encourages community contributions of specialized plugins
## Considerations
- Plugin lifecycle and failure isolation
- Versioning and compatibility guarantees
- Optional sandboxing for untrusted code (e.g., Wasm or subprocess isolation)
If this direction aligns with Retina’s roadmap, I would be happy to contribute to an initial draft or prototype implementation.
Contributor guide
Assessment
This issue has not been assessed yet.