microsoft / microsoft/retina

Feature Request: Pluggable Collector Architecture for Custom Data Ingestion

Open
#1,771 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
3.2k
Forks
304
Avg merge
1d 19h
Merged PRs (30d)
78

Description

## Motivation

In enterprise and multi-cloud environments, there is often a need to enrich flow data or forward telemetry to custom systems such as internal threat intelligence platforms, legacy SIEMs, or proprietary analytics engines. Currently, extending Retina’s collection or enrichment logic requires modifying the core codebase, which introduces maintenance challenges and limits safe integration paths.

## Proposal

Introduce a pluggable collector interface that allows external developers to define, load, and run custom collectors or enrichers without modifying Retina's core.

- Example interface in Go:

```Go
type CollectorPlugin interface {
Init(config map[string]string) error
Start(ctx context.Context, emitter EventEmitter) error
Stop() error
Metadata() PluginMetadata
}
```
- Collectors could be configured dynamically:

```yaml
customCollectors:
- name: "custom-enricher"
type: "native"
path: "/opt/retina/plugins/custom_enricher.so"
```

Optional support for Wasm-based plugins could provide security and isolation for untrusted modules.

## Benefits

- Enables domain-specific enrichment and data forwarding pipelines

- Reduces friction in adopting Retina across varied environments

- Decouples internal logic from enterprise-specific extensions

- Encourages community contributions of specialized plugins

## Considerations

- Plugin lifecycle and failure isolation

- Versioning and compatibility guarantees

- Optional sandboxing for untrusted code (e.g., Wasm or subprocess isolation)

If this direction aligns with Retina’s roadmap, I would be happy to contribute to an initial draft or prototype implementation.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.