Add built-in caching to translate_sid processor
- Dominant language
- Go
- Stars
- 12.7k
- Forks
- 5k
- Avg merge
- 2d 15m
- Merged PRs (30d)
- 385
Description
**Describe the enhancement:**
Add built-in caching to the `translate_sid` processor to reduce Active Directory queries and improve reliability when translating Windows Security Identifiers (SIDs) to account names.
The processor currently performs uncached, synchronous network queries to Active Directory domain controllers for non-local SID resolution ([libbeat/processors/translate_sid/translatesid.go:107](https://github.com/elastic/beats/blob/4350d4ee26db04c87c2bfdf119ba3c278c559f94/libbeat/processors/translate_sid/translatesid.go#L107-L111)). Each event triggers a fresh AD query, making the processor susceptible to network timeouts that result in `ERROR_NONE_MAPPED` errors, and creating unnecessary load on AD infrastructure.
The proposed enhancement would add optional in-memory caching with configurable capacity and TTL:
```yaml
- translate_sid:
field: winlog.event_data.MemberSid
account_name_target: member_user.name
domain_target: member_user.domain
cache:
capacity: 10000 # Maximum number of cached entries
ttl: 24h # Time-to-live for cached entries
```
**Describe a specific use case for the enhancement or feature:**
A user monitoring Windows Security Events processes thousands of events per hour, each containing SIDs that require translation. Without caching, the `translate_sid` processor performs AD lookups for the same SIDs repeatedly, causing:
- Network timeouts resulting in `ERROR_NONE_MAPPED` errors for SIDs that can be successfully translated in PowerShell
- High latency in event processing pipeline
- Increased load on domain controllers
The user implemented a workaround using the `cache` processor which successfully resolved the issue by caching successful SID translations. This validates that built-in caching would improve the processor's reliability and performance.
With native caching, the processor would:
- Perform AD lookup once per unique SID within the TTL window
- Reduce network traffic to domain controllers
- Eliminate timeout-related translation failures for frequently-seen SIDs
- Improve overall event processing throughput
- Simplify configuration by eliminating the need for manual `cache` processor workarounds
Contributor guide
Research direction
Start in libbeat/processors/translate_sid/translatesid.go around lines 107-111, where non-local SID lookups are performed. Review how the existing cache processor workaround is configured, then define the cache capacity and TTL behavior for repeated SID translations. Done means translate_sid supports the proposed optional cache settings while preserving uncached behavior when they are absent.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- backend, security
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100