elastic / elastic/detection-rules

`RuleLoader` cleanup and optimizations

Open
#2,609 5 comments 0 reactions 0 assignees View on GitHub
backlog python
Dominant language
Python
Stars
2.7k
Forks
696
Avg merge
4d 17h
Merged PRs (30d)
87

Description

## Overview

The purpose of this is to identify opportunities to clean up the code that makes up the rule loader (`Rule`, `rule_validator`s, etc.). Loading the rules has gotten significantly slower and while some of it is due to the necessity of expanding validation, this should explore opportunities for optimization.

rule loader profiling

![image](https://user-images.githubusercontent.com/16747370/222036895-07c5ff00-a5d5-4aa8-9c1f-bce160dbffc7.png)

## Observations

- [ ] `data` in https://github.com/elastic/detection-rules/blob/66359012c32980596473e4ace3ae73dbad89b52d/detection_rules/rule_validators.py#L251 is unused
- [ ] cache the `Version.parse` instead of parsing for every rule https://github.com/elastic/detection-rules/blob/66359012c32980596473e4ace3ae73dbad89b52d/detection_rules/rule.py#L356
- [ ] make these global constants https://github.com/elastic/detection-rules/blob/66359012c32980596473e4ace3ae73dbad89b52d/detection_rules/rule.py#L491-L492
- [ ] turn these into `cached_property` like `QueryRuleData` https://github.com/elastic/detection-rules/blob/66359012c32980596473e4ace3ae73dbad89b52d/detection_rules/rule.py#L489-L490
- [ ]

Contributor guide

Open the contributing guide

Research direction

Start by reading the referenced sections of detection_rules/rule_validators.py and detection_rules/rule.py, then review the rule-loader profiling data. Compare the listed cleanup and caching opportunities against current rule-loading behavior; done means the selected optimizations are implemented and their effect on loading performance is demonstrated without changing validation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
performance, security
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.