opensearch-project / opensearch-project/security-analytics
Refactor Threat Intel Feeds Job Scheduler
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 111
- Forks
- 111
- Avg merge
- 1d 17h
- Merged PRs (30d)
- 13
Description
Refactor threat intel feeds job scheduler to decouple it with creating and updating IOCs.
Part 1.
Refactor the transport layer into a service class that DetectorThreatIntel directly calls.
Part 2.
- Renamed
TIFJobUpdateServicetoThreatIntelFeedIndexService - Renamed
TIFJobParametertoTIFJobSchedulerMetadata - Refactored
ThreatIntelFeedDataServiceandThreatIntelFeedIndexServiceso that threat intel indices are created and deleted in theThreatIntelFeedIndexServiceclass but the data is parsed and saved into the index in theThreatIntelFeedDataServiceclass to provide better modularization.
More specifics to the refactoring
Methods moved from ThreatIntelFeedDataService to ThreatIntelFeedIndexService
getIndexMappingcreateIndexIfNotExistsdeleteThreatIntelDataIndexcreateIndexIfNotExists
Methods moved from ThreatIntelFeedIndexService to ThreatIntelFeedDataService
updateJobSchedulerMetadataAsSucceededwaitUntilAllShardsStarted
Refactored
createThreatIntelFeedDatatocreateThreatIntelFeedinThreatIntelFeedIndexService, refactored sogroupedActionListeneris done in data service where it uses the csvParser to parse and save feeds into the index. This method is the hook forThreatIntelHighLevelHandlerandTIFJobRunnerto set up and create a TIF index as well as populating it with data inThreatIntelFeedDataService
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Locate DetectorThreatIntel, ThreatIntelFeedDataService, ThreatIntelFeedIndexService, TIFJobSchedulerMetadata, ThreatIntelHighLevelHandler, and TIFJobRunner. Read the current scheduler and transport interactions first, then trace the listed methods and their callers. Done means the renamed services and metadata are wired correctly, with index lifecycle separated from feed parsing and persistence.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100