airbnb / airbnb/streamalert

Feature: Add Sysmon schema

Open
#252 2 comments 0 reactions 0 assignees View on GitHub
feature request help wanted log schemas logs
Dominant language
Python
Stars
2.9k
Forks
324
PR merge metrics
No merged PRs in 30d

Description

**Background**

From: https://docs.microsoft.com/en-us/sysinternals/downloads/sysmon

_"System Monitor (Sysmon) is a Windows system service and device driver that, once installed on a system, remains resident across system reboots to monitor and log system activity to the Windows event log. It provides detailed information about process creations, network connections, and changes to file creation time. By collecting the events it generates using Windows Event Collection or SIEM agents and subsequently analyzing them, you can identify malicious or anomalous activity and understand how intruders and malware operate on your network."_

**Data format**

_"On Vista and higher, events are stored in "Applications and Services Logs/Microsoft/Windows/Sysmon/Operational", and on older systems events are written to the System event log. Event timestamps are in UTC standard time."_

Regardless, the file is an EVT or EVTX file (Microsoft proprietary). These can be converted to XML which can in turn be converted to CSV or JSON.

This is largely handled by log forwarding agents / endpoints. Examples:

- https://www.elastic.co/guide/en/logstash/current/plugins-inputs-eventlog.html
- https://docs.fluentd.org/v0.12/articles/windows
- https://osquery.io/docs/tables/#windows_events

We simply need to

1. Determine how the schema changes across fluent/logstash/osquery
2. Develop schemas for them in `streamalert/conf/logs.json`
3. Profit

Contributor guide

Open the contributing guide

Research direction

Start by comparing the Sysmon formats described in the linked Microsoft, Logstash, Fluentd, and osquery references. Then inspect streamalert/conf/logs.json and determine how the schemas differ across those sources. Done means the relevant Sysmon schemas are defined there for the supported forwarding formats.

Written by the indexing model from the issue text.

Assessment

Domain
security
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.