Azure / Azure/Azure-Sentinel

GitHubAuditData parser dropping logs

Open
#14,659 3 comments 0 reactions 3 assignees Claimed by @hassanchawiche View on GitHub
Parser
Dominant language
Python
Stars
6.1k
Forks
3.8k
Avg merge
4d 7h
Merged PRs (30d)
125

Description

**Describe the bug**
I’ve identified the problem with the GitHubAuditData parser.

https://github.com/Azure/Azure-Sentinel/blob/master/Solutions/GitHub/Parsers/GitHubAuditData.yaml

It uses a function called unixtime_milliseconds_todatetime() on the column “CreatedAt” which doesn’t always exist. This means for those rows it’s returning null and effectively filtering them out from the source.

It doesn't seem to be isolated or a logging issue. I work for an MSSP and can see the same discrepancy across all of our customers with the data source as well.

**To Reproduce**
Steps to reproduce the behavior:
1. Go to 'Functions' in Log Analytics
2. Click on 'Load code to editor' when hovered over GitHubAuditData
3. Scroll down to lines 4 and 22.
4. See issue described above. If you comment these out the returned results will increase significantly.

**Expected behavior**
Implement a fallback or remove this filter entirely. When I compared the existing default TimeGenerated column to the new extended column (where CreatedAt was populated) there was no difference.

**Screenshots**
Image

**Desktop (please complete the following information):**
- OS: Windows
- Browser: Edge
- Version: 150.0.4078.48

**Smartphone (please complete the following information):**
N/A

**Additional context**
I'd be grateful if you could confirm when we can expect the fix to be pushed Sentinel side and what the remediation steps would be for us. Should it just be a case of updating the parser via the Content Hub or will there be some additional manual work required?

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.