Graylog2 / Graylog2/graylog2-server

Feature to use Lookup Tables in search queries

Open
#13,646 0 comments 0 reactions 0 assignees View on GitHub
feature
Dominant language
Java
Stars
8.1k
Forks
1.1k
Avg merge
1d 20h
Merged PRs (30d)
217

Description

## What?

I would be great to have a feature to use lookup tables in search queries.
I.e. for every event matching the search condition so far, make a lookup in the lookup table.

Useful queries to be supported could be:
- equals
- matches (regex matching)
- contains
- startswith
- endswith

For a given lookup Table _ioc-lookup_ and a key _domain_, a search syntax could look like the following:
`_exists_:domain AND lookup(ioc-lookup, equals(domain), destinationField)`
This should only return results if the field _domain_ exists and if there is an entry for the respective domain field in the lookup table and set the field destinationField to the looked up value for those results.

another example could be:

`_exists_:url AND lookup(ioc-lookup, matches(url, ".*(?:.com|.net).*"), destinationField)`
This should only return results where the field `url` exists and matches the regex, i.e. .com and .net domains with any string before and agter.
The search could return after the first match if there were multiple matches found in the lookup table and set the field destinationField to the looked up value for those results.

## Why?

Let’s assume all client DNS lookups are logged. A user clicks on a phishing link and enters his credentials.
Since the phishing site is very new, the domain is not contained in any IOC database and therefore a lookup of the domain in the Pipeline is not successful.
10 days later, the phishing domain is known and in the lookup table.
It would be awesome to be able to make a search if any client was affected in the past.

## Your Environment
Applicable for any environment.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.