EFForg / EFForg/rayhunter

Refactor analysis format to permit more than one event per (analyzer, msg)

Open
#1,139 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Rust
Stars
5.8k
Forks
490
Avg merge
3d 10h
Merged PRs (30d)
23

Description

Our current analyzer format has two arrays that are of the same length `N`:

* `analyzers` array contains all enabled analyzers
* within each analysis row, we have a `events` array with the exact same length. each entry is `null` if the respective analyzer didn't find anything, or an event if it is.

This has already led to bugs: https://github.com/EFForg/rayhunter/commit/33e4fbc54477c6d4afb43b9823b0d9e79d5f1155

and in #1132 we introduced another bug because there, either `update_timestamp` or `analyze_information_element` would return a finding (so the `events` array would be of length `2N` instead of `N`) -- this would then sometimes crash the UI

Fundamentally this format forces us to only emit one finding per row from each analyzer.

## Solution

I think we can change this in a backwards compatible way by adding an explicit `analyzer_index` property to each event.

When reading an event, we would first try to use the `analyzer_index` and fall back to the actual array index if that prop doesn't exist.

New versions of rayhunter would emit recordings that cannot be read by old versions, but not the other way around.

Contributor guide

Open the contributing guide

Research direction

Start by locating the analyzer event serialization and reading paths, including the UI code that consumes the events array. Trace the existing fallback behavior and the paths introduced by #1132; done means multiple events can identify one analyzer, older recordings still load, and the UI no longer crashes on the expanded format.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
backend-api-design
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
52/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.