Refactor analysis format to permit more than one event per (analyzer, msg)
- Dominant language
- Rust
- Stars
- 5.8k
- Forks
- 490
- Avg merge
- 3d 10h
- Merged PRs (30d)
- 23
Description
Our current analyzer format has two arrays that are of the same length `N`:
* `analyzers` array contains all enabled analyzers
* within each analysis row, we have a `events` array with the exact same length. each entry is `null` if the respective analyzer didn't find anything, or an event if it is.
This has already led to bugs: https://github.com/EFForg/rayhunter/commit/33e4fbc54477c6d4afb43b9823b0d9e79d5f1155
and in #1132 we introduced another bug because there, either `update_timestamp` or `analyze_information_element` would return a finding (so the `events` array would be of length `2N` instead of `N`) -- this would then sometimes crash the UI
Fundamentally this format forces us to only emit one finding per row from each analyzer.
## Solution
I think we can change this in a backwards compatible way by adding an explicit `analyzer_index` property to each event.
When reading an event, we would first try to use the `analyzer_index` and fall back to the actual array index if that prop doesn't exist.
New versions of rayhunter would emit recordings that cannot be read by old versions, but not the other way around.
Contributor guide
Research direction
Start by locating the analyzer event serialization and reading paths, including the UI code that consumes the events array. Trace the existing fallback behavior and the paths introduced by #1132; done means multiple events can identify one analyzer, older recordings still load, and the UI no longer crashes on the expanded format.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- rust
- Domain
- backend-api-design
- Issue type
- Refactor
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Active
- Clarity
- Mostly clear
- Newbie friendliness
- 52/100