EFForg / EFForg/rayhunter

Opt-in HTTP tap for the raw /dev/diag byte stream

Open
#984 12 comments 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Rust
Stars
5.8k
Forks
490
Avg merge
3d 10h
Merged PRs (30d)
23

Description

## Prerequisites

- [x] I have read [CONTRIBUTING.md](https://github.com/EFForg/rayhunter/blob/main/CONTRIBUTING.md).

## What problem does this feature solve or what does it enhance?

The Linux `diag` character device only permits **one consumer at a time in memory-device mode**. As soon as rayhunter holds that handle, any second tool that wants to read `/dev/diag` (QCSuper, other DIAG analyzers, bespoke debugging/capture scripts) is refused by the kernel with `EBUSY`.

Users on a supported device therefore have to choose: run rayhunter's IMSI-catcher analysis **or** run their second tool — never both at the same time. That forces a workflow of "stop rayhunter → run the other tool → restart rayhunter and lose the passive-monitoring gap", which undercuts rayhunter's core value proposition as a persistent background monitor.

This is a known rough edge. It's also a tiny sliver of the much larger multiplexer/unification argument that @trevormjaymes made in [#895 (comment)](https://github.com/EFForg/rayhunter/issues/895#issuecomment-3910593197) — where the proposed end-state is a full `DiagServer` that owns `/dev/diag`, with rayhunter and other clients as fan-out consumers. That is a substantial architectural shift; I'm not proposing to take it on here. I'm proposing the **minimum** change that lets a second DIAG tool coexist with rayhunter today, on one supported device, gated behind an opt-in flag — a stepping stone that could be replaced or absorbed by a fuller multiplexer later without changing the public wire contract much.

## Proposed Solution

Add an **opt-in, off-by-default** HTTP endpoint on the existing axum server:

```
GET /api/diag/stream
```

- Controlled by a new `diag_stream_enabled` boolean in `config.toml`. Default `false`; behavior unchanged for every existing user.
- Also gated on `!debug_mode`, since there is no real `DiagDevice` to tap in debug mode.
- When active, rayhunter's existing single-consumer read of `/dev/diag` is **tee'd** into a bounded `tokio::sync::broadcast` channel *before* parsing. Each HTTP subscriber gets its own `BroadcastStream` view.
- Response is `Content-Type: application/octet-stream`, `Transfer-Encoding: chunked`, `X-DTAP-Version: 1`. Body is the raw bytes rayhunter reads from `/dev/diag` — Qualcomm DIAG memory-device-mode `MessagesContainer` records, byte-for-byte identical to what a direct `/dev/diag` consumer would see. (HDLC framing lives inside each inner message's `data`.)
- Slow subscribers observe `RecvError::Lagged(n)` and drop frames; a single warning is logged per lag. **rayhunter's own parser is never gated on subscriber speed** — that's the key safety property.
- When disabled, the endpoint returns `503 Service Unavailable` with a descriptive message so clients get a clear signal instead of hanging.

The implementation is small (~150 lines of Rust, all behind `diag_stream_enabled`). I made a proof-of-concept and tested it end-to-end on hardware, but per the CONTRIBUTING.md guidance on substantial contributions I'm raising it here before opening a PR. The working branch — if it helps the discussion — is [`lukejenkins/rayhunter:feat/diag-stream-http`](https://github.com/lukejenkins/rayhunter/tree/feat/diag-stream-http).

**Testing already run on an Orbic RC400L:** wire-format smoke test, multi-subscriber byte-for-byte equality check, independent parser validation of captured output (unwrap `MessagesContainer` → HDLC CRC-CCITT verify → DLF decode → 7/7 `LteRrcOtaMsg` parsed cleanly), and a coexistence check confirming rayhunter's own QMDL recording runs uninterrupted while two subscribers are live.

## Alternatives Considered

- **Stop rayhunter, run the other tool, restart** — the status quo. Defeats the point of passive monitoring; creates a gap during which IMSI-catcher activity would go undetected.
- **Ship a separate `diag-multiplexer` daemon that owns `/dev/diag` and offers IPC to rayhunter + other clients** — closer to the full architecture sketched in [#895 (comment)](https://github.com/EFForg/rayhunter/issues/895#issuecomment-3910593197). Cleaner long-term, but a much larger change: new binary, new installer story, new recovery story, breaks the current single-binary model. Worth doing eventually; out of scope here.
- **Export only rayhunter's post-decode PCAP stream** — lossy. Rayhunter's PCAP conversion only covers a subset of log codes; a second tool running DIAG-level analysis needs the raw bytes. Also doesn't help tools that depend on DIAG request/response commands rather than passive log reads.
- **Background-export each read to a file, let a second tool tail the file** — works, but: disk I/O on small-flash devices, file-rotation complexity, and a tailer sees writes at rayhunter's read cadence with whatever buffering the filesystem adds. An HTTP stream is cheaper to reason about and doesn't touch storage.
- **Add a compile-time feature flag instead of a runtime config flag** — runtime is preferable: same binary works for everyone; users don't need to rebuild to enable it, and distributions don't have to pick a single default.

## AI-assisted contribution disclosure

~~This issue body was written by me.~~ The patch itself was drafted with AI assistance (Claude); design, tap insertion point, backpressure semantics, 503 gating, `X-DTAP-Version`, config-flag naming, and wording were reviewed line-by-line. Hardware validation on RC400L was run and reviewed by me.

Contributor guide

Open the contributing guide

Research direction

Review CONTRIBUTING.md, config.toml, and the existing axum server first; trace the current /dev/diag read path and where raw bytes are available before parsing. Done means an opt-in endpoint streams raw bytes with the stated headers and 503 behavior when disabled, without slowing rayhunter’s parser; validate with the wire-format, multi-subscriber, parser, and coexistence checks described in the issue.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
api, backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.