Darshan808 / Darshan808/NDN-Toolkit

Implement Real-Time NDN Anomaly Detection with Isolation Forest

Open
#1 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

## Description

We want to build a lightweight, real-time anomaly detection system for our Named Data Networking (NDN) experiments in miniNDN using **Isolation Forest**.

The system should:
- Run in real time (maybe 1,2 or 2,3 second intervals)
- Monitor 9–12 router nodes
- Use only **global NFD counters** (no per-face/FIB/RIB parsing)
- Train **one single Isolation Forest model** on normal behavior from all nodes
- Score each node independently in real time
- Produce per-node anomaly alerts/scores

### Target Features (10 features per sample)

Every 1–2 seconds, extract/compute these 10 values from each monitored node:

1. `pit_size` → `nPitEntries` (absolute)
2. `pit_growth_rate` → delta `nPitEntries` / seconds
3. `cs_size` → `nCsEntries`
4. `cache_hit_ratio` → `nHits / (nHits + nMisses)` (0 if no activity)
5. `satisfaction_ratio` → `nSatisfiedInterests / (nSatisfied + nUnsatisfied)`
6. `unsatisfied_ratio` → `nUnsatisfiedInterests / (nSatisfied + nUnsatisfied)`
7. `in_interests_rate` → delta `nInInterests` / seconds
8. `out_interests_rate` → delta `nOutInterests` / seconds
9. `in_data_rate` → delta `nInData` / seconds
10. `nack_rate` → delta `(nInNacks + nOutNacks)` / seconds

### Data Collection Plan

Capture **only** these two lightweight outputs per node (in a loop):

```bash
nfdc status. # → main counters (JSON-like)
nfdc cs info # → nHits, nMisses, nCsEntries
```

Do **not** use full `nfdc status report` (too verbose, contains Faces/FIB/RIB we don't need).

### Training Phase (Offline / One-time)

1. Run multiple long **normal-traffic** experiments in miniNDN (varied rates, topologies, cache sizes, Zipf parameters)
2. Collect 1-second samples from **all nodes** for 10–30 minutes each run
3. Parse into CSV with columns:
```
timestamp,node_id,pit_size,pit_growth_rate,cs_size,cache_hit_ratio,satisfaction_ratio,unsatisfied_ratio,in_interests_rate,out_interests_rate,in_data_rate,nack_rate
```
4. Drop `timestamp` and `node_id` → train Isolation Forest on the resulting (N, 10) matrix

```python
from sklearn.ensemble import IsolationForest
model = IsolationForest(contamination=0.01, n_estimators=200, random_state=42)
model.fit(X_normal)
```

Target: 15,000–50,000 normal samples total (across all nodes).

### Real-Time Inference Phase

Architecture:
- Each node (or central collector) computes its 10-feature vector every 1–2 seconds
- Send vector + node_id + timestamp to monitoring component
- Score with the pre-trained model:
```python
score = model.decision_function(vector) # more negative = more anomalous
```
- If score < threshold (e.g. –0.4 or –0.5), raise alert for that node

Goal: Per-node alerts like
`ANOMALY DETECTED on node router-5: score = -0.72 (high PIT growth + low satisfaction)`

### Todo for us
- [x] Script to collect & parse `nfdc status` + `nfdc cs info` into features from real NDN routers.
- [x] Simulation that generates tons of data. ( What topologies do we use? )
- [x] Train an isolation forest model.
- [x] Model saved (`joblib.dump`) and loaded in real-time component
- [x] Real-time monitoring loop / service that:
- Maintains previous state per node (for deltas)
- Computes 10 features
- Scores each vector
- Logs/alerts on anomalies (console, file, or simple HTTP endpoint)
- [x] Tested on a 9–12 node topology with normal traffic (no false positives)
- [x] Tested on similar topology with abnormal traffic

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.