Darshan808 / Darshan808/NDN-Toolkit

Train and Test Isolation Forest on miniNDN Logs

Open
#5 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
0
Forks
1
PR merge metrics
No merged PRs in 30d

Description

## Description

We now have clean, realistic normal + attack datasets from three topologies (Tree, Dumbbell, DFN) and three subtle attack types (Low-rate Distributed IFA, Cache Pollution, Content Poisoning).

The research focus for this milestone is:

- Achieve **maximum possible detection accuracy** on subtle attacks
- Specifically target attacks that **simple rule-based systems completely miss** (e.g. slow PIT creep, cache-hit-ratio collapse while satisfaction stays >95%, poisoned Data that still satisfies Interests)

This will allow us to build a compelling real-time monitoring system that demonstrably outperforms traditional threshold-based monitoring.

## Research Questions to Answer

- Does a single global model generalize across all topologies and node types?
- Do topology-specific or node-type-specific models give significantly better accuracy?
- Which features (pit_growth_rate, cache_hit_ratio, unsatisfied_ratio, etc.) are most discriminative for each attack type?
- How well does our model detect attacks that rule-based systems (e.g. “if unsatisfied > 20%” or “if PIT > 100”) completely ignore?

## Experiments to Run

- [x] **EDA & Feature Analysis**
- Correlation matrix, distribution plots, and time-series trends per topology and per node type
- Feature importance (via Isolation Forest’s `decision_function` + permutation importance or SHAP)
- Identify which features best separate subtle attacks from normal traffic

- [ ] **Model Strategy 1: Single Global Model**
Train one Isolation Forest on **all normal data** from all topologies + all node types combined
Test on all anomaly scenarios

- [ ] **Model Strategy 2: Per-Topology Models**
Train separate models for Tree / Dumbbell / DFN
Compare accuracy and false-positive rate across topologies

- [ ] **Model Strategy 3: Per-Node-Type Models**
Train three specialized models:
- Producer nodes only
- Consumer nodes only
- Router nodes only
(Because their baseline behavior is fundamentally different)

- [ ] **Rule-Based Baseline Comparison**
Implement simple rules used in literature (PIT threshold, satisfaction ratio < 90%, nInNacks spike, etc.)
Show side-by-side how many subtle attacks they miss vs. our Isolation Forest

- [ ] **Demo / Real-Time Monitoring**
Build a lightweight dashboard/script that shows:
- Live per-node anomaly score
- Highlight cases where rule-based system says “normal” but our model raises alert

## Acceptance Criteria

- [ ] At least 3 model variants trained and evaluated (global + per-topology or per-node-type)
- [ ] EDA notebook with clear insights on which features matter most for each attack
- [ ] Quantitative comparison table showing F1-score / Accuracy / False Positive Rate vs. rule-based baseline
- [ ] At least one clear demo case where rule-based monitoring stays silent but Isolation Forest correctly flags the attack
- [ ] Clean, reusable code structure (`train_global.py`, `train_per_topology.py`, `evaluate.py`, etc.)

Let’s aim for a strong result that shows ML (specifically Isolation Forest) can reliably catch the subtle, stealthy attacks that traditional NDN monitoring systems miss.

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.