aidotse / aidotse/LeakPro

web-app fix

Aberta
#426 0 comentários 0 reações 0 responsáveis Ver no GitHub
Linguagem predominante
Python
Estrelas
23
Forks
28
Merge médio
4d 12h
PRs com merge (30d)
5

Descrição

## Attacks temporarily removed

The following attacks have been removed from the webapp attack list for now, pending fixes:

- **Population (P-attack)** — see issues below
- **YOQO** — see issues below

## Attacks to add in the future

- **OSLO**
- **SeqMIA**

---

## Known issues

### Population attack — no risk value shown in results

**Setup:** ResNet18, epochs=20, lr=0.001, bs=128, optimizer=Adam, f_train=0.5, f_test=0.2, no DP-SGD.

**Problem:** The attack runs to completion but the results summary shows no risk value.

The ROC curve looks unusual. TPR@0%–10% FPR should be ~0 (near-random attack on a well-generalising model), but the webapp displays `NaN` for all TPR@X values, causing the risk score to also be `NaN`.

**Questions to resolve:**
- Should TPR@X be `NaN` or `0` when the attack is near-random?
- If `NaN` is correct, the risk label should reflect "extremely low" rather than being absent.
- Should the Population attack have different default settings (e.g. `attack_data_fraction`) that produce more meaningful results in typical webapp usage?

**Main concern:** A model with 100% train accuracy runs the Population attack successfully but shows no risk value in the summary — this is confusing to users.

---

### RaMIA —
**Error:**
"/home/fazeleh/LeakPro/leakpro/attacks/attack_scheduler.py", line 124, in run_attacks result = attack_obj.run_attack() ^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fazeleh/LeakPro/leakpro/attacks/mia_attacks/ramia.py", line 344, in run_attack latent_features = self.get_latent_features(aug_dataloader, method="umap") ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fazeleh/LeakPro/leakpro/attacks/mia_attacks/ramia.py", line 262, in get_latent_features f = feature_extractor(x) ^^^^^^^^^^^^^^^^^^^^ File "/home/fazeleh/miniconda3/envs/leakpro_py311/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fazeleh/miniconda3/envs/leakpro_py311/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fazeleh/miniconda3/envs/leakpro_py311/lib/python3.11/site-packages/torch/nn/modules/container.py", line 219, in forward input = module(input) ^^^^^^^^^^^^^ File "/home/fazeleh/miniconda3/envs/leakpro_py311/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1553, in _wrapped_call_impl return self._call_impl(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fazeleh/miniconda3/envs/leakpro_py311/lib/python3.11/site-packages/torch/nn/modules/module.py", line 1562, in _call_impl return forward_call(*args, **kwargs) ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^ File "/home/fazeleh/miniconda3/envs/leakpro_py311/lib/python3.11/site-packages/torch/nn/modules/batchnorm.py", line 176, in forward return F.batch_norm( ^^^^^^^^^^^^^ File "/home/fazeleh/miniconda3/envs/leakpro_py311/lib/python3.11/site-packages/torch/nn/functional.py", line 2512, in batch_norm return torch.batch_norm( ^^^^^^^^^^^^^^^^^ RuntimeError: NVML_SUCCESS == DriverAPI::get()->nvmlInit_v2_() INTERNAL ASSERT FAILED at "/home/conda/feedstock_root/build_artifacts/libtorch_1728241823685/work/c10/cuda/CUDACachingAllocator.cpp":806, please report a bug to PyTorch.

---

### YOQO — missing results

**Problem:** When YOQO runs, the results page shows:
- No TPR@0.1% FPR
- Risk is N/A
- No ROC curve
- No histogram
- No records

Root cause not yet investigated. Removed from webapp until resolved.

**Note:** Default number of shadow models should be 2 (not 1) when YOQO is re-added.

Guia de contribuição

Abrir o guia de contribuição

Avaliação

Esta issue ainda não foi avaliada.

Receba novas issues na sua caixa de entrada

Um resumo curto de issues do GitHub para quem está começando.