acts-project / acts-project/acts

GNN metric learning chain finds no tracks: all edges dropped at the filter classifier

Aperta
#5,911 3 commenti 0 reazioni 0 assegnatari Vedi su GitHub
Bug
Lingua principale
C++
Stelle
131
Fork
276
Merge medio
3g 13h
PR unite (30g)
112

Descrizione

## Summary

`test_gnn_metric_learning[gpu]` runs `Examples/Scripts/Python/gnn.py` (metric-learning graph construction, torchscript `filter.pt` + ONNX `gnn.onnx` edge classifiers, `BoostTrackBuilding`) and the chain finds **no tracks at all**: the `track_finder_tracks` tree in `ntuple_finding_gnn.root` is empty and the performance writer reports `Efficiency with particles = 0`.

## Evidence

From the `gnn_gpu` job of https://github.com/acts-project/acts/actions/runs/32143985906/job/95830219625 (Tesla T4):

```
TrackFinding INFO GNN timing info
TrackFinding INFO - preprocessing: 0.00729 +- 0.00146 [0.00583, 0.00875]
TrackFinding INFO - graph building: 142.718 +- 137.238 [5.48, 279.955]
TrackFinding INFO - classifier: 7.55664 +- 6.33034 [1.2263, 13.887]
TrackFinding INFO - classifier: 0 +- 0 [0, 0]
TrackFinding INFO - track building: 0 +- 0 [0, 0]
RootPatternR INFO Efficiency with tracks (nMatchedtracks/nAlltracks) = -nan
RootPatternR INFO Efficiency with particles (nMatchedParticles/nTrueParticles) = 0
```

The zero-padded classifier/track-building times are the `NoEdgesError` path in `GnnPipeline::run` (`Plugins/Gnn/src/GnnPipeline.cpp`): the first edge classifier ran (7.6 ms), the second one and track building never did. So the torchscript filter stage (`cut = 0.01`) removed every edge, and the pipeline returned 0 track candidates.

## Why CI never caught this

The test asserted a reference hash of `ntuple_finding_gnn.root` (`30f11c53…`). Since the chain has produced this empty result all along, the reference hash *is* the empty-track output — the assertion held while the test verified nothing about track finding. It also explains why this test never showed the Turing/Ampere numerical split that `test_gnn_module_map[gpu-onnx]` does: with no tracks, the only content left in the ntuple is the CPU-side truth-particle tree.

The pipeline logs `No edges left in GNN pipeline, return 0 track candidates` at DEBUG only, so nothing in the CI log flagged it.

## Possible causes (unverified)

- Feature scaling: `gnn.py` passes `featureScales = [1.0, 1.0, 1.0]` for `[R, Phi, Z]`, while the models may expect normalised inputs.
- A mismatch between the `torchscript_models` (`embed.pt` / `filter.pt`) in CI model storage and the cuts used in the script (`filter` cut `0.01`, `gnn` cut `0.5`).
- Metric-learning graph construction parameters (`embeddingDim = 8`, `rVal = 1.6`, `knnVal = 100`).

## Suggested follow-ups

- Re-run the chain with `acts.logging.DEBUG` to get node/edge counts per stage and confirm where the edges disappear.
- Consider logging `No edges left in GNN pipeline` at WARNING. The test already sets `ACTS_LOG_FAILURE_THRESHOLD=WARNING` for the subprocess, so that alone would surface this class of failure immediately.

## Test status

#5903 replaces the reference-hash comparison in the GNN GPU tests with a smoke check (the hashes are not stable across the heterogeneous GPU pool). The track-content assertion is disabled for this test and the counts are only printed, so the test keeps running end-to-end without pinning the broken output; it should be re-enabled once this is fixed.

Guida per i contributori

Apri la guida per i contributori

Valutazione

Questa issue non è ancora stata valutata.

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.