acts-project / acts-project/acts

EDM Memory usage

Open
#1,516 2 comments 0 reactions 0 assignees View on GitHub
Stale
Dominant language
C++
Stars
131
Forks
276
Avg merge
3d 13h
Merged PRs (30d)
112

Description

The peak memory consumption of the track state EDM strongly increases when pileup and even complexity is increased. For ttbar events at $\mu=200$ from Pythia8, it can go up to a few gigabytes per event, which is suboptimal.

This issue tracks progress to improve and optimize the memory consumption.

## Associated PRs:

- #1507
This helps centralize the `MultiTrajectory` usage: a single instance is used by the examples algorithms to accumulate tracks from all seeds
- #1512
Switches from overallocating vectors and matrices for the measurements to using a jagged vector structure that looks like this:
```
| x x x | x | x x | ...
^ ^ ^
M1,D=3 M2,D=1 M3,D=2
```
- #1511
Implements statistics on the memory usage of `Acts::VectorMultiTrajectory` to give us an idea of what's going on.

## Status quo
The tooling from #1511 gives the following measurements from `MultiTrajectory`:
### ODD full chain example with Pythia8 ttbar @ mu200
```
other:
count: 1877425.000000
index: 128.91M
parPred: 85.94M
covPred: 515.65M
parFilt: 0.00M
covFilt: 0.00M
parSmth: 85.94M
covSmth: 515.65M
meas: 25.08M
measCov: 150.46M
jac: 515.65M
sourceLinks: 18.50M
projectors: 4.18M
total: 2045.97M
meas:
count: 653022.00
index: 44.84M
parPred: 27.47M
covPred: 164.82M
parFilt: 29.89M
covFilt: 179.36M
parSmth: 29.89M
covSmth: 179.36M
meas: 29.89M
measCov: 179.36M
jac: 164.82M
sourceLinks: 9.96M
projectors: 4.98M
total: 1044.66M
```
### ODD truth tracking with Pythia8 ttbar @ mu200
```
count: 40146.000000
index: 2.76M
parPred: 1.84M
covPred: 11.03M
parFilt: 0.00M
covFilt: 0.00M
parSmth: 1.84M
covSmth: 11.03M
meas: 0.00M
measCov: 0.00M
jac: 11.03M
sourceLinks: 0.31M
projectors: 0.00M
total: 39.82M
meas:
count: 37159.00
index: 2.55M
parPred: 1.70M
covPred: 10.21M
parFilt: 1.70M
covFilt: 10.21M
parSmth: 1.70M
covSmth: 10.21M
meas: 1.70M
measCov: 10.21M
jac: 10.21M
sourceLinks: 0.57M
projectors: 0.28M
total: 61.24M
```

## Other ideas
- [ ] Switch to float for storage of parameters and covariances to float
- [ ] Only store lower left triangle of covariance matrices. @tboldagh points out there is actually a triangular view [directly in Eigen](https://github.com/acts-project/acts/issues/1516#issue-1372460714)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.