Visualize tractography-generated streamlines in Neuroglancer
- Dominant language
- TypeScript
- Stars
- 1.5k
- Forks
- 389
- Avg merge
- 3d 21h
- Merged PRs (30d)
- 3
Description
## Use case
We are looking to visualize tractography-generated streamlines from a Track File (.trk) in Neuroglancer. See [Track File docs](https://trackvis.org/docs/?subsect=fileformat) and an [example Track File](https://dandiarchive.org/dandiset/000724/draft/files?location=derivatives%2Fdwi%2Fsub-I58&page=1). We have a current implementation (see `Current implementation` section below), but are looking for advice on the optimal implementation (see `Open questions` section below). Thank you.
## Requirements
1. Add support in Neuroglancer to natively read and visualize a public Track File (.trk). (To prevent data duplication, we would prefer not to save into another format for visualization.)
1. Transform the streamlines from voxel space to world space.
3. To improve rendering performance, add front-end component(s) to filter the streamlines.
1. Add a slider to subselect a random number of streamlines (with a fixed seed for reproducible states).
2. Add a slider to subselect the number of points per streamline.
4. Add a front-end component to optionally display a colormap overlay on the streamlines. The colormap reflects the orientations of the streamlines at each vertex.
## Implementation details
1. Track File (.trk) [documentation](https://trackvis.org/docs/?subsect=fileformat)
2. Track Files store streamlines in voxel space. The `vox_to_ras` matrix is stored in the header and can convert the points from voxel space to world space.
3. A publicly available `trk` file can be found in [Dandiset 000724](https://dandiarchive.org/dandiset/000724/draft/files?location=derivatives%2Fdwi%2Fsub-I58&page=1). (Files can be much larger than this example.)
5. The [nibabel Python library](https://github.com/nipy/nibabel) can read `trk` files.
## Current implementation
My colleague (@balbasty) has created the [ngtools Python package](https://github.com/neuroscales/ngtools) with the accompanying [example notebook](https://github.com/neuroscales/ngtools/blob/main/notebooks/show_tract.ipynb). The [tract.py module](https://github.com/neuroscales/ngtools/blob/main/ngtools/tracts.py) performs the following operations:
1. Reads the streamlines into memory.
2. Selects a subset of streamlines (to improve performance).
3. For the subset of streamlines, creates concatenated arrays of the vertices, edges, and orientations. (Orientations are used for a colormap overlay.)
4. Converts the streamlines into a single, local skeleton layer by passing the above arrays into [neuroglancer.skeleton.Skeleton()](https://github.com/neuroscales/ngtools/blob/main/ngtools/tracts.py#L219). A previous version of this package converted the streamlines into a precomputed skeleton format.
After discussion with @balbasty, a downside to this approach is that a user must locally load all the streamlines into memory and perform this (minor) compute operation to convert the streamlines into a format that can be visualized. Thus the state of the Neuroglancer viewer with a Track File cannot readily be passed between users.
## Open questions
1. Is there a reference implementation for visualizing tractography-generated streamlines (from any file format) in Neuroglancer?
2. If there is not a reference implementation, what is the optimal way of displaying streamlines (Requirement 1)?
3. Would we need to display only a subset of streamlines to improve rendering performance at multiple scales (Requirement 2)?
cc @balbasty @aaronkanzer @mikeschutzman @satra @ayendiki
Contributor guide
Assessment
This issue has not been assessed yet.