google / google/neuroglancer

Visualize tractography-generated streamlines in Neuroglancer

Open
#555 9 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
1.5k
Forks
389
Avg merge
3d 21h
Merged PRs (30d)
3

Description

## Use case

We are looking to visualize tractography-generated streamlines from a Track File (.trk) in Neuroglancer. See [Track File docs](https://trackvis.org/docs/?subsect=fileformat) and an [example Track File](https://dandiarchive.org/dandiset/000724/draft/files?location=derivatives%2Fdwi%2Fsub-I58&page=1). We have a current implementation (see `Current implementation` section below), but are looking for advice on the optimal implementation (see `Open questions` section below). Thank you.

## Requirements

1. Add support in Neuroglancer to natively read and visualize a public Track File (.trk). (To prevent data duplication, we would prefer not to save into another format for visualization.)
1. Transform the streamlines from voxel space to world space.
3. To improve rendering performance, add front-end component(s) to filter the streamlines.
1. Add a slider to subselect a random number of streamlines (with a fixed seed for reproducible states).
2. Add a slider to subselect the number of points per streamline.
4. Add a front-end component to optionally display a colormap overlay on the streamlines. The colormap reflects the orientations of the streamlines at each vertex.

## Implementation details

1. Track File (.trk) [documentation](https://trackvis.org/docs/?subsect=fileformat)
2. Track Files store streamlines in voxel space. The `vox_to_ras` matrix is stored in the header and can convert the points from voxel space to world space.
3. A publicly available `trk` file can be found in [Dandiset 000724](https://dandiarchive.org/dandiset/000724/draft/files?location=derivatives%2Fdwi%2Fsub-I58&page=1). (Files can be much larger than this example.)
5. The [nibabel Python library](https://github.com/nipy/nibabel) can read `trk` files.

## Current implementation

My colleague (@balbasty) has created the [ngtools Python package](https://github.com/neuroscales/ngtools) with the accompanying [example notebook](https://github.com/neuroscales/ngtools/blob/main/notebooks/show_tract.ipynb). The [tract.py module](https://github.com/neuroscales/ngtools/blob/main/ngtools/tracts.py) performs the following operations:

1. Reads the streamlines into memory.
2. Selects a subset of streamlines (to improve performance).
3. For the subset of streamlines, creates concatenated arrays of the vertices, edges, and orientations. (Orientations are used for a colormap overlay.)
4. Converts the streamlines into a single, local skeleton layer by passing the above arrays into [neuroglancer.skeleton.Skeleton()](https://github.com/neuroscales/ngtools/blob/main/ngtools/tracts.py#L219). A previous version of this package converted the streamlines into a precomputed skeleton format.

After discussion with @balbasty, a downside to this approach is that a user must locally load all the streamlines into memory and perform this (minor) compute operation to convert the streamlines into a format that can be visualized. Thus the state of the Neuroglancer viewer with a Track File cannot readily be passed between users.

## Open questions

1. Is there a reference implementation for visualizing tractography-generated streamlines (from any file format) in Neuroglancer?
2. If there is not a reference implementation, what is the optimal way of displaying streamlines (Requirement 1)?
3. Would we need to display only a subset of streamlines to improve rendering performance at multiple scales (Requirement 2)?

cc @balbasty @aaronkanzer @mikeschutzman @satra @ayendiki

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.