Voxel annotation roadmap

# Voxel Annotation Roadmap

With #858 finally heading toward merge, here is a proposed plan for what comes next. The PR froze its scope to stay reviewable; this issue tracks what was descoped, deferred during review, and additional work, ordered by sections.

Within each section, needed items are part of the feature's intended scope; to evaluate items need a study or a community discussion first. The ordering and categorization are open to discussion.

Happy to discuss the content and ordering of this plan --- I'll keep it updated as discussions and work progress.

---

## 1. Merge

#### Needed:

- [ ] Merge #858 (final review in progress)

## 2. Consolidation and authenticated writing

#### Needed:

- [ ] Rework the drawing tool UX deferred from review
- paint value tool usefulness
```The paint value tool is currently not very useful beyond inspecting the value (the tool that shows on keybind, not the tool within the layer controls). I think that tool could either be changed in style and have a message to show only indicating. Or have an activation within it, to for e.g. increment/decrement the value, pick a random value, or use the segment picker to set the paint value to what is under the cursor.```
- undo/redo affordance
```the undo and redo tools that show on keybind are a bit misleading looking I think. The undo/redo action is performed on key press. But the tool looks like you need to press a button to perform the undo/redo.```
- picker vs. native selection
```I've had issues at times using the seg picker. Could it instead use the regular neuroglancer selection on control right click? Like for example (set value to match pinned selection). Maybe I'm missing what the seg picker is intending to add extra that needs its own tool.```
- floodfill handling when hitting the max limit
```on large fills the flood fill can just fail sometimes with no message. I'm aware this is really a lot of voxels, but maybe we should lower the max limit on the number of voxels allowed to be filled on tool? Not sure how the performance is for you at higher fill values.```
- [ ] An authentication mechanism for writing --- which one is still open
- [ ] Hardening of the image layer support, which currently lacks proper testing and user feedback

#### To evaluate:

- [ ] Replace the overlay's reserved sentinel values with an explicit edited-voxel mask
```I think it would be better to have a separate mask (ideally stored as a bitmask for space efficiency, though could also be stored as a bool array for simplicity) rather than relying on a sentinel value that could in fact occur in the data.```
- [ ] Write-consent model for shared links
```md
> I currently see two paths to solve this security flaw:
>
> 1. If the auth mechanism is permissive enough, maybe we could make it generate a unique token to prove that the creator of the link actually has write access.
> 2. We keep the annoying prompt but delay it until there is an actual drawing action. This serves a double purpose as a warning to ensure the user knows they are about to draw to _this specific_ datasource. We could reduce the annoyance by only prompting on links that haven't been opened before and/or adding a "do not ask again for this datasource" option, both stored in local storage.

Yes, I think only prompting when there is actually a write action is a good idea. I'm not sure that "do not ask again for this datasource" is a good idea because users will be tempted to use it for convenience but that effectively means: "allow anyone to trick me into corrupting this datasource". In particular, just because I have edited one datasource in Neuroglancer doesn't mean I want some random link sent to me to also accidentally edit it.

One possibility would be to key any persistent allowlist on the entire Neuroglancer state, i.e. a set of `(full_neuroglancer_viewer_state_hash, datasource_url)` tuples, but where any allowed datasources propagate automatically when the changes to the Neuroglancer state are made locally, and in particular adding a datasource or enabling writing on it interactively would automatically grant permission. Possibly some properties of the viewer state could be excluded though that introduces complications and I'm not sure how much benefit it provides. Somehow we'd have to avoid filling up local storage with allowed entries due to the constantly changing Neuroglancer state.

Potentially we could improve on this by having some mechanism to say: "allow writing to this datasource for any link created by this user". That would require some way to sign the Neuroglancer state with a per-user private key. The private key could just be generated by Neuroglancer and stored in browser local storage, though it might be better if it was tied to some signin mechanism so that it could be shared across browser profiles and machines.

In any case these strategies add a lot of complexity so as a starting point we can just always prompt on first write to a datasource.

>
> I think option 2 is better as it reduces dependency on specific external systems, but we can still search for other alternatives.

```

## 3. New capabilities

#### Needed:

- [ ] Write support for the core formats and stores the community already relies on, where voxel annotation applies
- precomputed and n5 datasources
- GCS and generic HTTP kvstores
- [ ] Mask mode: correction layers overriding another segmentation layer's rendering (prototyped downstream)

#### To evaluate:

- [ ] Lifting the current technical limitations
- zarr write gaps: sharded arrays, transposed arrays, remaining codecs
- Float32 data
- volumes of rank other than 3
- non-strict multiscale hierarchies
- [ ] Dataset creation from the viewer (descoped from #858)

## 4. Long term

#### To evaluate:

- [ ] Multi-user editing: concurrent writes, shared undo history, versioning --- possibly through commit-capable formats (OCDBT, Icechunk)
- [ ] Multi-resolution editing: efficient low-resolution drawing without upsampling, which requires a purpose-built editable multiscale format (#851)

No specific files, tests, or entry points are named. Start by reviewing #858 and the roadmap sections, then select a single item and define its scope, entry point, validation, and completion criteria in a separate issue.

Written by the indexing model from the issue text.