aws / aws/graph-explorer

Spike: Redesign Cytoscape abstraction layer for performance and simplicity

Open
#1,797 1 comment 0 reactions 1 assignee Claimed by @kmcginnes View on GitHub
exploration internal needs-triage performance tech debt
Dominant language
TypeScript
Stars
481
Forks
108
Avg merge
6d 8h
Merged PRs (30d)
5

Description

## Goal

Determine the best approach to redesign the React abstraction over Cytoscape to achieve:
- **React-native DX**: Setting graph nodes/edges should feel like setting React state. The imperative Cytoscape machinery should be invisible to consumers.
- **Performance at scale**: Target 10k+ nodes with smooth interaction. The current implementation degrades noticeably around 400-600 nodes.
- **Ambitious simplification**: The current abstraction (13+ hooks, multiple style conversion layers, bidirectional state sync) is too complex. A fresh design should dramatically reduce the surface area.

> [!CAUTION]
> **Internal spike only.** This issue requires deep familiarity with the existing Cytoscape integration and internal performance constraints. It is not suitable for community contribution. More research must be done before any implementation changes can be made.

## Context

### Prototype Findings

A D3-based replacement prototype was built on branch `prototype/d3-graph-canvas` (8 commits). The prototype validated several design ideas but hit a hard ceiling: **SVG rendering caps around 1-2k nodes regardless of tick() optimization, because each node is a DOM element.**

Conclusion: replacing Cytoscape with raw D3 SVG is just reimplementing Cytoscape worse. The right path is to keep Cytoscape's Canvas 2D renderer (which already handles large graphs well) but dramatically improve the React wrapper around it.

### Performance Research

Scaling graph rendering to 10k-50k nodes requires:

| Scale | Rendering | Physics |
|-------|-----------|---------|
| <1k | SVG fine | Main thread fine |
| 1k-5k | Canvas 2D | Main thread with tuning |
| 5k-50k | Canvas 2D or WebGL | Web Worker required |
| 50k+ | WebGL (PIXI/Sigma/cosmos) | GPU or Web Worker |

Key techniques that may apply to the Cytoscape wrapper:

1. **Web Worker for force layout** — Cytoscape supports `webWorker: true` on some layout extensions (fcose has experimental support)
2. **Batch/incremental updates** — The current code does `cy.json({elements: cloneDeep({nodes, edges})})` which replaces the entire graph. Incremental `cy.add()`/`cy.remove()` diffs would be far cheaper
3. **Viewport culling** — Cytoscape has `textureOnViewport` and `hideEdgesOnViewport` options
4. **Style caching** — The current `useManageStyles` rebuilds the full stylesheet on every style/layout/badge change. Targeted `cy.style().selector().css()` updates may be cheaper

### Current Architecture Problems

1. **Too many hooks with tangled dependencies** — 13 hooks all depend on the `cy` instance ref and trigger effects off each other
2. **Deep clone on every data change** — `cy.json({elements: cloneDeep(...)})` is O(n) allocation + O(n) reconciliation even for a single node addition
3. **Bidirectional selection sync is fragile** — Selection state flows React→Cy and Cy→React, causing potential feedback loops (mitigated by debouncing)
4. **Style conversion is a bottleneck** — Domain styles → intermediate format → Cytoscape CSS, rebuilt on every change
5. **40+ props on the Graph component** — consumers need to understand too much about the internal state model
6. **No incremental updates** — adding 1 node rebuilds the entire element set

### Preliminary Design Direction

A simplified API with ~10 props (down from 40+), incremental element diffing, unidirectional selection flow, stable event subscriptions, and 3-4 internal hooks instead of 13. This direction needs validation through:

- Benchmarking incremental `cy.add()`/`cy.remove()` vs `cy.json()` at various scales
- Testing whether `cy.style().selector().css()` actually avoids full re-render
- Determining if Cytoscape's web worker layout support is stable enough for production
- Identifying which of the 40+ props are actually needed vs. dead surface area
- Verifying that unidirectional selection doesn't introduce perceptible input lag

## Expected Outcome

1. **Benchmarks**: Performance comparison of current vs. proposed element sync strategies at 100, 1k, 5k, and 10k nodes
2. **API proposal**: Finalized public API for the new `GraphCanvas` component with type definitions
3. **Migration path**: Document which hooks/behaviors can be deleted outright vs. must be preserved
4. **Tasked-out implementation plan**: Break the redesign into shippable increments that can be merged independently

## Research Questions (must be answered before implementation)

- [ ] Does `cy.add()`/`cy.remove()` actually perform better than `cy.json()` at scale, or does Cytoscape do its own internal reconciliation?
- [ ] What is the real-world performance ceiling of Cytoscape's canvas renderer? Is 10k nodes achievable with layout tuning alone?
- [ ] Is `fcose` web worker support stable? What are the tradeoffs?
- [ ] Can `textureOnViewport` + `hideEdgesOnViewport` be enabled adaptively (only when node count exceeds a threshold)?
- [ ] Does the SchemaGraph consumer have any requirements that diverge from GraphViewer?
- [ ] What is the minimum viable set of props/behaviors that both consumers actually need?

## Related Issues

- Related to #1725
- Related to #1726
- Related to #1679
- Related to #1677
- Related to #601

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.