aws / aws/graph-explorer

Support graphs with very large numbers of node types (70k+)

Open
#1,677 2 comments 1 reaction 1 assignee Claimed by @kmcginnes View on GitHub
enhancement performance schema
Dominant language
TypeScript
Stars
481
Forks
108
Avg merge
6d 8h
Merged PRs (30d)
5

Description

## Description

Graph Explorer does not handle schemas with very large numbers of node types (70k+) gracefully. Multiple areas of the codebase assume a small-to-moderate number of types and perform operations that scale poorly — O(n²) merge loops, per-type Cytoscape style selectors, per-type TanStack Query instances, non-virtualized dropdowns, and thousands of sequential network requests during schema sync.

The result is that the React app locks up or becomes unresponsive when connected to a graph with tens of thousands of node types.

## High-Level Plan

Break the work into four phases, each independently shippable:

1. **Eliminate O(n²) in the state layer** — Index arrays into Maps in `mergeConfiguration()`, convert preference lookups from `.find()` to Map-based, replace `cloneDeep` with shallow spread.

2. **Scope graph styles to canvas types only** — Change `useGraphStyles`, `useBackgroundImageMap`, and the Legend to only process types that are actually present on the graph canvas, not all schema types.

3. **Virtualize UI for large type lists** — Replace non-virtualized Radix Select dropdowns with a searchable, virtualized combobox for node type selection. *(In progress on `large-schema-performance` branch.)*

4. **Improve schema sync for large schemas** — Add progress reporting and cancellation, increase batch concurrency, and consider lazy attribute fetching.

---

> [!IMPORTANT]
> If you are interested in working on this issue, please leave a comment.

> [!TIP]
> Please use a 👍 reaction to provide a +1/vote. This helps the community and maintainers prioritize this request.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.