aws / aws/graph-explorer

[Bug] RDF: Adding resource to the canvas is very slow or fails with out of memory

Open
#263 2 comments 0 reactions 0 assignees View on GitHub
exploration performance search
Dominant language
TypeScript
Stars
481
Forks
108
Avg merge
8d 9h
Merged PRs (30d)
7

Description

**Community Note**
* Please use a 👍 reaction to provide a +1/vote. This helps the community and maintainers prioritize this request.
* If you are interested in working on this issue or have submitted a pull request, please leave a comment.

**Describe the bug**
On a larger RDF database, when I add a resource from the search panel to the canvas it can take anywhere from 30 seconds to 10 minutes to complete. During this time no indication is given to the user that something is happening.

- OS: macOS 14.3.1
- Browser: Arc (Google Chromium)
- Graph Explorer Version: 1.5.1
- Graph Database & Version: Amazon Neptune

**To Reproduce**
Steps to reproduce the behavior:
1. Connect to a large RDF database with SPARQL
2. Search for a resource with many neighbors or relationships
3. Add that resource to the canvas
4. Observe nothing happening in the UI for a long time

You can see the pending request in the browser's network tab.

**Slow response**

In this example, the request took 1.4 min to complete:

![CleanShot 2024-03-08 at 16 14 56@2x](https://github.com/aws/graph-explorer/assets/212862/1a10751e-f852-44e4-ab17-6c14731bd3a5)

And here you can see there are not that many neighbors:

![CleanShot 2024-03-08 at 16 14 43@2x](https://github.com/aws/graph-explorer/assets/212862/9384d4e7-27d8-4513-93f6-f1626f58272a)

And here is the query that was executed:

```sparql
SELECT ?class (COUNT(?class) AS ?count) {
?subject a ?class {
SELECT DISTINCT ?subject ?class {
?subject a ?class .
{ ?subject ?p }
UNION
{ ?p ?subject }
}
LIMIT 500
}
}
GROUP BY ?class
```

**Out of Memory Error**

The query that was executed was:

```sparql
SELECT ?class (COUNT(?class) AS ?count) {
?subject a ?class {
SELECT DISTINCT ?subject ?class {
?subject a ?class .
{ ?subject ?p }
UNION
{ ?p ?subject }
}
LIMIT 500
}
}
GROUP BY ?class
```

This resulted in an out of memory error:

```json
{
"error": {
"status": 500,
"message": "\n{\n \"detailedMessage\": \"Operation terminated (out of memory)\",\n \"requestId\": \"38d41423-0bb8-446a-8d11-4de1ee8cfb24\",\n \"code\": \"MemoryLimitExceededException\",\n \"message\": \"Operation terminated (out of memory)\"\n}"
}
}
```

**Expected behavior**
Adding a single resource to the canvas should not be slow or cause errors.

Contributor guide

Open the contributing guide

Research direction

Start at the search panel's add-to-canvas flow and reproduce against a large RDF database such as Amazon Neptune. Inspect the pending browser network request and the generated SPARQL query, then trace the request path to identify the slow or memory-intensive operation. Done means adding one resource completes promptly without an out-of-memory error and gives the user progress feedback.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, react, typescript
Domain
databases, frontend, performance
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.