nodeSolidServer / nodeSolidServer/node-solid-server
Server limitations when querying multiple pods (>128) in parallel
Nobody has claimed this yet.
- Dominant language
- JavaScript
- Stars
- 1.8k
- Forks
- 308
- PR merge metrics
- No merged PRs in 30d
Description
Hi,
I am running some experiments querying more than 128 pods in parallel from the Inrupt server and SolidCommunity. In both cases when I query data from more than 128 pods (i.e., 256, 512, 1024) at the same time the server stops the queries and blocks them.
When I experiment with 64 or fewer pods there is no problem. However, I have to do it in batches of 64 pods when I want to query more than 64 pods in parallel otherwise the server starts blocking my HTTP requests.
Is it expected that a SOLID server block parallel queries for a large number of pods from the same client?
Thank you
Crossposted: https://github.com/inrupt/pod-server/issues/76
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
No source file, test, or entry point is named in the issue. Start by reproducing parallel queries against more than 128 pods and compare them with batches of 64 or fewer, then determine whether the blocking is expected server behavior. Done means the concurrency limit and expected behavior are identified and either documented or addressed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- backend
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100