Joystream / Joystream/storage-squid
Custom SQL for most heavy queries
@zeeshanakram3 is already working on this.
Since Dec 1, 2023.
- Dominant language
- TypeScript
- Stars
- 0
- Forks
- 3
- PR merge metrics
- No merged PRs in 30d
Description
While the squid by itself brings a huge performance difference compared to the QN (10-15 minutes -> 30 sec) when fetching all the objects that a given distributor should serve, 30 sec at 100% CPU usage is still not a great result considering we fetch "only" 1M objects. My guess is that the auto-generated SQL query is quite unoptimized. Since getting all objects for a given worker will be done quite often, there's a lot of space for optimization. I think we could write a custom query with hand-written SQL that ensures good performance for those big queries.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.