Custom cursors support
@unera is already working on this.
Since Jun 13, 2023.
- Dominant language
- Lua
- Stars
- 43
- Forks
- 17
- Avg merge
- 28m
- Merged PRs (30d)
- 2
Description
There is a problem with cursor based pagination on requests by secondary indexes. Consider a simple key-value schema:
| field | type | info |
|---|---|---|
| key | string | primary key |
| value | string | |
| tag | string | secondary index, non unique |
The call crud.select('kv', {{'=', 'tag', 'tagValue'}}, {first = 10, after = ...}) makes a fullscan of secondary index from starting point (tagValue) to tuple equaling cursor. Under heavy load it leads to CPU consumption and cluster performance degradation.
To solve this problem, we have to create a special unique index paging with two fields {tag, key} and implement pagination manually, by executing such calls like
local rs = crud.select('kv', {{'==', 'paging', {'tagValue', box.NULL}}}, {first = 10})
...
local last_key = rs.ros[#rs.rows][0]
rs = crud.select('kv', {{'>', 'paging', {'tagValue', last_key}}}, {first = 10})
...
It would be nice to introduce some feature like passing name of paging index and using key from this index as cursors
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.