JanusGraph / JanusGraph/janusgraph
Fix CQLPagingIterator for Amazon Keyspace
- Dominant language
- Java
- Stars
- 5.8k
- Forks
- 1.2k
- Avg merge
- 13h 53m
- Merged PRs (30d)
- 6
Description
Unfortunately, Amazon Keyspace diverges from Apache Cassandra when it comes to paging.
Due to the way Amazon Keyspace works, even if a page iterator returns empty results, there still can be a valid next page with data. This happens when `ALLOW FILTERING` is used. In JanusGraph, this means when a full scan is triggered. We should fix CQLPagingIterator in CQLKeyColumnValueStore.java.
At the moment, I am not 100% sure whether this would also require a fix from the DataStax java driver which implements paging internally as well.
C.C. @porunov since you did quite a bit of the CQLPagingIterator implementation.
References:
https://stackoverflow.com/questions/69940435/amazon-keyspaces-allow-filtering-without-all-keys-in-node-js
https://docs.aws.amazon.com/keyspaces/latest/devguide/working-with-queries.html#paginating-results
https://github.com/JanusGraph/janusgraph/discussions/3378#discussioncomment-4405659
Contributor guide
Research direction
Start in CQLKeyColumnValueStore.java by reading CQLPagingIterator and compare its paging behavior with the Amazon Keyspaces documentation and linked discussion. Determine whether the DataStax Java driver also needs changes. Done means full scans using ALLOW FILTERING continue past empty pages and retrieve valid data from later pages.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- aws, cassandra, java
- Domain
- databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 38/100