JanusGraph / JanusGraph/janusgraph

Fix CQLPagingIterator for Amazon Keyspace

Open
#3,390 5 comments 0 reactions 0 assignees View on GitHub
backend/awskeyspace kind/bug/confirmed
Dominant language
Java
Stars
5.8k
Forks
1.2k
Avg merge
13h 53m
Merged PRs (30d)
6

Description

Unfortunately, Amazon Keyspace diverges from Apache Cassandra when it comes to paging.

Due to the way Amazon Keyspace works, even if a page iterator returns empty results, there still can be a valid next page with data. This happens when `ALLOW FILTERING` is used. In JanusGraph, this means when a full scan is triggered. We should fix CQLPagingIterator in CQLKeyColumnValueStore.java.

At the moment, I am not 100% sure whether this would also require a fix from the DataStax java driver which implements paging internally as well.

C.C. @porunov since you did quite a bit of the CQLPagingIterator implementation.

References:
https://stackoverflow.com/questions/69940435/amazon-keyspaces-allow-filtering-without-all-keys-in-node-js
https://docs.aws.amazon.com/keyspaces/latest/devguide/working-with-queries.html#paginating-results
https://github.com/JanusGraph/janusgraph/discussions/3378#discussioncomment-4405659

Contributor guide

Open the contributing guide

Research direction

Start in CQLKeyColumnValueStore.java by reading CQLPagingIterator and compare its paging behavior with the Amazon Keyspaces documentation and linked discussion. Determine whether the DataStax Java driver also needs changes. Done means full scans using ALLOW FILTERING continue past empty pages and retrieve valid data from later pages.

Written by the indexing model from the issue text.

Assessment

Tech stack
aws, cassandra, java
Domain
databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.