apache / apache/pekko-persistence-cassandra

Reverse order query causes performance bottleneck of cassandra cluster

Open
#240 4 comments 0 reactions 0 assignees View on GitHub
Dominant language
Scala
Stars
15
Forks
17
Avg merge
6h 43m
Merged PRs (30d)
11

Description

CassandraJournalStatements.scala
```scala
def selectHighestSequenceNr =
s"""
SELECT sequence_nr FROM $tableName WHERE
persistence_id = ? AND
partition_nr = ?
ORDER BY sequence_nr
DESC LIMIT 1
"""
```

journal messages table of cassandra below:
```sql
CREATE TABLE IF NOT EXISTS pekko.messages (
persistence_id text,
partition_nr bigint,
sequence_nr bigint,
timestamp timeuuid,
timebucket text,
writer_uuid text,
ser_id int,
ser_manifest text,
event_manifest text,
event blob,
meta_ser_id int,
meta_ser_manifest text,
meta blob,
tags set,
PRIMARY KEY ((persistence_id, partition_nr), sequence_nr, timestamp)) --- sequence_nr default is asc order
WITH gc_grace_seconds =864000
AND compaction = {
'class' : 'SizeTieredCompactionStrategy',
'enabled' : true,
'tombstone_compaction_interval' : 86400,
'tombstone_threshold' : 0.2,
'unchecked_tombstone_compaction' : false,
'bucket_high' : 1.5,
'bucket_low' : 0.5,
'max_threshold' : 32,
'min_threshold' : 4,
'min_sstable_size' : 50
};

```

When business application actors are restored from cassandra disks, there will be a large number of slow queries. Is there any way to eliminate this reverse order query? Thank you!

Contributor guide

Open the contributing guide

Research direction

Start with CassandraJournalStatements.scala and the messages table schema shown in the issue. Investigate how selecting the highest sequence number during actor restoration interacts with Cassandra clustering order, then compare a viable query or schema approach against the current reverse-order query. Done means restoring actors without the reported cluster performance bottleneck while still selecting the highest sequence number correctly.

Written by the indexing model from the issue text.

Assessment

Tech stack
cassandra, scala
Domain
database, performance
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.