apache / apache/cassandra-gocql-driver

Batch size in bytes

Open
#1,448 3 comments 4 reactions 0 assignees View on GitHub
enhancement
Dominant language
Go
Stars
2.7k
Forks
658
PR merge metrics
No merged PRs in 30d

Description

Is there any way to calculate the byte size of the batch while building it?

### What did you do?

Trying to batch same-partition updates to speed up data ingestion the way sstableloader does (and claim 5x improvement).

### What did you expect to see?

Not that I've expected it, but a function to calculate the size of the batch to avoid hitting `batch_size_fail_threshold_in_kb` would be very helpful.

### What did you see instead?

Only Java versions of it :)
[BatchStatement:: requestSizeInBytes](https://github.com/dbraley/datastax/blob/eace672d4532e5533ca516fc7dc9a051489c76d4/driver-core/src/main/java/com/datastax/driver/core/BatchStatement.java#L205)
[RegularStatement:: requestSizeInBytes](https://github.com/dbraley/datastax/blob/eace672d4532e5533ca516fc7dc9a051489c76d4/driver-core/src/main/java/com/datastax/driver/core/RegularStatement.java#L196)

### What version of Cassandra are you using?

3.11.4

### What version of Gocql are you using?

v0.0.0-20190523124812-0680bfb96414

Contributor guide

Open the contributing guide

Research direction

Start by comparing the linked Java BatchStatement::requestSizeInBytes and RegularStatement::requestSizeInBytes implementations, then locate the corresponding batch-building and request-serialization code in the Go driver. Done means providing a way to calculate a batch's byte size before sending it and covering the behavior with tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
cassandra, go
Domain
databases
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.