[BUG] Chunk cleanup logic should make sure RMD CF data is deleted for AAWC
Nobody has claimed this yet.
- Dominant language
- Java
- Stars
- 611
- Forks
- 124
- Avg merge
- 3d 1h
- Merged PRs (30d)
- 26
Description
### Willingness to contribute
No. I cannot contribute a bug fix at this time.
### Venice version
0.4.139
### System information
- **OS Platform and Distribution (e.g., Linux Ubuntu 20.0)**: Mariner 5.15.111.1-1.cm2
- **JDK version**: 17
### Describe the problem
When the server resolves to clean up a chunked record, it calls the ReplicationMetadataRocksDBStoragePartition#deleteWithReplicationMetadata API. Unfortunately, it will not actually delete the RMD CF, but instead leave 4 bytes garbage behind for the value schema ID in that CF. A fix should make sure that the API properly cleans things up. **Note:** This only happens for those stores which have active/active enabled as this is the only mode which actually uses RMD's.
### Tracking information
_No response_
### Code to reproduce bug
_No response_
### What component(s) does this bug affect?
- [ ] `Controller`: This is the control-plane for Venice. Used to create/update/query stores and their metadata.
- [ ] `Router`: This is the stateless query-routing layer for serving read requests.
- [X] `Server`: This is the component that persists all the store data.
- [ ] `VenicePushJob`: This is the component that pushes derived data from Hadoop to Venice backend.
- [ ] `VenicePulsarSink`: This is a Sink connector for Apache Pulsar that pushes data from Pulsar into Venice.
- [ ] `Thin Client`: This is a stateless client users use to query Venice Router for reading store data.
- [ ] `Fast Client`: This is a stateful client users use to query Venice Server for reading store data.
- [ ] `Da Vinci Client`: This is an embedded, stateful client that materializes store data locally.
- [ ] `Alpini`: This is the framework that fast-client and routers use to route requests to the storage nodes that have the data.
- [ ] `Samza`: This is the library users use to make nearline updates to store data.
- [ ] `Admin Tool`: This is the stand-alone client used for ad-hoc operations on Venice.
- [ ] `Scripts`: These are the various ops scripts in the repo.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by tracing the server's chunk cleanup path to ReplicationMetadataRocksDBStoragePartition#deleteWithReplicationMetadata and inspect how the RMD column family is handled. Confirm that cleanup removes the RMD data rather than leaving the value schema ID bytes, including for active/active stores; the payload names no test file or reproduction case.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 25/100