lance-format / lance-format/lance
feat(java): expose mergeInsertUncommitted (execute_uncommitted) in Java SDK
Nobody has claimed this yet.
- Dominant language
- Rust
- Stars
- 7.1k
- Forks
- 852
- Avg merge
- 3d 18h
- Merged PRs (30d)
- 272
Description
Problem
The Java SDK's Dataset.mergeInsert() always commits immediately. There is no way to perform a merge-insert and hold the resulting Transaction for manual commit via CommitBuilder.
Rust core already supports this via MergeInsertJob::execute_uncommitted() and MergeInsertJob::execute_uncommitted_batches(), and the Python SDK already exposes MergeInsertBuilder.execute_uncommitted() and execute_uncommitted_batches().
Use Case
Callers need to inspect or mutate the uncommitted Transaction (e.g., add extra operations, validate before commit) before finalizing, rather than having the merge-insert auto-commit.
Current State
| Layer | mergeInsert (auto-commit) |
mergeInsertUncommitted |
|---|---|---|
| Rust core | Yes | Yes |
| Python SDK | Yes | Yes |
| Java SDK | Yes | No |
What's Missing
All the building blocks are already in place — CommitBuilder, Transaction, MergeInsertParams, MergeInsertStats all exist in Java. The gap is small:
-
JNI (
java/lance-jni/src/merge_insert.rs): New JNI function callingexecute_uncommittedinstead ofexecute_reader, returning(Transaction, MergeStats)across the JNI boundary. -
Java (
java/src/main/java/org/lance/):- New
UncommittedMergeInsertResultPOJO holdingDataset,Transaction,MergeInsertStats - New
Dataset.mergeInsertUncommitted(MergeInsertParams, ArrowArrayStream)method - Corresponding
nativedeclaration
- New
Estimated total: ~150 lines of binding code. No new Rust core logic needed.
Proposed API
UncommittedMergeInsertResult result = dataset.mergeInsertUncommitted(
new MergeInsertParams(List.of("id"))
.withMatchedUpdateAll()
.withNotMatched(WhenNotMatched.InsertAll),
sourceStream
);
// Inspect / modify transaction before commit
Transaction txn = result.transaction();
Dataset committed = new CommitBuilder(result.dataset()).execute(txn);
References
- Rust:
rust/lance/src/dataset/write/merge_insert.rs—execute_uncommitted()/execute_uncommitted_batches() - Python binding:
python/src/dataset.rs—MergeInsertBuilder::execute_uncommitted()(line 468) - Existing Java
mergeInsert:java/src/main/java/org/lance/Dataset.java(line 2154) - Existing Java JNI:
java/lance-jni/src/merge_insert.rs
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with java/src/main/java/org/lance/Dataset.java and java/lance-jni/src/merge_insert.rs, then compare the Rust execute_uncommitted APIs with the Python binding in python/src/dataset.rs. Done means the Java SDK exposes the uncommitted result, its native declaration, and JNI support for returning the Transaction and merge statistics so callers can commit manually.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java, rust
- Domain
- api
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Clearly specified
- Newbie friendliness
- 63/100