apache / apache/fluss

[client] Write sender I/O thread crashes with "Accounted size went negative" due to Arrow memory double-release in ArrowWriter

Open
#2,892 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
2.1k
Forks
625
Avg merge
3d 14h
Merged PRs (30d)
97

Description

### Search before asking

- [x] I searched in the [issues](https://github.com/apache/fluss/issues) and found nothing similar.

### Fluss version

0.9.0 (latest release)

### Please describe the bug 🐞

The Fluss write sender I/O thread crashes with `java.lang.IllegalArgumentException: Accounted size went negative`, originating from Apache Arrow's internal memory accounting layer. The error surfaces during server upgrade (though the exact correlation is unclear).

**Full Stack Trace:**

```
2026-03-16 18:26:55,245 [fluss-write-sender-thread-1] ERROR org.apache.fluss.client.write.Sender [] - Uncaught error in Fluss write sender I/O thread:
java.lang.IllegalArgumentException: Accounted size went negative.
at org.apache.fluss.shaded.arrow.org.apache.arrow.util.Preconditions.checkArgument(Preconditions.java:136)
at org.apache.fluss.shaded.arrow.org.apache.arrow.memory.Accountant.releaseBytes(Accountant.java:219)
at org.apache.fluss.shaded.arrow.org.apache.arrow.memory.RootAllocator.releaseBytes(RootAllocator.java:29)
at org.apache.fluss.shaded.arrow.org.apache.arrow.memory.AllocationManager.release(AllocationManager.java:152)
at org.apache.fluss.shaded.arrow.org.apache.arrow.memory.BufferLedger.decrement(BufferLedger.java:157)
at org.apache.fluss.shaded.arrow.org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:124)
at org.apache.fluss.shaded.arrow.org.apache.arrow.memory.BufferLedger.release(BufferLedger.java:104)
at org.apache.fluss.shaded.arrow.org.apache.arrow.vector.ipc.message.ArrowRecordBatch.close(ArrowRecordBatch.java:243)
at org.apache.fluss.row.arrow.ArrowWriter.serializeToOutputView(ArrowWriter.java:266)
at org.apache.fluss.record.MemoryLogRecordsArrowBuilder.build(MemoryLogRecordsArrowBuilder.java:145)
at org.apache.fluss.record.MemoryLogRecordsArrowBuilder.close(MemoryLogRecordsArrowBuilder.java:228)
at org.apache.fluss.client.write.ArrowLogWriteBatch.close(ArrowLogWriteBatch.java:100)
at org.apache.fluss.client.write.RecordAccumulator.drainBatchesForOneNode(RecordAccumulator.java:731)
at org.apache.fluss.client.write.RecordAccumulator.drain(RecordAccumulator.java:286)
at org.apache.fluss.client.write.Sender.sendWriteData(Sender.java:243)
at org.apache.fluss.client.write.Sender.runOnce(Sender.java:194)
at org.apache.fluss.client.write.Sender.run(Sender.java:164)
```

### Solution

_No response_

### Are you willing to submit a PR?

- [ ] I'm willing to submit a PR!

Contributor guide

No contributing guide indexed for this repository

Research direction

Start at org.apache.fluss.row.arrow.ArrowWriter.serializeToOutputView, especially the failing line 266, and trace buffer ownership through MemoryLogRecordsArrowBuilder.close and ArrowLogWriteBatch.close. Reproduce the client write path during a server upgrade if possible; done means the sender no longer crashes with Arrow's "Accounted size went negative" error.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
distributed-systems
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.