[Bug] Transactions timeout when messages exceed a certain size.
- Dominant language
- Java
- Stars
- 15.3k
- Forks
- 3.8k
- Avg merge
- 1d 14h
- Merged PRs (30d)
- 160
Description
### Search before asking
- [X] I searched in the [issues](https://github.com/apache/pulsar/issues) and found nothing similar.
### Version
Version: 2.11.x standalone, and 3.0
When executing the following code against a standalone cluster, a timeout exception is thrown when the message size is 140Kb.
```
public static void main(String[] args) throws PulsarClientException, InterruptedException, ExecutionException {
String serviceUrl = “pulsar://localhost:6650/”;
String topicName = “test-topic”;
String producerName = “test-producer”;
int producerTimout = 10;
PulsarClient pulsarClient = PulsarClient.builder()
.serviceUrl(serviceUrl)
.enableTransaction(true)
.build();
Producer partProducer = pulsarClient
.newProducer(Schema.AVRO(String.class))
.producerName(producerName)
.topic(topicName)
.sendTimeout(producerTimout, TimeUnit.SECONDS)
.enableChunking(false)
.enableBatching(true)
.create();
System.out.println(“Successfully created producer: ” + partProducer.getProducerName());
Transaction transaction = pulsarClient.newTransaction()
.withTransactionTimeout(2, TimeUnit.SECONDS)
.build()
.get();
System.out.println(“Pulsar transaction created with txnID: ” + transaction.getTxnID());
// The issue happens when message size great than 140 KB
// DEBUG org.apache.pulsar.client.impl.ProducerImpl
// -- [test-topic] [test-producer] Closing out batch to accommodate large message with size 140003
int messageSize = 140; // payload size in KB
String message = “a”.repeat(messageSize * 1000);
partProducer.newMessage(transaction).value(message).send();
transaction.commit();
System.out.println(“Pulsar transaction was committed with txnID: ” + transaction.getTxnID());
}
```
### Minimal reproduce step
You can just run the code above and an exception will be thrown.
### What did you expect to see?
"Pulsar transaction was committed with txnID: " in the console output
### What did you see instead?
Timeout exception
### Anything else?
If you add the following configuration setting to the producer, then the exception is not thrown
` .batchingMaxPublishDelay(1, TimeUnit.SECONDS)`
### Are you willing to submit a PR?
- [ ] I'm willing to submit a PR!
Contributor guide
Research direction
Reproduce the timeout against a standalone Pulsar cluster using the provided Java transaction and batching example, then start at the ProducerImpl batching path indicated by the debug output. Investigate why a message over 140 KB times out without batchingMaxPublishDelay, and consider the issue resolved when the transaction commits and prints the expected confirmation without that workaround.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100