apache / apache/pulsar-dotpulsar
Message loss with message delivery not guaranteed
- Dominant language
- C#
- Stars
- 271
- Forks
- 87
- PR merge metrics
- No merged PRs in 30d
Description
### Description
I have a pulsar client created with a one producer created for a topic and ran a load tests and I could see that few of the messages are lost. In the current implementation is message delivery guaranteed with a publisher confirmation ?
Pulsar broker :
Current version of pulsar is: 4.1.0
Broker Config
[broker.conf.txt](https://github.com/user-attachments/files/22680110/broker.conf.txt)
Library Version: 4.3.2
**Client and Producer code**
builder = PulsarClient.Builder()
.Authentication(AuthenticationFactory.Token(TOKEN))
.CheckCertificateRevocation(true)
.ExceptionHandler(new CustomExceptionHandler())
.KeepAliveInterval(new TimeSpan(0, 0, 30))
.RetryInterval(new TimeSpan(0, 0, 2))
.CloseInactiveConnectionsInterval(new TimeSpan(0, 0, 30));
builder.ConnectionSecurity(EncryptionPolicy.EnforceEncrypted)
.ServiceUrl(new Uri($"pulsar+ssl://{pulsarHostName}:6651"))
.TrustedCertificateAuthority(Certificate.GenerateCertificate(certificateOption.Value.Pulsar.Ca))
.VerifyCertificateAuthority(true)
.VerifyCertificateName(false);
_client = builder.Build();
_producer = _client.NewProducer(Schema.ByteArray)
.ProducerName($"{@event}-{Environment.MachineName}")
.StateChangedHandler(producerStateChanged =>
{
var topicName = producerStateChanged.Producer.Topic;
var state = producerStateChanged.ProducerState;
_logger.LogDebug(new ApplicationLogMessage
{
Message = $"The producer for topic '{topicName}' changed state to '{state}'"
});
})
.Topic($"persistent://global/seed/Test)
.Create();
**Publisher code**
var messageId = await producer.Send(new MessageMetadata(), byteArray, CancellationToken.None).ConfigureAwait(false);
### Reproduction Steps
The publisher code is exposed via an http end point.
Run a load test
Number of users : 10
Number of iterations : 20
### Expected behavior
Expected: 200 Messages to be published.
### Actual behavior
There are few messages loss with EntryId and LedgerId returned as negative values
### Regression?
_No response_
### Known Workarounds
_No response_
### Configuration
_No response_
### Other information
_No response_
Contributor guide
Research direction
Start with the publisher's producer.Send call and the attached broker.conf.txt; reproduce the 10-user, 20-iteration load test and inspect returned MessageId EntryId and LedgerId values. Compare publisher confirmations with the broker configuration and determine whether all 200 sends are acknowledged; done means the loss or negative-ID behavior is explained and covered by an appropriate regression check.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- distributed-systems
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100