Support for RecyclableMemoryManager, client write support for ReadOnlySequence as payload
- Dominant language
- C#
- Stars
- 5.1k
- Forks
- 1.2k
- PR merge metrics
- No merged PRs in 30d
Description
### Describe the feature request
In a streaming scenario for telemetry sending Megabytes per second to a MQTT Broker with JSON or compressed payload,
it is beneficial to encode into fixed size buffers in the processing pipeline to avoid memory fragmentation.
Using a normal MemoryStream or ToArray leads to memory fragmentation and specifically if the client is hosted in Kubernetes, it can cause eviction of the pod.
The solution is the `RecyclableMemoryStream` which can be used to encode the payload. After encoding the stream, it has a method to pass a `ReadOnlySequence` along the encoding pipeline which is valid until the memory stream is disposed. The ReadOnlySequence contains a list of buffers which contain the payload in chunks.
The usage is quite efficent because the buffers are recycled without allocations when the RecyclableMemoryStream is disposed.
However, the MQTTNet write method only accepts an `ArraySegment` which is a view into a single buffer. It is currently necessary to call `ToArray` which leads to a full size memory allocation and copy of the whole payload.
As a result, perfomance is affected by additional copies and memory is fragment by the additional random allocation of a big buffer.
In order to natively support `ReadOnlySequence`, `MqttPublishPacket` should support a `PayloadSequence` instead of a `PayloadSegment` in the MQTTnet client.
Turns out the support can be added with a few surgical changes. see #1918
### Which project is your feature request related to?
- Client
### Describe the solution you'd like
#1918 is a draft PR. Please provide the feedback if and how such an enhancement can be accepted by the project.
### Describe alternatives you've considered
A clear and concise description of any alternative solutions or features you've considered.
### Additional context
Add any other context or screenshots about the feature request here.
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.