dotnet / dotnet/MQTTnet

IManagedMqttClientStorage passed all message after each add/remove

Open
#1,157 0 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
C#
Stars
5.1k
Forks
1.2k
PR merge metrics
No merged PRs in 30d

Description

### Describe the bug

`IManagedMqttClientStorage.SaveAsync` is invoked for every add/remove from the collection. This results in a lot of unneeded serialization. Especially when there are a lot of messages. A better approach would be if the API would allow for partial updates.

For now I've made a smarter but simple storage implementation to reduce IO by keeping track of all message saved but with the drawback that each message is a file but that most messages easily fit within most cluster sizes of disks.

```c#
using System.Collections.Generic;
using System.IO;
using System.Linq;
using System.Text.Json;
using System.Threading.Tasks;
using MQTTnet.Extensions.ManagedClient;

public class ClientStorage : IManagedMqttClientStorage
{
readonly HashSet stored = new HashSet();

public async Task SaveQueuedMessagesAsync(IList messages)
{
foreach (var m in messages)
{
if (!stored.Contains(m))
{
var fn = $"message_{m.Id:N}.json";
await using var s = File.Create(fn);
await JsonSerializer.SerializeAsync(s, m);
stored.Add(m);
}
}

var deleted = stored.Except(messages);

foreach (var m in deleted)
{
var fn = $"message_{m.Id:N}.json";
File.Delete(fn);
stored.Remove(m);
}
}

public async Task> LoadQueuedMessagesAsync()
{
var result = new List();

var files = Directory.EnumerateFiles(".", "message_*.json");

foreach (var fn in files)
{
await using var s = File.OpenRead(fn);
var m = await JsonSerializer.DeserializeAsync(s);
if (m != null)
{
result.Add(m);
stored.Add(m);
}
}

return result;
}
}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.