Log tail iteration does not end when log completed
Nobody has claimed this yet.
- Dominant language
- C#
- Stars
- 6.6k
- Forks
- 595
- PR merge metrics
- No merged PRs in 30d
Description
The documentation seems to suggest (in the Log Completion section), that an iterator can check for log completion with the Ended property.
However when a log is completed, it seems that the iterator's WaitAsync method does not return during tail iteration in order for this property to be checked.
In this example code, the 'Scan complete' message is never sent to the console, and the scanning iterator waits forever after the log is completed.
using System.Diagnostics;
using FASTER.core;
using var settings = new FasterLogSettings("./Test", deleteDirOnDispose: true)
{
AutoRefreshSafeTailAddress = true
};
using var log = new FasterLog(settings);
byte[] buffer = new byte[2048];
Random.Shared.NextBytes(buffer);
await Task.WhenAll(EnqueueThread(), ScanThread());
return;
async Task EnqueueThread()
{
for (int count = 0; count < 5; ++count)
{
await log.EnqueueAsync(buffer);
await Task.Delay(1000);
}
log.CompleteLog();
Console.WriteLine("Enqueue complete");
}
async Task ScanThread()
{
using var iterator = log.Scan(log.BeginAddress, long.MaxValue, scanUncommitted: true);
while (true)
{
byte[] result;
while (!iterator.GetNext(out result, out _, out _))
{
if (iterator.Ended)
{
Console.WriteLine("Scan complete");
return;
}
await iterator.WaitAsync();
}
Console.WriteLine("Received buffer");
Debug.Assert(result.SequenceEqual(buffer));
}
}
Is the correct usage in this case that the iterator's WaitAsync method requires a CancellationToken passed in that is cancelled separately at the time the log is completed? The documentation and sample don't suggest this.
Another minor issue: the documentation also mentions the void RefreshUncommitted(bool spinWait = false) method, which is no longer part of the API, but instead seems to be configured in the FasterLogSettings (as in the code sample above).
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the FasterLog.Scan iterator APIs shown in the report, especially GetNext, Ended, WaitAsync, and CompleteLog, and compare their behavior with the Log Completion documentation. Reproduce the supplied example, then check whether completion allows the iterator to finish without cancellation. Also verify and update the documentation's RefreshUncommitted reference to match FasterLogSettings.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- databases, documentation
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 30/100