HangfireIO / HangfireIO/Hangfire
Limit retention time (shorter instead of longer)
- Dominant language
- C#
- Stars
- 10.1k
- Forks
- 1.8k
- Avg merge
- 1h 19m
- Merged PRs (30d)
- 1
Description
The current project i'm working on has to process a lot of SQL tasks. This works by dumping a task to the queue every couple of milliseconds (100), when the task is done, it returns to an internal queue. So the tasks rotate 24/7 in memory.
The problem here is that this creates a lot of "waste" entries. After around 2 minutes, a couple of thousand of records are created. Standard retention is 1 day, so we are speaking of a couple million of records every day. If these records succeeded, they shouldn't be kept that long, in fact, if they would be cleaned up immediately, that would be fine too.
I tried using a custom filter:
```
public class TestExpireAttribute : JobFilterAttribute, IApplyStateFilter
{
public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
context.JobExpirationTimeout = TimeSpan.FromSeconds(1);
}
public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
context.JobExpirationTimeout = TimeSpan.FromSeconds(1);
}
}
```
Note the 1 second is a test.
But this seems to only remove those entries on startup. Is there a way to keep the entries only for the past couple of minutes and directly remove them after a few minutes of being completed?
Contributor guide
Research direction
Start by tracing the IApplyStateFilter implementation and how JobExpirationTimeout is applied to completed entries; the issue's TestExpireAttribute provides a reproduction. Verify behavior with a short expiration interval and consider the existing startup cleanup path. Done means completed entries are removed after the configured interval without requiring a restart.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp
- Domain
- backend
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100