HangfireIO / HangfireIO/Hangfire

Limit retention time (shorter instead of longer)

Open
#981 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
10.1k
Forks
1.8k
Avg merge
1h 19m
Merged PRs (30d)
1

Description

The current project i'm working on has to process a lot of SQL tasks. This works by dumping a task to the queue every couple of milliseconds (100), when the task is done, it returns to an internal queue. So the tasks rotate 24/7 in memory.

The problem here is that this creates a lot of "waste" entries. After around 2 minutes, a couple of thousand of records are created. Standard retention is 1 day, so we are speaking of a couple million of records every day. If these records succeeded, they shouldn't be kept that long, in fact, if they would be cleaned up immediately, that would be fine too.

I tried using a custom filter:

```
public class TestExpireAttribute : JobFilterAttribute, IApplyStateFilter
{
public void OnStateApplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
context.JobExpirationTimeout = TimeSpan.FromSeconds(1);
}

public void OnStateUnapplied(ApplyStateContext context, IWriteOnlyTransaction transaction)
{
context.JobExpirationTimeout = TimeSpan.FromSeconds(1);
}
}
```

Note the 1 second is a test.

But this seems to only remove those entries on startup. Is there a way to keep the entries only for the past couple of minutes and directly remove them after a few minutes of being completed?

Contributor guide

Open the contributing guide

Research direction

Start by tracing the IApplyStateFilter implementation and how JobExpirationTimeout is applied to completed entries; the issue's TestExpireAttribute provides a reproduction. Verify behavior with a short expiration interval and consider the existing startup cleanup path. Done means completed entries are removed after the configured interval without requiring a restart.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
backend
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.