HangfireIO / HangfireIO/Hangfire

Violation of PRIMARY KEY constraint 'PK_HangFire_Set'. Cannot insert duplicate key in object 'HangFire.Set'. when UseIgnoreDupKeyOption is set to true

Open
#1,773 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C#
Stars
10.1k
Forks
1.8k
Avg merge
1h 19m
Merged PRs (30d)
1

Description

I'm receiving this error whenever I try to trigger a job that's registered as a recurring job manually through the dashboard Recurring Jobs section by checking the job to trigger, and clicking the `Trigger now` button.

This only occurs when the `UseIgnoreDupKeyOption` setting is set to `true`

The error displayed in the console is:
```
Microsoft.Data.SqlClient.SqlException (0x80131904): Violation of PRIMARY KEY constraint 'PK_HangFire_Set'. Cannot insert duplicate key in object 'HangFire.Set'. The duplicate key value is (recurring-jobs, LocalizedStringIngestion).
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)
at Microsoft.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String methodName)
at Microsoft.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Hangfire.SqlServer.SqlCommandBatch.ExecuteNonQuery()
at Hangfire.SqlServer.SqlServerWriteOnlyTransaction.b__12_0(DbConnection connection, DbTransaction transaction)
at Hangfire.SqlServer.SqlServerStorage.<>c__DisplayClass34_0.b__0(DbConnection connection, DbTransaction transaction)
at Hangfire.SqlServer.SqlServerStorage.<>c__DisplayClass35_0`1.b__0(DbConnection connection)
at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)
at Hangfire.SqlServer.SqlServerStorage.UseTransaction[T](DbConnection dedicatedConnection, Func`3 func, Nullable`1 isolationLevel)
at Hangfire.SqlServer.SqlServerStorage.UseTransaction(DbConnection dedicatedConnection, Action`2 action)
at Hangfire.SqlServer.SqlServerWriteOnlyTransaction.Commit()
at Hangfire.RecurringJobManager.Trigger(String recurringJobId)
at Hangfire.Dashboard.DashboardRoutes.<>c.<.cctor>b__2_27(IRecurringJobManager manager, String jobId)
at Hangfire.Dashboard.RouteCollectionExtensions.<>c__DisplayClass4_0.b__0(DashboardContext context, String jobId)
at Hangfire.Dashboard.BatchCommandDispatcher.Dispatch(DashboardContext context)
at Hangfire.Dashboard.AspNetCoreDashboardMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
ClientConnectionId:16480047-e279-4d7a-9258-ff0a1636a188
Error Number:2627,State:1,Class:14
```

The project is an ASP.net core 5 project
In ConfigureServices:
```cs
services.AddHangfire(c =>
{
c.UseSqlServerStorage(() =>
{
var csb = new SqlConnectionStringBuilder(connectionString);
var connection = new SqlConnection(csb.ConnectionString);
if (csb["Data Source"].ToString().Contains("database.windows.net"))
{
connection.AccessToken = AppTokenProvider.GetAccessTokenAsync("https://database.windows.net/").GetAwaiter().GetResult();
}

return connection;
},
new SqlServerStorageOptions
{
EnableHeavyMigrations = true,
QueuePollInterval = TimeSpan.FromSeconds(30),
UseIgnoreDupKeyOption = true,
CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
DisableGlobalLocks = true,
UseFineGrainedLocks = true,
CommandTimeout = TimeSpan.FromMinutes(5),
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
UseRecommendedIsolationLevel = true
});
});
services.AddHangfireServer(o =>
{
o.WorkerCount = 1;
});
services.AddScoped();
```

in Configfure
```cs
app.UseHangfireDashboard("/ingestion", new DashboardOptions
{
IsReadOnlyFunc = (context) => !context.GetHttpContext().User.IsInRole("admin"),
Authorization = new[] { new HangfireAuthFilter() },
AppPath = "/admin",
});

RecurringJob.RemoveIfExists(nameof(TitlesWithPresenceIngestion));
RecurringJob.AddOrUpdate(nameof(LocalizedStringIngestion), a => a.Execute(CancellationToken.None), localizedStringPattern);
```
the key/value called out in the exception message are indeed in the Set table

Contributor guide

Open the contributing guide

Research direction

Start with RecurringJobManager.Trigger and SqlServerWriteOnlyTransaction.Commit in the stack trace, then inspect the UseIgnoreDupKeyOption configuration and the HangFire.Set operation. Reproduce the dashboard “Trigger now” flow with the option enabled; done means the recurring job triggers without a duplicate-primary-key exception and the behavior is covered by a regression test.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp, sql
Domain
backend, databases
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.