HangfireIO / HangfireIO/Hangfire
Violation of PRIMARY KEY constraint 'PK_HangFire_Set'. Cannot insert duplicate key in object 'HangFire.Set'. when UseIgnoreDupKeyOption is set to true
- Dominant language
- C#
- Stars
- 10.1k
- Forks
- 1.8k
- Avg merge
- 1h 19m
- Merged PRs (30d)
- 1
Description
I'm receiving this error whenever I try to trigger a job that's registered as a recurring job manually through the dashboard Recurring Jobs section by checking the job to trigger, and clicking the `Trigger now` button.
This only occurs when the `UseIgnoreDupKeyOption` setting is set to `true`
The error displayed in the console is:
```
Microsoft.Data.SqlClient.SqlException (0x80131904): Violation of PRIMARY KEY constraint 'PK_HangFire_Set'. Cannot insert duplicate key in object 'HangFire.Set'. The duplicate key value is (recurring-jobs, LocalizedStringIngestion).
at Microsoft.Data.SqlClient.SqlConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.SqlInternalConnection.OnError(SqlException exception, Boolean breakConnection, Action`1 wrapCloseInAction)
at Microsoft.Data.SqlClient.TdsParser.ThrowExceptionAndWarning(TdsParserStateObject stateObj, Boolean callerHasConnectionLock, Boolean asyncClose)
at Microsoft.Data.SqlClient.TdsParser.TryRun(RunBehavior runBehavior, SqlCommand cmdHandler, SqlDataReader dataStream, BulkCopySimpleResultSet bulkCopyHandler, TdsParserStateObject stateObj, Boolean& dataReady)
at Microsoft.Data.SqlClient.SqlCommand.FinishExecuteReader(SqlDataReader ds, RunBehavior runBehavior, String resetOptionsString, Boolean isInternal, Boolean forDescribeParameterEncryption, Boolean shouldCacheForAlwaysEncrypted)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReaderTds(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, Boolean isAsync, Int32 timeout, Task& task, Boolean asyncWrite, Boolean inRetry, SqlDataReader ds, Boolean describeParameterEncryptionRequest)
at Microsoft.Data.SqlClient.SqlCommand.RunExecuteReader(CommandBehavior cmdBehavior, RunBehavior runBehavior, Boolean returnStream, TaskCompletionSource`1 completion, Int32 timeout, Task& task, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String method)
at Microsoft.Data.SqlClient.SqlCommand.InternalExecuteNonQuery(TaskCompletionSource`1 completion, Boolean sendToPipe, Int32 timeout, Boolean& usedCache, Boolean asyncWrite, Boolean inRetry, String methodName)
at Microsoft.Data.SqlClient.SqlCommand.ExecuteNonQuery()
at Hangfire.SqlServer.SqlCommandBatch.ExecuteNonQuery()
at Hangfire.SqlServer.SqlServerWriteOnlyTransaction.b__12_0(DbConnection connection, DbTransaction transaction)
at Hangfire.SqlServer.SqlServerStorage.<>c__DisplayClass34_0.b__0(DbConnection connection, DbTransaction transaction)
at Hangfire.SqlServer.SqlServerStorage.<>c__DisplayClass35_0`1.b__0(DbConnection connection)
at Hangfire.SqlServer.SqlServerStorage.UseConnection[T](DbConnection dedicatedConnection, Func`2 func)
at Hangfire.SqlServer.SqlServerStorage.UseTransaction[T](DbConnection dedicatedConnection, Func`3 func, Nullable`1 isolationLevel)
at Hangfire.SqlServer.SqlServerStorage.UseTransaction(DbConnection dedicatedConnection, Action`2 action)
at Hangfire.SqlServer.SqlServerWriteOnlyTransaction.Commit()
at Hangfire.RecurringJobManager.Trigger(String recurringJobId)
at Hangfire.Dashboard.DashboardRoutes.<>c.<.cctor>b__2_27(IRecurringJobManager manager, String jobId)
at Hangfire.Dashboard.RouteCollectionExtensions.<>c__DisplayClass4_0.b__0(DashboardContext context, String jobId)
at Hangfire.Dashboard.BatchCommandDispatcher.Dispatch(DashboardContext context)
at Hangfire.Dashboard.AspNetCoreDashboardMiddleware.Invoke(HttpContext httpContext)
at Microsoft.AspNetCore.Builder.Extensions.MapMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Builder.Extensions.MapWhenMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authorization.AuthorizationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Authentication.AuthenticationMiddleware.Invoke(HttpContext context)
at Microsoft.AspNetCore.Diagnostics.DeveloperExceptionPageMiddleware.Invoke(HttpContext context)
ClientConnectionId:16480047-e279-4d7a-9258-ff0a1636a188
Error Number:2627,State:1,Class:14
```
The project is an ASP.net core 5 project
In ConfigureServices:
```cs
services.AddHangfire(c =>
{
c.UseSqlServerStorage(() =>
{
var csb = new SqlConnectionStringBuilder(connectionString);
var connection = new SqlConnection(csb.ConnectionString);
if (csb["Data Source"].ToString().Contains("database.windows.net"))
{
connection.AccessToken = AppTokenProvider.GetAccessTokenAsync("https://database.windows.net/").GetAwaiter().GetResult();
}
return connection;
},
new SqlServerStorageOptions
{
EnableHeavyMigrations = true,
QueuePollInterval = TimeSpan.FromSeconds(30),
UseIgnoreDupKeyOption = true,
CommandBatchMaxTimeout = TimeSpan.FromMinutes(5),
DisableGlobalLocks = true,
UseFineGrainedLocks = true,
CommandTimeout = TimeSpan.FromMinutes(5),
SlidingInvisibilityTimeout = TimeSpan.FromMinutes(5),
UseRecommendedIsolationLevel = true
});
});
services.AddHangfireServer(o =>
{
o.WorkerCount = 1;
});
services.AddScoped();
```
in Configfure
```cs
app.UseHangfireDashboard("/ingestion", new DashboardOptions
{
IsReadOnlyFunc = (context) => !context.GetHttpContext().User.IsInRole("admin"),
Authorization = new[] { new HangfireAuthFilter() },
AppPath = "/admin",
});
RecurringJob.RemoveIfExists(nameof(TitlesWithPresenceIngestion));
RecurringJob.AddOrUpdate(nameof(LocalizedStringIngestion), a => a.Execute(CancellationToken.None), localizedStringPattern);
```
the key/value called out in the exception message are indeed in the Set table
Contributor guide
Research direction
Start with RecurringJobManager.Trigger and SqlServerWriteOnlyTransaction.Commit in the stack trace, then inspect the UseIgnoreDupKeyOption configuration and the HangFire.Set operation. Reproduce the dashboard “Trigger now” flow with the option enabled; done means the recurring job triggers without a duplicate-primary-key exception and the behavior is covered by a regression test.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- csharp, sql
- Domain
- backend, databases
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 42/100