Azure / Azure/hpcpack

HPC queue cleanup not working correctly

Open
#68 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
PowerShell
Stars
34
Forks
19
Avg merge
41m
Merged PRs (30d)
4

Description

#### Problem Description

We had a job with a large number of tasks added some time ago. This caused the job cleanup to fail and the queue to build up.

We debugged the issue in the underlying database. All queries in HpcScheduler.dbo.sp_DeleteOldJobs stored procedure are executed as expected up to "delete from TaskGroup..." which takes a very long time to execute. Our DBA tracked it back to a constraint on one of the tables that was looking up TaskGroupDependency. There is a missing index on the TaskGroupDependency table - see below.

USE [HPCScheduler]
GO

/****** Object: Index [IX_TaskGroupDependency] Script Date: 20/09/2025 13:33:23 ******/
CREATE NONCLUSTERED INDEX [IX_TaskGroupDependency] ON [dbo].[TaskGroupDependency]
(
[ParentId] ASC
)WITH (PAD_INDEX = OFF, STATISTICS_NORECOMPUTE = OFF, SORT_IN_TEMPDB = OFF, DROP_EXISTING = OFF, ONLINE = OFF, ALLOW_ROW_LOCKS = ON, ALLOW_PAGE_LOCKS = ON, OPTIMIZE_FOR_SEQUENTIAL_KEY = OFF) ON [PRIMARY]
GO

This change also significantly improves the performance when deleting all jobs from the queue.

#### Steps to Reproduce

Add job with large number of tasks with a lot of interdependency.

#### Expected Results

Job is removed when queue cleanup starts

#### Actual Results

Job is not removed (nor are any other jobs in HPC Pack 2016)

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.