kubernetes-sigs / kubernetes-sigs/kernel-module-management

DRA cleanup can be skipped entirely because no finalizer holds the Module

Open
#1,331 5 comments 0 reactions 0 assignees View on GitHub
priority/critical-urgent
Dominant language
Go
Stars
130
Forks
44
Avg merge
5d 16h
Merged PRs (30d)
8

Description

DRA cleanup runs from the DRA reconciler, but nothing keeps the Module around long enough for it to finish. I noticed this while working on #1329, which adds a node label to the set of things that need cleaning up, so I am filing it separately.

The only finalizer on a Module is `constants.ModuleFinalizer`, added and removed by the Module reconciler. `finalizeModule` waits for the NMC in-use labels to go away and then removes it, without knowing anything about DRA. Once it is gone the API server deletes the object, and `reconcile.AsReconciler` gets NotFound the next time the DRA reconciler tries, so the deletion branch never runs.

That leaves behind:

- DeviceClasses, which are cluster-scoped. A namespaced Module cannot be a valid owner of a cluster-scoped object, so ownerReferences cannot cover this and the only cleanup path is the reconciler's own `DeleteAllOf`.
- `dra-target` node labels, once #1329 lands.
- Any device-plugin-target labels, which have the same shape today.

It is most likely for a Module with `spec.dra` but no `spec.moduleLoader`: there is no NMC usage to wait for, so the Module finalizer can be removed almost immediately.

A recreated Module with the same namespace and name then finds pre-labeled nodes, so a driver Pod can be scheduled before the new generation has evaluated whether those nodes are schedulable at all.

A DRA-specific finalizer, for example `kmm.sigs.x-k8s.io/dra-cleanup`, would let the reconciler delete the DaemonSets, wait for the Pods to be gone, delete the DeviceClasses, remove the target labels and clear the status before letting the Module go. Adding the Module UID to the managed resources would also stop a new generation from inheriting the previous one's leftovers.

Happy to work on this if it sounds right.

Contributor guide

Open the contributing guide

Research direction

Trace the DRA reconciler, the Module reconciler's finalizeModule path, and reconcile.AsReconciler behavior when the Module is deleted. Verify how DeleteAllOf handles DeviceClasses and how target labels and managed Pods are cleaned up. Done means a DRA-specific finalizer keeps the Module until DRA resources, Pods, DeviceClasses, labels, and status are cleaned up, while a new Module generation cannot inherit stale resources.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
devops, infrastructure
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.