Mutatingwebhook seems to add quotes for resources.*.cpu that leads to constant diffs in ArgoCD

Checks

• I've already read https://github.com/actions/actions-runner-controller/blob/master/TROUBLESHOOTING.md and I'm sure my issue is not covered in the troubleshooting guide.
• I'm not using a custom entrypoint in my runner image

Controller Version

v0.27.5

Helm Chart Version

No response

CertManager Version

No response

Deployment Method

ArgoCD

cert-manager installation

cert-manager is not involved in this problem.

Checks

• This isn't a question or user support case (For Q&A and community support, go to Discussions. It might also be a good idea to contract with any of contributors and maintainers if your business is so critical and therefore you need priority support
• I've read releasenotes before submitting this issue and I'm sure it's not due to any recently-introduced backward-incompatible changes
• My actions-runner-controller version (v0.x.y) does support the feature
• I've already upgraded ARC (including the CRDs, see charts/actions-runner-controller/docs/UPGRADING.md for details) to the latest and it didn't fix the issue
• I've migrated to the workflow job webhook event (if you using webhook driven scaling)

Resource Definitions

apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerDeployment
metadata:
  labels:
    argocd.argoproj.io/instance: xxx
  name: xxx
spec:
  replicas: 1
  template:
    metadata: {}
    spec:
      dockerdContainerResources: {}
      dockerdWithinRunnerContainer: true
      image: xxx
      labels:
      - self-hosted-xxx
      organization: xxx
      resources:
        limits:
          cpu: 4
          memory: 8Gi
        requests:
          cpu: 3
          memory: 7Gi
      securityContext:
        fsGroup: 65534
      serviceAccountName: xxx
      terminationGracePeriodSeconds: 110
---
apiVersion: actions.summerwind.dev/v1alpha1
kind: RunnerSet
metadata:
  labels:
    argocd.argoproj.io/instance: xxx
  name: xxx
spec:
  dockerdWithinRunnerContainer: true
  labels:
  - self-hosted-xxx
  organization: xxx
  replicas: 1
  selector:
    matchLabels:
      app.kubernetes.io/instance: xxx
  serviceName: xxx
  template:
    metadata:
      labels:
        app.kubernetes.io/instance: xxx
    spec:
      containers:
      - image: xxx
        name: runner
        resources:
          limits:
            cpu: 8
            memory: 16Gi
          requests:
            cpu: 7
            memory: 15Gi
      restartPolicy: never
      securityContext:
        fsGroup: 65534
      terminationGracePeriodSeconds: 110

To Reproduce

1. Create RunnerDeployment (with or without quotes for resources.*.cpu) or add quotes to resources.*.cpu to existing RunnerDeployment
2. See log entries in controller that the object is being mutated

controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	DEBUG	controller-runtime.webhook.webhooks	received request	{"webhook": "/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment", "UID": "6483c1bf-3799-4f99-8988-c997d4627701", "kind": "actions.summerwind.dev/v1alpha1, Kind=RunnerDeployment", "resource": {"group":"actions.summerwind.dev","version":"v1alpha1","resource":"runnerdeployments"}}
controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	DEBUG	controller-runtime.webhook.webhooks	wrote response	{"webhook": "/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment", "code": 200, "reason": "", "UID": "6483c1bf-3799-4f99-8988-c997d4627701", "allowed": true}
controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	DEBUG	controller-runtime.webhook.webhooks	received request	{"webhook": "/validate-actions-summerwind-dev-v1alpha1-runnerdeployment", "UID": "7a53bfb6-5a07-4c23-846c-859fc0981173", "kind": "actions.summerwind.dev/v1alpha1, Kind=RunnerDeployment", "resource": {"group":"actions.summerwind.dev","version":"v1alpha1","resource":"runnerdeployments"}}
controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	INFO	runnerdeployment-resource	validate resource to be updated	{"name": "gha-anaconda-arm64-m-priv-on-demand-gh-actions-runner"}

3. See that the object has changed and quotes are always added for cpu

      resources:
        limits:
          cpu: "4"
          memory: 8Gi
        requests:
          cpu: "3"
          memory: 7Gi

If you do the same for a RunnerSet, the controller-webhook won't add the quotes to the cpus field.

Describe the bug

We are managing our RunnerDeployments and RunnerSets with a basic helm chart.

This helm chart contains:

      resources:
        {{- toYaml .Values.resources | nindent 8 }}

Since toYaml always removes quotes (see https://github.com/helm/helm/issues/4262) the manifest always renders without quotes. Using toYaml for resources is best practice and also comes when creating a new helm template via helm create templatename and using any other mechanism or code in this place would be very very workaround-ish.

When using this helm chart in ArgoCD, ArgoCD always tries to enforce this rendered manifest without the quotes. For all RunnerDeployment applications this shows a constant diff. It works fine for RunnerSets.

Describe the expected behavior

I would expect the mutating webhook to not add quotes to the cpu field for RunnerDeployments (just like it doesn't for RunnerSets).

Whole Controller Logs

controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	DEBUG	controller-runtime.webhook.webhooks	received request	{"webhook": "/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment", "UID": "6483c1bf-3799-4f99-8988-c997d4627701", "kind": "actions.summerwind.dev/v1alpha1, Kind=RunnerDeployment", "resource": {"group":"actions.summerwind.dev","version":"v1alpha1","resource":"runnerdeployments"}}
controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	DEBUG	controller-runtime.webhook.webhooks	wrote response	{"webhook": "/mutate-actions-summerwind-dev-v1alpha1-runnerdeployment", "code": 200, "reason": "", "UID": "6483c1bf-3799-4f99-8988-c997d4627701", "allowed": true}
controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	DEBUG	controller-runtime.webhook.webhooks	received request	{"webhook": "/validate-actions-summerwind-dev-v1alpha1-runnerdeployment", "UID": "7a53bfb6-5a07-4c23-846c-859fc0981173", "kind": "actions.summerwind.dev/v1alpha1, Kind=RunnerDeployment", "resource": {"group":"actions.summerwind.dev","version":"v1alpha1","resource":"runnerdeployments"}}
controller-anaconda-distribution-7d5d88d769-7528t manager 2023-09-07T08:31:23Z	INFO	runnerdeployment-resource	validate resource to be updated	{"name": "gha-anaconda-arm64-m-priv-on-demand-gh-actions-runner"}

Start at the /mutate-actions-summerwind-dev-v1alpha1-runnerdeployment webhook entry point and compare its handling with RunnerSet, using the supplied RunnerDeployment and RunnerSet resource definitions. Reproduce with the Helm toYaml-rendered resources and verify that repeated ArgoCD reconciliation no longer changes the RunnerDeployment CPU representation or produces a constant diff.