How to apply dynamic labels to workflow job pods

How to apply dynamic labels to workflow job pods

We are running ARC v0.13.0 on K8s in containerMode: Kubernetes. Can you please inform how can we apply the below key-value pairs as labels to the workflow job pod dynamically. Just an FYI - The variables can be accessed in the workflow job as $<variable-name>. Though the variables don't appear when we exec to the workflow job pod and run env

GITHUB_REPOSITORY
GITHUB_WORKFLOW
GITHUB_JOB
RUNNER_NAME
ACTIONS_RUNNER_POD_NAME

We would like to apply the below labels so we can group resource utilization metrics while presenting in Grafana for a specific actions workflow. We can run a RunnerScaleSet instance for each repo, but this might not be feasible if the GitHub org has a few hundred repos

Tried below

1. Tried applying dynamic labels through runner-container-hooks, but seems it works on applying static labels, as Kubernetes complains on rendering the dynamic labels. Please refer below:

• configmap

apiVersion: v1
kind: ConfigMap
metadata:
  name: workflow-job-hook-extension-cm
  namespace: arc-poc
data:
  template.yaml: |
    metadata:
      labels:
        github-workflow: "{{ .WorkflowName }}"
    spec:
      containers:
      - name: "$job"
        resources:
          requests:
            cpu: 1000m
            memory: 1Gi
          limits:
            cpu: 2000m
            memory: 2Gi

• Workflow job execution gives below error:
  Error: failed to create job pod: Pod "my-runners-workflow" is invalid: metadata.labels: Invalid value: "{{ .WorkflowName }}": a valid label must be an empty string or consist of alphanumeric characters, '-', '_' or '.', and must start and end with an alphanumeric character (e.g. 'MyValue', or 'my_value', or '12345', regex used for validation is '(([A-Za-z0-9][-A-Za-z0-9_.]*)?[A-Za-z0-9])?')

Additional context

Possible solutions

1. Update runner-container-hooks as per suggestions - Though not sure if its feasible as don't see the above variables as environment variables by running kubectl exec -it <pod> -n <namespace> -- env for runner and workflow job pods
2. Use K8s Admission Controllers as per suggestions - Though not sure if it's feasible as don't see these variables as labels or annotations in runner job pods

Start with the runner-container-hooks ADR 0134 linked in the issue and inspect the shown workflow-job-hook-extension ConfigMap. Determine whether workflow variables are available when the pod metadata is rendered and whether admission controllers can access them. Done means documenting a supported way to apply dynamic labels, or clearly documenting the limitation and an alternative.