coschedule isolate-pipelinerun doesn't work
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 608
- Forks
- 263
- Avg merge
- 1d 3h
- Merged PRs (30d)
- 125
Description
Expected Behavior
I have configured my tekton operator with:
pipeline:
disable-affinity-assistant: true
coschedule: isolate-pipelinerun
enable-api-fields: "alpha"
I'm using the autoscaler on my node group on aws eks is using a ASG multi AZ. My trigger template has this configuration:
(...)
podTemplate:
securityContext:
fsGroup: 65532
affinity:
nodeAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
nodeSelectorTerms:
- matchExpressions:
- key: pipelines
operator: In
values:
- "pipelines"
podAntiAffinity:
requiredDuringSchedulingIgnoredDuringExecution:
- labelSelector:
matchExpressions:
- key: app.kubernetes.io/component
operator: In
values:
- "affinity-assistant"
topologyKey: "kubernetes.io/hostname" # Add the topologyKey here
nodeSelector:
pipelines: tom-pipelines
tolerations:
- key: dedicated
operator: Equal
value: pipelines
effect: NoSchedule
(...)
Actual Behavior
My pipelinerun has PVC and if I understand correctly the documentation, when we use coschedule isolate-pipelinerun, each pipelinerun should run in a different physical node, but it doesn't happen and retun this error:
pod status "PodScheduled":"False"; message: "0/7 nodes are available: 1 node(s) didn''t match pod anti-affinity rules, 1 node(s) had untolerated taint {app: permanentpod}, 2 node(s) had untolerated taint {app: 24h}, 3 node(s) had untolerated taint {eks.amazonaws.com/compute-type: fargate}. preemption: 0/7 nodes are available: 1 No preemption victims found for incoming pod, 6 Preemption is not helpful for scheduling."
Steps to Reproduce the Problem
Additional Info
-
Kubernetes version: 1.30
Output of
kubectl version:
Client Version: version.Info{Major:"1", Minor:"27+", GitVersion:"v1.27.4-eks-8ccc7ba", GitCommit:"892db4a4e439987d7addade5f9595cadfa06db2e", GitTreeState:"clean", BuildDate:"2023-08-15T16:06:56Z", GoVersion:"go1.20.6", Compiler:"gc", Platform:"linux/amd64"}
Kustomize Version: v5.0.1
Server Version: version.Info{Major:"1", Minor:"30+", GitVersion:"v1.30.3-eks-2f46c53", GitCommit:"69ba22bf73c1112e7933fc61b220c00b554a7f66", GitTreeState:"clean", BuildDate:"2024-07-25T04:23:44Z", GoVersion:"go1.22.5", Compiler:"gc", Platform:"linux/amd64"}
WARNING: version difference between client (1.27) and server (1.30) exceeds the supported minor version skew of +/-1
-
Tekton Pipeline version: latest-v
Output of
tkn versionorkubectl get pods -n tekton-pipelines -l app=tekton-pipelines-controller -o=jsonpath='{.items[0].metadata.labels.version}'
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the reported Tekton configuration, including coschedule: isolate-pipelinerun, the PVC, node affinity, pod anti-affinity, taints, and tolerations. Use the reported PodScheduled=False event to determine whether the scheduling behavior or the expectation about separate physical nodes is incorrect; the issue is complete when the cause and a verified resolution are documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go, kubernetes
- Domain
- devops, infrastructure
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100