tektoncd / tektoncd/pipelines-as-code

watcher crash-loops on a malformed execution-order annotation

Open
#2,945 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Go
Stars
214
Forks
144
Avg merge
2d 11h
Merged PRs (30d)
27

Description

📝 Description

Every PipelineRun managed by Pipelines-as-Code carries an annotation named
pipelinesascode.tekton.dev/execution-order. It holds a comma-separated list
of namespace/name entries. The watcher reads that list in
FilterPipelineRunByState (pkg/queue/queue_manager.go:185) and immediately
indexes both halves of each entry without checking that the split produced two
parts.

Any entry without a slash makes the watcher panic with index out of range.
An empty string is enough, since splitting "" on a comma yields one empty
entry. The knative worker loop has no recover, so the whole process dies.

The worst path is startup. InitQueues parses the same annotation inside
NewController, before any worker runs. The watcher restarts, reads the same
broken annotation, and dies again. That is a permanent CrashLoopBackOff for
the entire cluster, and it only takes one PipelineRun with a bad annotation.

The validating webhook only covers Repository resources, so anyone who can
edit a PipelineRun in one namespace can take the watcher down for everyone.

This was found during a post-merge review of #2890. That PR added the missing
length checks further down the line (pkg/reconciler/queue_pipelineruns.go:87,
pkg/reconciler/reconciler.go:459), but the code panics before reaching them.

🔁 How to reproduce

  1. Set concurrency_limit on a Repository.
  2. Edit any PipelineRun in that namespace and set
    pipelinesascode.tekton.dev/execution-order: garbage.
  3. Restart the watcher and watch it crash-loop.

🛠️ Suggested fix

Use strings.Cut and skip entries that do not have a non-empty namespace and
name. pkg/reconciler/finalizer.go:80 splits and indexes the same way and
should get the same guard.

🧪 Testing Strategy

  • Unit tests: malformed entries (empty, no slash, extra slash) mixed with a valid one

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with FilterPipelineRunByState in pkg/queue/queue_manager.go:185 and the InitQueues path in NewController, then inspect the related parsing in pkg/reconciler/finalizer.go:80. Run the relevant queue and reconciler unit tests, covering malformed entries mixed with a valid one; done means malformed annotations no longer crash the watcher or startup and valid entries still work.

Written by the indexing model from the issue text.

Assessment

Tech stack
go, kubernetes
Domain
ci-cd
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Active
Clarity
Clearly specified
Newbie friendliness
78/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.