VictoriaMetrics / VictoriaMetrics/operator
BUG: vmalert config-reloader fails to start when spec.configMaps is set (operator v0.74.1)
@AndrewChubatiuk is already working on this.
Since Sep 16, 2026.
- Dominant language
- Go
- Stars
- 589
- Forks
- 229
- Avg merge
- 1d 9h
- Merged PRs (30d)
- 77
Description
Describe the bug
The operator builds a config-reloader sidecar spec that its own reloader binary rejects.
When a VMAlert sets spec.configMaps, the sidecar crashes on start:
--target-dir count (1) must match watched directory count (2) from --watched-dir/--rules-dir
The pod never becomes ready. The rollout is stuck.
To Reproduce
- Run operator v0.74.1 with
config-reloader-v0.74.1. - Create a ConfigMap
my-templateswith a.tmplfile. - Create a
VMAlertwith:
spec:
configMaps:
- my-templates
extraArgs:
rule.templates: /etc/vm/configs/**/*.tmpl
- The
config-reloadercontainer fails. Exit code 2.
Generated args:
--target-dir=/etc/vmalert/rules-out/rules-src-0
--watched-dir=/etc/vm/configs/my-templates
--watched-dir=/etc/vmalert/rules-src-0
Two watched dirs, one target dir.
Root cause
Two separate problems, both in v0.74.1.
1. Unpaired --watched-dir
internal/controller/operator/factory/build/container.go:418-422 adds a bare --watched-dir for every entry in spec.configMaps. No --target-dir goes with it.
AddWatchTargetDir at container.go:525-536 does it correctly. Its own comment says it appends the pair "atomically".
cmd/config-reloader/main.go:100 then calls validateTargetDirCount, which fails at main.go:400-403.
So the operator and its reloader disagree inside a single release.
2. sort.Strings destroys the pairing
internal/controller/operator/factory/vmalert/vmalert.go:355 calls sort.Strings(crc.Args).
The flags are matched by position. Sorting puts /etc/vm/configs/... before /etc/vmalert/rules-src-0. Even with a matching count, the templates would be written into the rules output directory.
The strict check added in reloader v0.73.0 is what stops this. On reloader v0.72.0 the flag exists but there is no validation, so the corruption would be silent.
Why the init container is fine
container.go:416 takes the --only-init-config branch. That branch does not emit the crMounts watched dirs at all. So only the sidecar is affected.
Related
#1370 asked for spec.configMaps to be watched by the reloader. The bare --watched-dir comes from that. The validation added later in the reloader makes the two incompatible.
Workaround
Move the ConfigMap out of spec.configMaps into spec.volumes plus spec.volumeMounts at the same path:
spec:
configMaps: []
volumes:
- name: my-templates
configMap:
name: my-templates
volumeMounts:
- name: my-templates
mountPath: /etc/vm/configs/my-templates
readOnly: true
The vmalert container still sees the files. The reloader gets no extra watched dir. Templates are static, so no reload is needed.
Suggested fix
Use AddWatchTargetDir for crMounts too, or drop the bare --watched-dir loop. Also replace the positional pairing with something order independent, so sort.Strings cannot break it.
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Assessment
This issue has not been assessed yet.