ansible-community / ansible-community/molecule-plugins
Podman driver fails with Error: reference parameter cannot be empty on Podman 5 when cmd_args is present
- Dominant language
- Python
- Stars
- 164
- Forks
- 113
- Avg merge
- 4d 20h
- Merged PRs (30d)
- 1
Description
### Summary
When running Molecule with the Podman driver (`molecule-podman`) against Podman $5.0.3$, the *Create* step fails with:
```text
Error: reference parameter cannot be empty
exit code 125
```
This happens whenever the `cmd_args` parameter is present in the `containers.podman.podman_container` task in `playbooks/create.yml`, even if `cmd_args` should logically resolve to an empty list or a simple list of strings.
If I remove `cmd_args` entirely from that task, the scenario passes.
---
### Environment
- OS: (please fill in: macOS / Linux, version)
- Python: `3.13.12`
- `molecule`: `25.12.0`
- `molecule-podman`: `2.0.3`
- `ansible-core`: `2.20.3`
- Podman: `5.0.3`
- Collections: includes `containers.podman` (version as shipped with the above)
---
### Minimal `molecule.yml` (failing)
```yaml
---
dependency:
name: galaxy
driver:
name: podman
platforms:
- name: teleport-agent-test
image: "cgm-hub.oci.jfrog.cgm.ag/library/ubuntu:24.04"
pre_build_image: true
tty: true
extra_opts: [] # explicitly a list, to avoid odd values
provisioner:
name: ansible
env:
TPA_TELEPORT_TOKEN: ""
TPA_APT_USERNAME: ""
TPA_APT_PASSWORD: ""
TELEPORT_AGENT_LOGIN_IDENTITY_FILE: ""
inventory:
host_vars:
teleport-agent-test:
ansible_os_family: Debian
ansible_distribution: Ubuntu
ansible_distribution_release: jammy
ansible_hostname: test-host
verifier:
name: ansible
```
This is essentially a trivial Podman-based scenario just to bring up an Ubuntu container and run the role.
---
### Command
```bash
molecule --debug test -s default --no-parallel
```
---
### Error output (relevant part)
```text
TASK [Create molecule instance(s)] *********************************************
ok: [localhost] => (item=teleport-agent-test)
TASK [Wait for instance(s) creation to complete] *******************************
[ERROR]: Task failed: Module failed: Container teleport-agent-test exited with code 125 when runed
Origin: /.../site-packages/molecule_plugins/podman/playbooks/create.yml:179:7
failed: [localhost] (item=teleport-agent-test) => {
"msg": "Container teleport-agent-test exited with code 125 when runed",
"stderr": "Error: reference parameter cannot be empty\n",
"stdout": "",
...
}
```
Manual Podman usage works fine:
```bash
podman pull cgm-hub.oci.jfrog.cgm.ag/library/ubuntu:24.04
podman run --rm -d --name teleport-agent-test \
cgm-hub.oci.jfrog.cgm.ag/library/ubuntu:24.04 \
/bin/bash -c "sleep infinity"
# container starts successfully
```
So the image and Podman itself are fine.
---
### Relevant snippet from `playbooks/create.yml` (as shipped)
This is the `Create molecule instance(s)` task from `molecule-podman 2.0.3`:
```yaml
- name: Create molecule instance(s)
containers.podman.podman_container:
name: "{{ item.name }}"
cap_add: "{{ item.capabilities | default(omit) }}"
# it's necessary to exclude empty args as ansible modules run_command method doesn't handle this well
cmd_args: "{{ molecule_podman_args | reject('equalto', '') | list + item.extra_opts | default([]) }}"
command: "{{ (command_directives_dict | default({}))[item.name] | default('') }}"
detach: "{{ item.detach | default(omit) }}"
device: "{{ item.devices | default(omit) }}"
dns: "{{ item.dns_servers | default(omit) }}"
env: "{{ item.env | default(omit) }}"
etc_hosts: "{{ item.etc_hosts | default(omit) }}"
executable: "{{ podman_exec }}"
expose: "{{ item.exposed_ports | default(omit) }}"
hostname: "{{ item.hostname | default(omit) }}"
image: "{{ item.pre_build_image | default(false) | ternary('', 'molecule_local/') }}{{ item.image }}"
ip: "{{ item.ip | default(omit) }}"
network: "{{ item.network | default(omit) }}"
pid: "{{ item.pid_mode | default(omit) }}"
privileged: "{{ item.privileged | default(omit) }}"
publish: "{{ item.published_ports | default(omit) }}"
security_opt: "{{ item.security_opts | default(omit) }}"
systemd: "{{ item.systemd | default(omit) }}"
tmpfs: "{{ item.tmpfs | default(omit) }}"
tty: "{{ item.tty | default(omit) }}"
ulimits: "{{ item.ulimits | default(omit) }}"
volume: "{{ item.volumes | default(omit) }}"
state: started
register: server
changed_when: false
with_items: "{{ molecule_yml.platforms }}"
loop_control:
label: "{{ item.name }}"
async: 7200
poll: 0
vars:
molecule_podman_args:
- "{% if item.cgroup_manager is defined %}--cgroup-manager={{ item.cgroup_manager }}{% endif %}"
- >-
{% if item.restart_policy is defined %}
--restart={{ item.restart_policy }}{% if item.restart_retries is defined %}:{{ item.restart_retries }}{% endif %}
{% endif %}
- "{% if item.storage_opt is defined %}--storage-opt={{ item.storage_opt }}{% endif %}"
- "{% if item.storage_driver is defined %}--storage-driver={{ item.storage_driver }}{% endif %}"
```
---
### What I tested
1. **As-shipped `create.yml` with Podman 5.0.3 (failing)**
With the `cmd_args` line above present, `molecule test` fails on the “Wait for instance(s) creation to complete” task, reporting that the container exited with code 125 and `Error: reference parameter cannot be empty`.
2. **Simplified `podman_container` call (works)**
If I temporarily simplify the task to:
```yaml
- name: Create molecule instance(s)
containers.podman.podman_container:
name: "{{ item.name }}"
image: "{{ item.pre_build_image | default(false) | ternary('', 'molecule_local/') }}{{ item.image }}"
tty: "{{ item.tty | default(omit) }}"
state: started
register: server
changed_when: false
with_items: "{{ molecule_yml.platforms }}"
loop_control:
label: "{{ item.name }}"
async: 7200
poll: 0
vars:
molecule_podman_args: []
```
then `molecule test` **succeeds** consistently. So the core `containers.podman.podman_container` → Podman 5 path works.
3. **Reintroducing fields gradually**
I then reintroduced all the other parameters (`env`, `hostname`, volumes, network, pid, systemd, tmpfs, security, ulimits, etc.) one by one. All of them work together without error *until* I add back `cmd_args`.
4. **`cmd_args` is the trigger**
As soon as I add back:
```yaml
cmd_args: "{{ molecule_podman_args | reject('equalto', '') | list + item.extra_opts | default([]) }}"
```
the container exits with code 125 and Podman prints `Error: reference parameter cannot be empty` again.
5. **Trying a “safer” `cmd_args` expression — still fails**
I tried replacing the expression with a more explicit form to be sure about Jinja precedence and defaults:
```yaml
cmd_args: >-
{{
(
(molecule_podman_args | reject('equalto', '') | list)
+
(item.extra_opts | default([]))
)
}}
```
I also explicitly set `extra_opts: []` in `molecule.yml` (a proper empty list). Even with this change, the error still occurs.
If I comment out `cmd_args` entirely, the error disappears.
---
### Hypothesis
- With Podman 5.0.3, the combination of `containers.podman.podman_container` and a non-empty `cmd_args` (or possibly even `cmd_args=[]`) appears to generate a `podman run` invocation that Podman interprets as having an “empty reference” argument, hence the `Error: reference parameter cannot be empty`.
- The issue is not in the `image` parameter (it works fine when `cmd_args` is removed) and not in the other container options (env, volumes, network, systemd, etc.), but specifically tied to `cmd_args`.
- Given that simplifying the task to omit `cmd_args` makes the scenario pass, this seems to be either:
- a bug in how Molecule’s Podman driver constructs `cmd_args` and passes it to the module, or
- a compatibility issue between `containers.podman.podman_container` and Podman 5 when `cmd_args` is used.
---
### Request
- Is `cmd_args` known to work with Podman 5.x in this context?
- Could the `Create` task be adjusted to:
- skip setting `cmd_args` entirely when the computed list would be empty, and/or
- avoid patterns known to trigger “reference parameter cannot be empty” with Podman 5?
- Alternatively, is there a recommended version combination (Podman / `containers.podman` / `molecule-podman`) for Podman-based Molecule testing that avoids this issue?
I’m happy to provide:
- Full `molecule --debug` logs,
- `podman info`,
- or to test proposed changes to `create.yml` against my environment.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start with molecule_plugins/podman/playbooks/create.yml, especially the Create molecule instance(s) task and its cmd_args expression. Run `molecule --debug test -s default --no-parallel` with Podman 5.0.3, compare the failing task with the simplified version that omits cmd_args, and inspect how the containers.podman.podman_container invocation handles the computed list. Done means the scenario starts successfully without Podman reporting an empty reference.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- tooling
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 45/100