canonical / canonical/checkbox

Not-equals of two lists is not working in `requires:`

Open
#582 5 comments 0 reactions 0 assignees View on GitHub
bug
Dominant language
Python
Stars
52
Forks
81
Avg merge
2d 17h
Merged PRs (30d)
69

Description

### Bug Description

Given this resource:
```yaml
case: GLMark2Xwayland.fullscreen
xwayland: True
hosted: False

case: GLMark2Xwayland.windowed
xwayland: True
hosted: False

case: HostedGLMark2Wayland.fullscreen
xwayland: False
hosted: True

case: HostedGLMark2Wayland.windowed
xwayland: False
hosted: True
```

And this manifest:
```ini
[manifest]
io.mir-server::skip_xwayland_tests = true
```

With a `==` this correctly skips the cases that don't match.
```plain
requires:
[manifest.skip_xwayland_tests, "{xwayland}"] == ["True", "True"]

# ...

☑ : Run Mir performance test case: GLMark2Xwayland.fullscreen
☑ : Run Mir performance test case: GLMark2Xwayland.windowed
☐ : Run Mir performance test case: HostedGLMark2Wayland.fullscreen
☐ : Run Mir performance test case: HostedGLMark2Wayland.windowed
```

But flip the condition and they all run…

```plain
requires:
[manifest.skip_xwayland_tests, "{xwayland}"] != ["True", "True"]

# ...

☑ : Run Mir performance test case: GLMark2Wayland.fullscreen
☑ : Run Mir performance test case: GLMark2Wayland.windowed
☑ : Run Mir performance test case: GLMark2Xwayland.fullscreen
☑ : Run Mir performance test case: GLMark2Xwayland.windowed
```

Unless I'm missing something weird with requires…

### To Reproduce

See above

### Environment

N/A

### Relevant log output

_No response_

### Additional context

_No response_

Contributor guide

Open the contributing guide

Research direction

Start by reproducing the manifest and requires expressions shown in the issue, then trace the requires expression evaluation that handles list comparisons. Confirm the behavior for both == and != with the two lists, and consider the work done when the != form filters the same cases as the inverted == condition.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing-qa
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.