lablup / lablup/backend.ai

`./backend.ai test run-cli` command returns `exit code 0` even on failure

Open
#2,666 0 comments 0 reactions 1 assignee Assigned to @Yaminyam View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

### What Operating System(s) are you seeing this problem on?

Linux (x86-64)

### Backend.AI version

24.09

### Describe the bug

While investigating why our integration tests in a GitHub Action workflow were not failing as expected,I discovered that the `./backend.ai test run-cli` command returns an **exit code of 0** even when it encounters a failure.

This behavior is misleading because it causes CI/CD pipelines to treat failing tests as successful, leading to false positives.

### To Reproduce

1. Run the `./backend.ai test run-cli admin`
1. Check the exit code immediately after the command finishes using `echo $?`
### Expected Behavior

The command should return a non-zero exit code if it encounters any errors during execution.

### Anything else?

_No response_

JIRA Issue: BA-95

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.