kernelci / kernelci/kernelci-pipeline

Report test configuration details in the nodes

Open
#636 1 comment 0 reactions 0 assignees View on GitHub
enhancement
Dominant language
Python
Stars
11
Forks
40
Avg merge
2d 13h
Merged PRs (30d)
14

Description

This issue is to initiate an open discussion regarding the possibility of providing users with more comprehensive information about the specific configurations of test suites through the KernelCI nodes.

Tests may be executed with different parameters across different platforms (depending on factors such as hardware configuration). Thus, it's important to provide details on the configuration/parameters used on a specific test run. This enables users to know how the test was conducted on each platform and to easily identify if a test failure correlates with the specific configuration in use.

This may become particularly relevant with the addition of support for running kselftests with parameters (see https://github.com/Linaro/test-definitions/pull/511).

Some possible solutions:
- Consider adding extra node fields to report the command used for the test and/or the parameters applied (where applicable).
- Ensure that test suites running with different parameters are assigned distinct names or IDs. This may be challenging to enforce as a rule, and users may still need to refer to the YAML files for detailed test information.

Some test suites can be harder to describe than others, so I feel like the challenge is to find a way to effectively describe the test configuration that works for different test suites (maybe a link to the test job template used is enough?).

Any thoughts on this?

Contributor guide

No contributing guide indexed for this repository

Research direction

Review the YAML files and the kselftests parameter work in PR #511, then examine how test suites are represented in KernelCI nodes. The work is done when users can distinguish runs with different parameters and inspect the command, parameters, or test job template used for each run.

Written by the indexing model from the issue text.

Assessment

Tech stack
yaml
Domain
testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.