oneapi-src / oneapi-src/unified-memory-framework

Extend FAIL_ON_SKIP to gtest

Open
#640 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
C
Stars
98
Forks
48
Avg merge
3d 20h
Merged PRs (30d)
6

Description

Problem Statement

The current CMake option UMF_TESTS_FAIL_ON_SKIP is designed to handle skipped tests. However, it does not work with GTEST_SKIP in Google Test. This limits its functionality and does not enforce consistent behavior for skipped tests.

Objective

Extend the UMF_TESTS_FAIL_ON_SKIP option to work with GTEST_SKIP to ensure that skipped tests are treated as failures when this option is enabled.

Solution

Implement a macro to replace GTEST_SKIP with a custom macro (UMF_SKIP). This macro will conditionally either call GTEST_SKIP or fail the test based on the UMF_FAIL_ON_SKIP option.

Implementation Steps
  1. Define the Custom Macro:

Create a new macro UMF_SKIP which will conditionally handle skipped tests based on the UMF_FAIL_ON_SKIP option.

#ifdef UMF_FAIL_ON_SKIP
#define UMF_SKIP GTEST_FAIL() << "UMF_TESTS_FAIL_ON_SKIP Enabled: Treating skip as fail"
#else
#define UMF_SKIP GTEST_SKIP()
#endif
  1. Replace GTEST_SKIP in Tests:

Replace all occurrences of GTEST_SKIP in tests with UMF_SKIP.

  1. Define UMF_FAIL_ON_SKIP Through Compiler Argument:

Ensure the UMF_FAIL_ON_SKIP macro can be defined via a compiler argument (e.g., -DUMF_FAIL_ON_SKIP).
Modify the CMake configuration to include this option.

if(UMF_TESTS_FAIL_ON_SKIP)
    add_definitions(-DUMF_FAIL_ON_SKIP)
endif()
  1. Update CI Configuration:

Explicitly exclude tests that are expected to skip on the CI environment in the YAML configuration file for CI pipeline.
This is the hardest part, as we do not have option to disable single "gtest" test from YAML level. We did something like this for valgrind tests - but it required separate .sh file for those tests, so better approach is to add extra cmake configuration flags <TEST_NAME>_FILTER and modify add_test function to handle it

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the CMake handling for UMF_TESTS_FAIL_ON_SKIP, all GTEST_SKIP occurrences in the tests, and the CI YAML configuration. Implement and verify the UMF_SKIP behavior, compiler definition, test replacements, and a way to exclude expected skips through CMake or CI filters; done means skipped tests fail when enabled without breaking normal skips.

Written by the indexing model from the issue text.

Assessment

Tech stack
c, cmake
Domain
build-system, ci-cd, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.