KhronosGroup / KhronosGroup/Vulkan-ValidationLayers

BestPractices: Warning messages do not fail the tests

Open
#8,030 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

BestPractices
Dominant language
C++
Stars
1k
Forks
505
Avg merge
11h 25m
Merged PRs (30d)
229

Description

The warning messages are detected when surrounded by SetDesiredFailureMsg/VerifyFound in the negative tests, but they do not fail the positive tests, and also do fail the negative tests where the api usage is assumed to be correct but in fact violates some BP rule.

Warning error reporting can be enabled with:

void VkBestPracticesLayerTest::InitBestPracticesFramework() {
    m_errorMonitor->ExpectSuccess(kErrorBit | kWarningBit);
    ..

With this fix, BP will start reporting issues in the existing BP tests, they should be fixed.

Here are the few categories of errors I noticed:

  • debug utils are enabled as part of the test framework, and BP complains about it. One option is to ignore this for all the tests except the ones that check this specific use case.
  • some tests violate BP rules, should be just fixed
  • one special case is that there is a warning against using ALL_COMMANDS_BIT stage, and all stage masks are checked against it. That might be a questionable advice, because there are situations when ALL_COMMANDS_BIT is the only choice. We can discuss if this check should be removed.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start at VkBestPracticesLayerTest::InitBestPracticesFramework and the existing positive and negative BestPractices tests. Enable warning reporting there, run the affected tests, and review each newly reported warning. Done means the existing tests pass with valid API usage, while the ALL_COMMANDS_BIT warning policy and debug-utils handling are resolved consistently.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.