KhronosGroup / KhronosGroup/SPIRV-Tools

Improve test coverage for TransformationCompositeConstruct

Open
#4,243 0 comments 0 reactions 0 assignees View on GitHub
component:fuzzer
Dominant language
C++
Stars
1.4k
Forks
709
Avg merge
1d 22h
Merged PRs (30d)
28

Description

The tests in test_transformation_composite_construct.cpp do not provide 100% statement coverage of the transformation_composite_construct.cpp file. It would be good to improve the coverage achieved by these tests, and doing so would provide a nice introduction to the spirv-fuzz project.

To get started on this we recommend using the CLion IDE, if you have access to it.

Make a new GoogleTest run configuration, to run all tests in the TransformationCompositeConstructTest suite. Specify test_fuzz as the target.

Select "Run 'All in TransformationCompositeConstructTest' with Coverage" to run these tests with coverage enabled.

When the tests finish you should get a coverage report (it might take a bit of time to generate the coverage report).

In the coverage report, click through to the file source/fuzz/transformation_composite_construct.cpp.

I find that 89% of lines in this file are covered. It would be good to bring this to 100% or close.

This can be achieved in four stages:

(0) Read "message TransformationCompositeConstruct" in spvtoolsfuzz.proto, and study the existing tests for this transformation, to get some understanding for how it works.

(1) Concentrate on writing a test or tests that cover lines in IsApplicable.

A test should be added to test_transformation_composite_construct.cpp, and should:

- declare a piece of SPIR-V
- declare some transformations that are designed to hit particular "return false" statements in IsApplicable that are not currently covered
- assert that IsApplicable does not hold for each transformation

(2) Get coverage of the ToMessage method and the constructor that takes a message argument by adapting one of the existing tests as follows:

- if the existing test creates a transformation using the other constructor, you could then use ToMessage to turn this into a protobuf message, and then construct a new transformation using the resulting message.

(3) Write new tests to get extra coverage of ComponentsForArrayConstructionAreOK. This will involve understanding how the transformation works in more detail.

Contributor guide

Open the contributing guide

Research direction

Read the TransformationCompositeConstruct message in spvtoolsfuzz.proto and the existing tests in test_transformation_composite_construct.cpp. Run the TransformationCompositeConstructTest suite with coverage for source/fuzz/transformation_composite_construct.cpp, then add tests for uncovered IsApplicable branches, ToMessage, the message constructor, and ComponentsForArrayConstructionAreOK. Done means coverage reaches 100% or close.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp
Domain
testing
Issue type
Refactor
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
42/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.