NVIDIA / NVIDIA/cccl

FileCheck validation fails for some examples when building with OMP/TBB host systems.

Open
#740 1 comment 0 reactions 1 assignee Claimed by @gevtushenko View on GitHub
thrust
Dominant language
C++
Stars
2.5k
Forks
486
Avg merge
2d 6h
Merged PRs (30d)
295

Description

# Background

The Thrust examples are currently validated by comparing their output to a reference output using the LLVM `FileCheck` utility. This additional validation is not enabled by default, and as a result the ctest tests that run example code cannot fail by default.

# Failures

After enabling `FileCheck`, examples start failing on configurations with a non-`cpp` host:

```
6844 - thrust.omp.cpp.cpp11.example.bucket_sort2d (Failed)
6880 - thrust.omp.cpp.cpp11.example.sum (Failed)
6895 - thrust.omp.cpp.cpp14.example.bucket_sort2d (Failed)
6931 - thrust.omp.cpp.cpp14.example.sum (Failed)
6946 - thrust.omp.cpp.cpp17.example.bucket_sort2d (Failed)
6982 - thrust.omp.cpp.cpp17.example.sum (Failed)
6997 - thrust.omp.omp.cpp11.example.bucket_sort2d (Failed)
7033 - thrust.omp.omp.cpp11.example.sum (Failed)
7048 - thrust.omp.omp.cpp14.example.bucket_sort2d (Failed)
7084 - thrust.omp.omp.cpp14.example.sum (Failed)
7099 - thrust.omp.omp.cpp17.example.bucket_sort2d (Failed)
7150 - thrust.omp.tbb.cpp11.example.bucket_sort2d (Failed)
7201 - thrust.omp.tbb.cpp14.example.bucket_sort2d (Failed)
7252 - thrust.omp.tbb.cpp17.example.bucket_sort2d (Failed)
7288 - thrust.omp.tbb.cpp17.example.sum (Failed)
7474 - thrust.tbb.cpp.cpp11.example.bucket_sort2d (Failed)
7525 - thrust.tbb.cpp.cpp14.example.bucket_sort2d (Failed)
7576 - thrust.tbb.cpp.cpp17.example.bucket_sort2d (Failed)
7627 - thrust.tbb.omp.cpp11.example.bucket_sort2d (Failed)
7678 - thrust.tbb.omp.cpp14.example.bucket_sort2d (Failed)
7729 - thrust.tbb.omp.cpp17.example.bucket_sort2d (Failed)
7780 - thrust.tbb.tbb.cpp11.example.bucket_sort2d (Failed)
7831 - thrust.tbb.tbb.cpp14.example.bucket_sort2d (Failed)
7882 - thrust.tbb.tbb.cpp17.example.bucket_sort2d (Failed)
```

## `example.sum`

The `example.sum` seems to have a serious issue with producing stable results:

```
$ for i in `seq 10`;do examples/thrust.omp.omp.cpp11.example.sum; done
sum is 509119
sum is 498850
sum is 521001
sum is 510424
sum is 527787
sum is 507166
sum is 500323
sum is 519696
sum is 524764
sum is 528394
```

## `example.bucket_sort2d`

The `example.bucket_sort2d` failures seem to be from the output being written in non-deterministic order when the host system is not `cpp`.

## RCCA: Replace `FileCheck` with more robust runtime checks

We need to address these failures and fix them, but should also reconsider the use of optional dependencies like `FileCheck` for testing, since these failures have done undetected for a while. Instead, the example executables should validate themselves at runtime and use their exit code to indicate failure. This will simplify our ctest harness significantly.

# Open Question: DVS?

I'm not sure how difficult it would be to update DVS to expect runtime failures. Updating CMake for this would be trivial.

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.