[EPIC] Floating-Point Deterministic Algorithms
- Dominant language
- C++
- Stars
- 2.5k
- Forks
- 486
- Avg merge
- 2d 6h
- Merged PRs (30d)
- 295
Description
Determinism and reproducibility of results from GPU algorithms is of increasing importance to many use cases.
There are several types of determinism guarantees one might care about. CCCL defines the following vocabulary for identifying and describing different kinds of determinism:
- GPU-to-GPU - results will always be bitwise identical for identical invocations, no matter the GPU
- Run-to-run - results will be identical for identical invocations on the same GPU, but may differ from one GPU to the next
- Not Guaranteed - results may differ in identical invocations
The non-associativity of floating-point arithmetic and the lack of order of execution guarantees from parallel execution generally mean that two identical invocations of an algorithm on identical floating-point inputs may result in different results.
In https://github.com/NVIDIA/cccl/issues/1558 we introduced our first new major deterministic algorithm, `cub::DeviceReduce` and the new `cuda::execution::determinism::gpu_to_gpu` guarantees API for enabling specifying the desired determinism behavior from an algorithm.
This issue is meant to track work related to adding new features/algorithms to CCCL aimed at improving determinism and reproducibility guarantees.
This issue is also meant to collect input from our users on what use cases for determinism are important to you to help us prioritize what we build next. **If there is a feature you would like to see, please leave a comment below** telling us as much as you can about: What algorithm/feature for determinism do you need? What kind of determinism do you require?
See also:
- Blog post: [Controlling Floating-Point Determinism in NVIDIA CCCL](https://developer.nvidia.com/blog/controlling-floating-point-determinism-in-nvidia-cccl/)
- segmented_scan (scan_by_key) https://nvbugspro.nvidia.com/bug/3477443
- segmented_reduce (reduce_by_key) https://github.com/NVIDIA/cccl/issues/794
Contributor guide
Assessment
This issue has not been assessed yet.