[SYCL] Poor Performance - Very Low bandwidth of SYCL kernel for Batch Normalization
- Dominant language
- LLVM
- Stars
- 1.5k
- Forks
- 854
- Avg merge
- 3d 17h
- Merged PRs (30d)
- 137
Description
**Describe the bug**
This reproducer is created for enhancing the performance i.e., achieving higher bandwidth for the SYCL implementation of
Batch Normalization primitive on Nvidia.
This reproducer performs the batch normalization operation and calculates the memory bandwidth utilized by the kernel for forward and backward propagations of the primitive.
**To Reproduce**
1. For the reproducer code, refer the attachments _setup.sh_, _ref_batch_normalization.cpp_ and _ref_batch_normalizations_kernels.hpp_
2. Go to the directory having the reproducer files and run the below script to setup the environment.
`source setup.sh`
To compile, run.
`clang++ -O2 -fsycl -fsycl-targets=nvptx64-nvidia-cuda ref_batch_normalization.cpp`
3. The above generates the output file. To see the output bandwidth, run
`./a.out`
To test the code for different inputs, Please refer Additional Context section.
4. The below illustrates the observed and expected behaviour.
##### Observed behavior
For higher values, say N = 1024, C = 2048, D = 1, H =7, W = 7 for both the propagations and with all flags ENABLED (refer Additional Context section),
the bandwidth observed is too low.
```
Forward-Bandwidth:9.544018 gb/sec
Backward-Bandwidth:5.654824 gb/sec
```
##### Expected behavior
The ideal behavior is to attain the maximum bandwidth (i.e., CLPeak value for Float16 = 251.94 GBPS) for any input size.
For the current reproducer at least 40-60% bandwidth is expected i.e., a minimum of 100 GBPS is desired.
**Environment**
- OS: Ubuntu 22.04.1 LTS
- Target device and vendor: Nvidia, Tesla T4
- DPC++ version: clang version 15.0.0 (https://github.com/intel/llvm.git 0c7a1e18978754451f5c2c95129721297e2c2411)
- Dependencies version: Driver Version: 495.29.05 CUDA Version: 11.5
**Additional context**
1. The inputs {N, C, D, H, W} and flags for the primitives in this reproducer are hard coded.
2. Please run _a.out_ multiple times after compilation to see the accurate results.
3. The usage of compiler flag `-ffast-math option` is resulting in accuracy drop.
4. Below contains the code file
[batch_norm_reproducer.zip](https://github.com/intel/llvm/files/10566324/batch_norm_reproducer.zip)
Contributor guide
Assessment
This issue has not been assessed yet.