intel / intel/llvm

[SYCL] Poor Performance - Very Low bandwidth of SYCL kernel for Batch Normalization

Open
#8,177 3 comments 0 reactions 0 assignees View on GitHub
bug cuda
Dominant language
LLVM
Stars
1.5k
Forks
854
Avg merge
3d 17h
Merged PRs (30d)
137

Description

**Describe the bug**

This reproducer is created for enhancing the performance i.e., achieving higher bandwidth for the SYCL implementation of
Batch Normalization primitive on Nvidia.
This reproducer performs the batch normalization operation and calculates the memory bandwidth utilized by the kernel for forward and backward propagations of the primitive.

**To Reproduce**

1. For the reproducer code, refer the attachments _setup.sh_, _ref_batch_normalization.cpp_ and _ref_batch_normalizations_kernels.hpp_

2. Go to the directory having the reproducer files and run the below script to setup the environment.
`source setup.sh`

To compile, run.
`clang++ -O2 -fsycl -fsycl-targets=nvptx64-nvidia-cuda ref_batch_normalization.cpp`

3. The above generates the output file. To see the output bandwidth, run
`./a.out`

To test the code for different inputs, Please refer Additional Context section.

4. The below illustrates the observed and expected behaviour.

##### Observed behavior

For higher values, say N = 1024, C = 2048, D = 1, H =7, W = 7 for both the propagations and with all flags ENABLED (refer Additional Context section),
the bandwidth observed is too low.

```
Forward-Bandwidth:9.544018 gb/sec
Backward-Bandwidth:5.654824 gb/sec
```

##### Expected behavior

The ideal behavior is to attain the maximum bandwidth (i.e., CLPeak value for Float16 = 251.94 GBPS) for any input size.
For the current reproducer at least 40-60% bandwidth is expected i.e., a minimum of 100 GBPS is desired.

**Environment**

- OS: Ubuntu 22.04.1 LTS
- Target device and vendor: Nvidia, Tesla T4
- DPC++ version: clang version 15.0.0 (https://github.com/intel/llvm.git 0c7a1e18978754451f5c2c95129721297e2c2411)
- Dependencies version: Driver Version: 495.29.05 CUDA Version: 11.5

**Additional context**

1. The inputs {N, C, D, H, W} and flags for the primitives in this reproducer are hard coded.

2. Please run _a.out_ multiple times after compilation to see the accurate results.

3. The usage of compiler flag `-ffast-math option` is resulting in accuracy drop.

4. Below contains the code file
[batch_norm_reproducer.zip](https://github.com/intel/llvm/files/10566324/batch_norm_reproducer.zip)

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.