Object detection using GPU on Windows is about 5 times slower than on Ubuntu

System information

• What is the top-level directory of the model you are using:
  ssd_mobilenet_v1_coco_11_06_2017
• Have I written custom code (as opposed to using a stock example script provided in TensorFlow):
  No
• OS Platform and Distribution (e.g., Linux Ubuntu 16.04):
  Ubuntu 16.04
  Windows 10 build 15063.413
• TensorFlow installed from (source or binary):
  Binary
• TensorFlow version (use command below):
  v1.2.0-rc2-21-g12f033d
• Bazel version (if compiling from source):
  NA
• CUDA/cuDNN version:
  8.0/5.1
• GPU model and memory:
  Ubuntu GTX970 4GB
  Windows GTX1080 12GB
• Exact command to reproduce:
  Using Object Detection Demo with the 2 images provided

Describe the problem

Installed Nvidia Cuda and cuDNN on both systems.
Installed tensorflow on Ubuntu and Windows with GPU support both times using Python 3.5.2 and the native pip installation.
Installed and setup models as required.

My Windows is running on a machine with a GTX1080, my Ubuntu box with GTX970.
Comparing the results between the 2 setups shows that on Windows the object detection is about 5 times slower than on Ubuntu.
Measuring the GPU load on Windows using GPU-Z shows that the GPU is barely being used when the object detection is running, only jumping up to about 16% from time to time.
I did the same test with tensorflow without GPU support on the same machine and the result is about 20% slower than compared to with GPU.

Interestingly when using the mnist_deep.py example from site-packages/tensorflow/examples/tutorials/mnist the GPU load on my Windows machine goes up to about 62%.

It looks like object_detection is for some reason not using the full capabilities of the GPU on Windows, the question is why?

Are there any additional configurations I have to perform to speed it up?
Are there any Nvidia windows graphic driver settings I can/should use?
Anyone else having the same issue?

Cheers,
Ralf