reinforcement-learning/cartpole-on-single-compute generates non-playable videos

I'm running https://github.com/Azure/MachineLearningNotebooks/tree/6f893ff0b40d3d6af94ac43867676a3d9df1fcbd/how-to-use-azureml/reinforcement-learning/cartpole-on-single-compute on a default compute instance with "Python 3.6 AzureML" kernel.

The generated videos have size of 262 bytes and seem non-playable. The issue is possibly related to https://github.com/openai/gym/issues/1925.