cbar_kwargs are ignored for `plot.contourf`

Code Sample, a copy-pastable example if possible

I just stumbled on some weird behaviour, possibly a bug. When plotting with the contourf method, xarray ignores the extend value passed to cbar_kwargs

import numpy as np
import xarray as xr
%matplotlib inline

test_data = np.random.rand(10, 10)
da = xr.DataArray(test_data)
h = da.plot.contourf(vmax=0.5, cbar_kwargs={"extend": "neither"})
print(h.colorbar.extend)

When doing a similar plot with matplotlib this happens

# matplotlib example
import matplotlib.pyplot as plt
ha = plt.contourf(test_data, vmax=0.3)
cb = plt.colorbar(ha, extend='neither')

Is xarray adjusting the colorbar to the vmin and vmax inputs but not carrying the cbar_kwargs for that step?

This makes it especially hard to produce movies with (e.g. xmovie) because the colorbar flickers.

If someone could point me to the right location in the code and tests, I would be happy to put together a PR.

Output of xr.show_versions()

Start at the plot.contourf entry point and reproduce the supplied DataArray example, comparing cbar_kwargs={"extend": "neither"} with the matplotlib example. Trace how vmin and vmax are passed to colorbar creation; done means the requested extend value is respected and a regression test covers the behavior.