For numeric data scatter_matrix also displays the colour column in the grid

Hi there,

I was trying to plot some pandas data as [scatter_matrix](https://hvplot.holoviz.org/reference/pandas/scattermatrix.html). However, I noticed that the colour column must be specified as a non-numeric value. If the column to be used for colouring is also a numeric value, it will be plotted as a separate variable within the grid.

This is most likely due to [line 166](https://github.com/holoviz/hvplot/blob/cfa578d0045364f66244735e41cb6e6038e83c5f/hvplot/plotting/scatter_matrix.py#L166) in the scatter_matrix file, as the grid is first plotted with all columns and then the colour is added. Unfortunately, if this line is replaced with a holoviews.dataset without the colour column, some other errors occur...

If the colour column is converted to a string, everything works fine. If this is the way to use this function, I would appreciate a small note in the documentation!
This issue is relevant when colouring with cluster labels, which are usually ints.

#### Minimal reproducible example

```
from hvplot.plotting import scatter_matrix
from sklearn.datasets import load_iris
import pandas as pd

iris = load_iris()
X = iris.data
y = iris.target
df = pd.DataFrame(X, columns = iris.feature_names)

df['color'] = y#.astype(str)
scatter_matrix(df, c='color', alpha=0.8)

```

##### Without String cast:


##### With String cast:


##### software versions

Python implementation: CPython
Python version : 3.10.12
IPython version : 8.14.0

pandas : 1.5.3
matplotlib: 3.7.2
hvplot : 0.8.4
holoviews : 1.16.2
sklearn : 1.3.0
seaborn : 0.11.2
jupyterlab: 4.0.3

Working on macOS 13.4.1 (c) (22F770820d) in Visual Studio Code (Version: 1.80.1)