The Affinity Matrix Is NON-BINARY with`affinity="precomputed_nearest_neighbors"`

Describe the issue linked to the documentation

Issue Source:

https://github.com/scikit-learn/scikit-learn/blob/59dd128d4d26fff2ff197b8c1e801647a22e0158/sklearn/cluster/_spectral.py#L452-L454

Issue Description

The Affinity Matrix Is non-binary withaffinity="precomputed_nearest_neighbors". I.e., when a precomputed distance matrix is given as x, the affinity matrix from SpectralClustering.fit().affinity_matrix_ is NOT binary (as described in the document). It has 3 values: 0.0, 1.0, and 0.5.

Reproducible Code Snippet

Generate a random distance ,a

from sklearn.cluster import SpectralClustering
import numpy as np

## generate a random distance matrix --> symmetric
np.random.seed(0)
distmat=np.random.rand(200,200)
distmat=(np.triu(distmat,1)+np.triu(distmat,1).T)/2
print(f"Check asymmetric locations (if any):\t{np.where(distmat!=distmat.T)}")

## affinity matrix 
aff_mat=SpectralClustering(n_clusters=30,affinity='precomputed_nearest_neighbors',assign_labels='discretize', n_neighbors=50 ,n_jobs=-1).fit(distmat).affinity_matrix_.toarray()
print(f"Unique values (ought to be 'binary'):\t{np.unique(aff_mat)}")

Machine & Version Info

System:
    python: 3.8.3 (default, Jul  2 2020, 16:21:59)  [GCC 7.3.0]
executable: /opt/share/linux-rocky8-x86_64/gcc-12.2.0/anaconda3-2020.07-yv6vdwqiouaru27jxhpezh6t6mdpqf3e/bin/python
   machine: Linux-4.18.0-425.3.1.el8.x86_64-x86_64-with-glibc2.10

Python dependencies:
          pip: 20.1.1
   setuptools: 65.6.3
      sklearn: 0.23.1
        numpy: 1.22.3
        scipy: 1.5.0
       Cython: 0.29.21
       pandas: 1.4.2

Built with OpenMP: True

Suggest a potential alternative/fix

Since the affinity matrix is calculated as (connectivity+connectivity.T)*0.5 source_code, and that the connectivity is calculated by kneighbors_graph source_code, it is intrinsically not symmetric -- i might be j's K nearest neighbor, while j could not be i's when n_quirey == n_samples.

Start with sklearn/cluster/_spectral.py at the cited affinity construction and compare it with the documented binary behavior. Run the provided SpectralClustering example and inspect sklearn/neighbors/_base.py where kneighbors_graph is created. Done means resolving the documentation-versus-behavior discrepancy and updating the relevant documentation or coverage accordingly.