scikit-learn / scikit-learn/scikit-learn
Label Spreading: knn Kernel Variant differs from Reference Paper
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 67.3k
- Forks
- 27.4k
- Avg merge
- 1d 15h
- Merged PRs (30d)
- 58
Description
According to label_propagation.py#L487, the reference paper for the Label Spreading implementation is Zhou et al.: Learning with local and global consistency (2004).
But there are two things not in line with the paper when using the knn kernel variant for Label Spreading:
(A) Self-loops in affinity matrix
According to the paper, the adjacency matrix should have no self-loops (W_ii = 0). But when taking a look on the relevant line in _get_kernel, L134 (called with y=None, see L516 for Label Spreading and L397 for Label Propagation, kneighbors_graph get's called with mode='connectivity', which results in include_self=True, according to the docs.
Therefore, a more correct way would be calling it with include_self=False.
Note that this change would not only modify the kneighbors_graph affinity matrix used for Label Spreading, but also the one for Label Propagation. When looking at the reference paper for Label Propagation, Zhu et al.: Learning from labeled and unlabeled data with label propagation (2002), they seem not to require that there should be no-self loops.
But in sklearn's other reference, Bengio et al.: In Semi-Supervised Learning (2006), mentioned here, they advocate forcing W_ii=0 anyways - so I would use include_self=False for Label Propagation's kneighbors_graph as well.
(B) Laplacian Matrix Calculation
In label_propagation#L517, we see that sparse.csgraph.laplacian gets called. The default value for this method's use_out_degree is False. This is not a problem if the affinity matrix is symmetric, but as knn graphs usually are not symmetric, use_out_degree=True should be used.
One can better see this issue when getting the diagonal via sparse.csgraph.laplacian(..., normed=False, return_diag=True) - when use_out_degree=False, the diagonal values won't resemble the k from kneighbors_graph.
Summary
To conform with the reference paper, we should
- In label_propagation#L517, call
kneighbors_graphwithinclude_self=False - In label_propagation#L134, call
sparse.csgraph.laplacianwithuse_out_degree=True
If you agree with me, I can submit this small PR and fix the tests, in case one breaks. 😊
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start in semi_supervised/label_propagation.py by reading _get_kernel and the Label Spreading path around the referenced lines, then compare the current affinity and Laplacian behavior with the cited papers. Update the focused label-propagation tests as needed and verify both kernel variants produce the intended graph behavior without regressions.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Clearly specified
- Newbie friendliness
- 38/100