biolab / biolab/orange3

Allow Neighbors to accept sparse data

Open
#6,749 3 comments 0 reactions 0 assignees View on GitHub
snack
Dominant language
Python
Stars
5.7k
Forks
1.1k
Avg merge
12d 2h
Merged PRs (30d)
1

Description

**What's your use case?**
I want to use Neighbors to search a corpus of documents for items similar to one or more reference documents. Since Neighbors requires that Reference and Data have the same features, I have to apply either Text Embedding, Similarity Hashing or Topic Modeling in order to represent the corpora quantitatively. But for most ML tasks with text, I find Bag of Words usually producing more convincing results.

**What's your proposed solution?**
Allow Neighbors to accept datasets with different features, at least when it comes to sparse data from Bag of Words. So, before computing distances, the words that are in Reference but not in Data are added to Data with value 0, and the other way around.

**Are there any alternative solutions?**
Not that I'm aware of.

Contributor guide

Open the contributing guide

Research direction

Start by locating the Neighbors widget and reading how it currently handles Reference and Data feature compatibility. Compare that behavior with the proposed Bag-of-Words use case and determine how sparse features would be aligned before distance computation. Done means Neighbors accepts compatible sparse datasets with differing feature sets and preserves the existing behavior for matching features.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
data, machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.