dssg / dssg/triage

Postmodeling: Jaccard Incorrect for Features

Open
#649 0 comments 0 reactions 0 assignees View on GitHub
postmodeling
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

When running the Jaccard Similarity for the top 100 features from a set of models on the JoCo project, I'm getting near-zero similarities:

![image](https://user-images.githubusercontent.com/960183/54932684-37ade300-4ee9-11e9-9bd5-928a87fc72e9.png)

However, when looking at the feature importances for these models, they do in fact share many top features in common. For instance, models 199548 & 199485, shown as having a 0 jaccard score here, actually have 69 of their top 100 features in common.

Contributor guide

Open the contributing guide

Research direction

The issue concerns the Jaccard Similarity calculation for the top 100 features from models on the JoCo project, but it names no file or test. Reproduce the result with models 199548 and 199485, compare the reported score with their 69 shared features, and verify that a regression check covers the corrected result.

Written by the indexing model from the issue text.

Assessment

Tech stack
machine-learning, python
Domain
machine-learning
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.