dssg / dssg/tyra

Between model: jaccard similarity

Open
#106 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
JavaScript
Stars
4
Forks
1
PR merge metrics
No merged PRs in 30d

Description

Jaccard similarity gives the proportion of matches in two lists.

- Produce Jaccard similarities for top 1%, 5%, etc. of predictions so we can see if the models are flagging the same people.

- Produce Jaccard similarities for top 1%, etc of feature importances so we can see if the models are using the same set of features

Contributor guide

No contributing guide indexed for this repository

Research direction

Start by locating the prediction-model evaluation entry point and the inputs for predictions and feature importances. Implement Jaccard similarities at the requested top 1%, 5%, and other stated cutoffs, then verify that results show overlap between models for both people flagged and features used.

Written by the indexing model from the issue text.

Assessment

Tech stack
javascript
Domain
machine-learning
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.