dssg / dssg/triage

Individual importances break for models that don't write overall importances

Open
#295 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Jupyter Notebook
Stars
201
Forks
62
PR merge metrics
No merged PRs in 30d

Description

_From @shaycrk on October 5, 2017 15:36_

Models such as the scaled logistic regression simply write "Algorithm does not support a standard way to calculate feature importance." to the overall feature importances, which generates a key error when the individual importances code tries to find this value in the matrix, such as [here](https://github.com/dssg/catwalk/blob/master/catwalk/individual_importance/uniform.py#L18), killing the child process. In at least my most recent run, this seems to have caused the process to die before evaluations could be written to the database.

cc @thcrock @nanounanue

_Copied from original issue: dssg/catwalk#31_

Contributor guide

Open the contributing guide

Research direction

Start with catwalk/individual_importance/uniform.py, especially the lookup at line 18, and trace how models that do not provide overall feature importances are represented. Reproduce the scaled logistic regression case and confirm that individual-importance processing no longer kills the child process; verify that evaluations can still be written to the database.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
machine-learning
Issue type
Bug
Difficulty
3/5
Estimated time
1-2 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.