Individual importances break for models that don't write overall importances
- Dominant language
- Jupyter Notebook
- Stars
- 201
- Forks
- 62
- PR merge metrics
- No merged PRs in 30d
Description
_From @shaycrk on October 5, 2017 15:36_
Models such as the scaled logistic regression simply write "Algorithm does not support a standard way to calculate feature importance." to the overall feature importances, which generates a key error when the individual importances code tries to find this value in the matrix, such as [here](https://github.com/dssg/catwalk/blob/master/catwalk/individual_importance/uniform.py#L18), killing the child process. In at least my most recent run, this seems to have caused the process to die before evaluations could be written to the database.
cc @thcrock @nanounanue
_Copied from original issue: dssg/catwalk#31_
Contributor guide
Research direction
Start with catwalk/individual_importance/uniform.py, especially the lookup at line 18, and trace how models that do not provide overall feature importances are represented. Reproduce the scaled logistic regression case and confirm that individual-importance processing no longer kills the child process; verify that evaluations can still be written to the database.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100