Implement an Interpretable linear model with stochastic gradient descent learning
- Dominant language
- C++
- Stars
- 1.9k
- Forks
- 164
- Avg merge
- 7h 31m
- Merged PRs (30d)
- 1
Description
A linear model we are going to add to `datatable.models` should support binomial and multinomial classifications, as well as numeric regression. It should be a regularized model with stochastic gradient decent (SGD) learning similar to the implementation provided by [scikit-learn](https://scikit-learn.org/stable/modules/classes.html#module-sklearn.linear_model).
The model should support all the column types that are currently supported by `datatable`, i.e. numeric and string columns, and will include
**I. The preparation step: feature binning**
- for string columns we will do one hot encoding, it should also be possible to specify category groups for a particular column;
- for numeric columns we will do binning first, the number of bins or the corresponding intervals will be an input to the model. Second, we will do one hot encoding based on the binned data.
These preparation steps will ensure the model has high interpretability.
**II. Training the model**
- each row of data will be converted on-the-fly to a set of zeros and ones — an input to our model;
- then, we will train the model by using the SGD method;
- Its API should be similar to the datatable FTRL API: regularization parameters, learning rate, precision, etc. but will exclude parameters related to the hashing trick.
**Preliminary plan**
- [x] `cut()` function for equal-interval binning of numeric data;
- [x] `qcut()` function for equal-population binning of numeric data;
- [x] support binning by intervals in `cut()`;
- [x] linear model algo and API;
- [ ] encoding of categoricals;
- [x] documentation and tutorials.
Contributor guide
Assessment
This issue has not been assessed yet.