CosmicHorrorDev / CosmicHorrorDev/rust_text_classifier
Revamp posts database
- Dominant language
- Python
- Stars
- 13
- Forks
- 1
- PR merge metrics
- No merged PRs in 30d
Description
Right now the posts database is pretty barebones. It would be nice if there was more information stored. Particularly if it stored posts instead of having a separate script so that it could build the corpus used to train (with some sort of manual step for classification still). I think the information that could be helpful to store would be the action done to the post (commented on, or ignored w/ the reasoning), more post information (title, body, author, time, link, id), the predicted classification with accuracy and the actual classification (form manual verification), and finally some metadata on the current classifier being used (so that changes can be tracked specific to the classifier). I think the number of lang vs game posts, oldest post date (or id) and newest post date, and then a hash of the classifier would be enough. This gives a good idea of the posts used in the classifier and allows for distinguishing between different classifiers that were trained on the same corpus.
Overall that gives a lot more to work with and opens the door to having `PostsLoader` load in posts from a database instead of a bunch of different files which I think it an improvement in many regards
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.