Create smoke tests for the scripts like langmodel training, classifier training, etc.

There is a certain test coverage of the project with unit-tests. However, when it comes to end-to-end tests, we don't have any. We can often see how after refactorings certain things can break (Most often, we have problems with path ). So, to check that everything is all right, we need to run the scripts manually. That's why it would be good to write smoke tests that will run the scripts on some small test sets and clean after themselves afterward. Scripts for which smoke tests are needed:
* lang_model.py
* log_level_classifier.py
* vocabsize.py
* dataset_generator.py
* dataset_stats.py
* parse_projects.py
* to_repr.py

Start by inspecting the entry points for lang_model.py, log_level_classifier.py, vocabsize.py, dataset_generator.py, dataset_stats.py, parse_projects.py, and to_repr.py, then determine how each accepts a small test set. Add smoke coverage that runs each script and cleans up its temporary outputs; done means the scripts complete successfully through the tests without path-related failures.

Written by the indexing model from the issue text.