dotnet / dotnet/machinelearning
Discrepancy in NgramExtractorTrasform, NgramExtractingTransformer and NgramExtractingEstimator.
- Dominant language
- C#
- Stars
- 9.4k
- Forks
- 2k
- Avg merge
- 2d 20h
- Merged PRs (30d)
- 11
Description
If you search for `NgramExtract` in the solution, the following three main classes pop up.
1. NgramExtractorTransform (in WordBagTransform.cs)
2. NgramExtractingTransformer (in NgramTransform.cs)
3. NgramExtractingEstimator (in NgramTrasnform.cs)
`2` and `3` seem to be the actual classes where ngram extraction logic is written. However, `1` uses `2` and `3` with a pre-processing step where if input is text it is first converted to terms using ValueToKeyMappingTransformer.
First, `NgramExtractorTransform` does not seem to be in correct file i.e filename and class name do not match.
Second, the `NgramExtractorTransform` is not doing ngram extraction instead composing two different estimators (NgramExtractingEstimator and ValueToKeyMappingEstimator).
I think `NgramExtractorTransform` be renamed to `WordBagTransform` or something appropirate.
CC: @Ivanidzo4ka, @TomFinley, @sfilipi, @rogancarr.
Contributor guide
Assessment
This issue has not been assessed yet.