alteryx / alteryx/nlp_primitives

Add Language primitive

未關閉
#142 2 則留言 0 個 reaction 已指派 0 人 在 GitHub 檢視
主要語言
Python
星號
37
分支
13
PR 合併指標
30 天內沒有已合併 PR

描述

- We can use this library to detect the language of a text
- https://github.com/facebookresearch/fastText/
- https://fasttext.cc/docs/en/language-identification.html
```python
import fasttext
filepath = os.path.join(FILE_PATH, "../data/lid.176.ftz")
model = fasttext.load_model(filepath)
predictions = self.model.predict(text="hej", k=2) # returns top 2 matching languages
predictions
```
> The object returned by the model is of the form ((‘__label__pl’, ‘__label__sv’), array([0.40688798, 0.23321952])) of where pl and sv are the ISO 639 code for Polish and Swedish. The prediction for both languages is correct as Hej means Hello in both languages. The second part indicates the respective confidence of the sentence belonging to those languages.

貢獻指南

這個儲存庫沒有索引到貢獻指南

評估

這個 Issue 還沒有評估資料。

把新 issue 寄到你的電子郵件信箱

精選適合新手參與的 GitHub issue 摘要。