Write a utility to assign weights to a compiled transducer based on a corpus
- Dominant language
- C++
- Stars
- 21
- Forks
- 26
- PR merge metrics
- No merged PRs in 30d
Description
I imagine it will be called `lt-reweight`
It should have two arguments:
1) a binary lttoolbox file e.g. ` grn.automorf.bin `
2) a tagged corpus `grn.tagged`
```
$ lt-reweight grn.automorf.bin grn.tagged
```
Where grn.tagged looks like:
```
^Avañeʼẽ/avañeʼẽ$
^ha/ha$
^Guarani/guarani$
^ñeʼẽ/ñeʼẽ$
^ombohéra/o+mbohéra$
^hikuái/hikuái$
^umi/umi$
^Guaranikuéra/guarani+kuéra$
^pe/pe$
^ñeʼẽ/ñeʼẽ$
^teépe/tee+pe$
^./.$
^Guarani/guarani$
^haʼe/haʼe$
^peteĩva/peteĩ+va$
^umi/umi$
^teʼyikuéra/teʼyi+kuéra$
^Amérika-gua/Amérika+gua$
^ñeʼẽnguéra/ñeʼẽ+kuéra$
^apytépe/apytépe$
^hetave/heta+ve$
^iñeʼẽhárava/iñeʼẽhárava$
^,/,$
^oñemohendáva/o+je+mohenda+va$
^irundy/irundy$
^tetãnguéra/tetã+kuéra$
^iñambuévape/iñambuéva+pe$
^(/($
^Paraguái/Paraguái$
^,/,$
^Argentina/Argentina$
^,/,$
^Volívia/Volívia$
^ha/ha$
^Brasil/Brasil$
^)/)$
^./.$
^Avei/avei$
^,/,$
^haʼe/haʼe$
^ñoite/ñoite$
^ojehechakuaáva/o+je+hechakuaa+va$
^ñeʼẽ/ñeʼẽ$
^teéramo/tee+ramo$
^peteĩ/peteĩ$
^tetã/tetã$
^Ñembyamérika-guápe/Ñembyamérika+gua+pe$
^./.$
^Tupi/Tupi$
^ha/ha$
^guarani/guarani$
^ñeʼẽ/ñeʼẽ$
^aty/aty$
^guasu/guasu$
^rehegua/rehegua$
^,/,$
^oguereko/o+guereko$
^hetáichagua/hetáichagua$
^ñeʼẽnunga/ñeʼẽnunga$
^,/,$
^upéicharõ/upéicha+rõ$
^jepe/jepe$
^oĩ/oĩ$
^jekupyty/jekupyty$
^ijapytepekuéra/i+japyte+pe+kuéra$
^ha/ha$
^heta/heta$
^mbaʼépe/mbaʼe+pe$
^ojojogua/ojojogua$
^koʼã/koʼã$
^ñeʼẽnungakuéra/ñeʼẽnunga+kuéra$
^./.$
^Avañeʼẽ/avañeʼẽ$
^ha/ha$
^karaiñeʼẽ/karaiñeʼẽ$
^haʼe/haʼe$
^Paraguái/Paraguái$
^retãme/tetã+pe$
^ñeʼẽ/ñeʼẽ$
^tee/tee$
^ary/ary$
^1992/1992$
^guive/guive$
^./.$
^Japypateĩ/Japypateĩ$
^2006/2006$
^guive/guive$
^haʼe/haʼe$
^avei/avei$
^ñeʼẽ/ñeʼẽ$
^tee/tee$
^Mercosur-pe/Mercosur+pe$
^,/,$
^karaiñeʼẽ/karaiñeʼẽ$
^ha/ha$
^poytugañeʼẽ/poytugañeʼẽ$
^ykére/ykére$
^./.$
```
And the output of the analyser for e.g. `poytugañeʼẽ` is:
```
^poytugañeʼẽ/poytugañeʼẽ/a+poytugañeʼẽ/re+poytugañeʼẽ$^./.$
```
So, the analyses should be weighted
```
poytugañeʼẽ : poytugañeʼẽ = 1.0
poytugañeʼẽ : a+poytugañeʼẽ = 0.0
poytugañeʼẽ : re+poytugañeʼẽ = 0.0
```
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.