baidu / baidu/lac

测试自定义词典分词效果较差

Open
#232 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
4k
Forks
592
PR merge metrics
No merged PRs in 30d

Description

case示例:
```
from LAC import LAC
l = LAC(mode='seg')
l.add_word("广东省/n")
l.add_word("人/n")
l.add_word("民/n")
l.add_word("人民/n")
l.add_word("民政/n")
l.add_word("政府/n")
```
```
l.run("人民政府")
['人', '民', '政府']
```
期望得到 ['人民', '政府']
我观察在源码 `parse_customization` 中仅根据前缀树来修正,有办法引入概率或者其他方式来解决优化吗?

Contributor guide

No contributing guide indexed for this repository

Research direction

Reproduce the reported case through the Python LAC API and inspect the parse_customization path that currently applies prefix-tree corrections. Compare the output for the registered words with the expected ['人民', '政府']; done means custom dictionary segmentation handles this case without regressing other segmentation behavior.

Written by the indexing model from the issue text.

Assessment

Tech stack
cpp, python
Domain
machine-learning
Issue type
Bug
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.