PyThaiNLP / PyThaiNLP/pythainlp
Ideas for pythainlp.lm function
Open
Nobody has claimed this yet.
enhancement
- Dominant language
- Python
- Stars
- 1.2k
- Forks
- 304
- Avg merge
- 2d 14h
- Merged PRs (30d)
- 14
Description
I think pythainlp.lm class should collect the function for doing preprocessing or post-processing Thai text from LLM and include a small language model that can run in computers for home users to do simple NLP jobs.
Preprocessing
pythainlp.lm.calculate_ngram_counts: Calculates the counts of n-grams in the list words for the specified range. #1054
Post-processing
pythainlp.lm.remove_repeated_ngrams: Remove repeated n-grams (to fixed lm) #1043
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reviewing the existing pythainlp.lm entry point and the related proposals for calculate_ngram_counts in #1054 and remove_repeated_ngrams in #1043. Clarify whether the goal is to collect preprocessing and post-processing utilities, add a small local language model, or both. Done requires an agreed scope and an implementation plan for the selected functionality.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- machine-learning
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100