JuliaText / JuliaText/TextAnalysis.jl
Replacement function for list of stuff.
- Dominant language
- Julia
- Stars
- 384
- Forks
- 92
- PR merge metrics
- No merged PRs in 30d
Description
We need some kind of thing where you input a dictionary or list and have words get replaced. This is important for standardizing synonyms, dates, and contractions. I think code like this make it easier to tokenize stuff especially since you eliminate apostrophes.
In python had code that replace contractions using regex.
'''
replacement_patterns = [
(r'(?i)won\'t', 'will not'),
(r'((?i)can\'t|(?i)can not)', 'cannot'),
(r'(?i)i\'m', 'i am'),
(r'(?i)ain\'t', 'is not'),
(r'(\w+)\'ll', '\g<1> will'),
(r'(\w+)n\'t', '\g<1> not'),
(r'(\w+)\'ve', '\g<1> have'),
(r'(\w+t)\'s', '\g<1> is'),
(r'(\w+)\'re', '\g<1> are'),
(r'(\w+)\'d', '\g<1> would'),
(r'\'cause', 'because'),]
'''
Not sure if it is possible to do this in julia
Contributor guide
No contributing guide indexed for this repository
Assessment
This issue has not been assessed yet.