JuliaText / JuliaText/TextAnalysis.jl

Replacement function for list of stuff.

Open
#23 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Julia
Stars
384
Forks
92
PR merge metrics
No merged PRs in 30d

Description

We need some kind of thing where you input a dictionary or list and have words get replaced. This is important for standardizing synonyms, dates, and contractions. I think code like this make it easier to tokenize stuff especially since you eliminate apostrophes.

In python had code that replace contractions using regex.

'''
replacement_patterns = [
(r'(?i)won\'t', 'will not'),
(r'((?i)can\'t|(?i)can not)', 'cannot'),
(r'(?i)i\'m', 'i am'),
(r'(?i)ain\'t', 'is not'),
(r'(\w+)\'ll', '\g<1> will'),
(r'(\w+)n\'t', '\g<1> not'),
(r'(\w+)\'ve', '\g<1> have'),
(r'(\w+t)\'s', '\g<1> is'),
(r'(\w+)\'re', '\g<1> are'),
(r'(\w+)\'d', '\g<1> would'),
(r'\'cause', 'because'),]
'''
Not sure if it is possible to do this in julia

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.