How to train/modify collocation model with existing (ngram) dictionary? (question)

Dear Dmitriy,

thank you again for solving issue #218 concerning replacement of terms by multiple synonyms. I now have a question concerning how to best incorporate dictionaries that include information on ngrams/collocations, e.g., city names. A standard solution would be to simply replace all matched patterns in the text by the dashed_version_of_patterns, e.g., via `stri_replace_all`. However, this is slow for large corpora and I am interested how you would solve this task in text2vec.

As a workaround, I trained a collocation model on a modified dictionary containing all terms bound by dashes leaving the first unigram unbound so the model sees only one prefix and suffix, e.g, "new york_city". Please, see below code example.

I was wondering if you would incorporate such dictionary information differently, e.g., without training a model and manually defining colloaction_stats or so.

I would appreciate your thoughts. Thank you in advance.

```
library(text2vec)

txt <- c("new york city", "new york city district in new york", "san francisco")

dict_ngrams <- c("new york", "san francisco", "new york city", "the state of new york city", "city district")

#modify dict for limiting to one prefix/suffix
dict_ngrams_dashed <- gsub(" ", "_", dict_ngrams)
dict_ngrams_dashed <- sub("_", " ", dict_ngrams_dashed)

#train model based on dict
cc_model <- Collocations$new(collocation_count_min = 1
,pmi_min = 0
,gensim_min = 0
,lfmd_min = -Inf
)

it_dict_dashed <- itoken(dict_ngrams_dashed, progressbar = FALSE)
cc_model$partial_fit(it_dict_dashed)
# cc_model$collocation_stat
# prefix suffix n_i n_j n_ij pmi lfmd gensim rank_pmi rank_lfmd rank_gensim
# 1: city district 1 1 1 3.321928 -3.321928 0 1 1 3
# 2: the state_of_new_york_city 1 1 1 3.321928 -3.321928 0 2 2 4
# 3: san francisco 1 1 1 3.321928 -3.321928 0 3 3 5
# 4: new york_city 2 1 1 2.321928 -4.321928 0 4 4 1
# 5: new york 2 1 1 2.321928 -4.321928 0 5 5 2

it_txt <- itoken(txt, progressbar = FALSE)
it_txt_cc <- cc_model$transform(it_txt)

v <- create_vocabulary(it_txt_cc)
#side effect of cc model that might be interesting: "city_district" is ranked lower than "new_yor_city"
#and thus the latter is preferred over the former , see below
v
# Number of docs: 3
# 0 stopwords: ...
# ngram_min = 1; ngram_max = 1
# Vocabulary:
# term term_count doc_count
# 1: district 1 1
# 2: in 1 1
# 3: new_york 1 1
# 4: san_francisco 1 1
# 5: new_york_city 2 2
```