Sphinx pronounciation dictionary with additional symbols?

It seems that in the current cmudict repository, the cmu dictionary uses additional symbols which are existing phonemes with some sort of numeric marker at the end:

From the current cmudict.dict:

invariant IH2 N V EH1 R IY0 AH0 N T
invasion IH2 N V EY1 ZH AH0 N

And from the included pronounciation-dictionary.dict:

invariant IH N V EH R IY AH N T
invasion IH N V EY ZH AH N

It seems that these additional symbols are defined in the accompanying symbols file or something? I'm not entirely familiar with this format. If you use a dictionary derived from the current cmudict (with the new symbols), you get totally invalid values. For example, with the built-in dictionary, you get 'one two three' for the examples/english.wav example, whereas with cmudict, you get hmm.

In addition to using the built-in language an accoustic models, I have tried using the latest acoustic and language models (specifically en-70k-0.1.lm.gz and cmusphinx-en-us-5.2.tar.gz), but neither works with the current format of cmudict.

What does work (in both situations) is to take the cmudict values and strip off the numbers from all the syllables. So:

1. Is there a way to specify that these additional syllables should be used? Passing a symbols file or something?
2. Should there be?
3. Is the speech recognition made worse by the lack of these numbers? I'm guessing they are used to indicate something.

I have prepared a minimal working example for this, which assumes you have a clone of (or symlink to) the cmudict and speech_recognition repos in your working directory, plus a "stripped version" of the cmudict (not required to run the script):

#! /usr/bin/env python3.7
import speech_recognition as sr

from pathlib import Path

CUR_DIR = Path(__file__).parent
CMU_PATH = CUR_DIR / 'cmudict'
SR_PATH = CUR_DIR / 'speech_recognition'
MODEL_PATH = SR_PATH / 'speech_recognition/pocketsphinx-data/en-US'

AUDIO_FILE = SR_PATH / 'examples/english.wav'

def get_language(pro_dict):
    acoustic_model = MODEL_PATH / 'acoustic-model'
    language_model = MODEL_PATH / 'language-model.lm.bin'
    rv = (acoustic_model, language_model, pro_dict)
    return tuple(map(str, map(Path.resolve, rv)))

r = sr.Recognizer()


with sr.AudioFile(str(AUDIO_FILE.resolve())) as source:
    ad = r.record(source)
    # Transcribe with the standard dictionary
    s_lang = get_language(MODEL_PATH / 'pronounciation-dictionary.dict')
    words_sd = r.recognize_sphinx(ad, language=s_lang)

    # Transcribe with the CMU dictionary
    c_lang = get_language(CMU_PATH / 'cmudict.dict')
    words_cd = r.recognize_sphinx(ad, language=c_lang)

    # Transcribe with a stripped version of the CMU dictionary
    cmu_stripped = CUR_DIR / 'cmudict_strip.dict'
    if cmu_stripped.exists():
        cs_lang = get_language(cmu_stripped)
        words_cds = r.recognize_sphinx(ad, language=cs_lang)
    else:
        words_cds = 'dict not found'

print(words_sd)
# one two three

print(words_cd)
# hmm

print(words_cds)
# one two three

To prepare the environment to run the above script, I did:

# Clone speech_recognition
git clone git@github.com:Uberi/speech_recognition.git

# Clone the cmudict
git clone git@github.com:cmusphinx/cmudict.git

# Checkout the commit I used explicitly
cd speech_recognition
git checkout 19dc36eb6a6173b500e2cc8cf2161ea2fe8cb891
cd ..

cd cmudict
2e7a88225cc129726aefcbd28911b5ebe40842da

Here is the script for converting the cmudict as formatted in the master branch of the GH repo to the "stripped" version I use in the above demo script

#! /usr/bin/env python3.7
from pathlib import Path

CUR_DIR = Path(__file__).parent

with open(CUR_DIR / 'cmudict/cmudict.dict', 'r') as fi:
    cmu_dict = {
        word: syllables for word, *syllables in
        (line.strip().split(' ') for line in fi if line)
    }

    with open(CUR_DIR / 'cmudict_strip.dict', 'w') as fo:
        for word, syllables in cmu_dict.items():
            line = ' '.join([word] + [syl.rstrip('123456789') for syl in syllables])
            print(line, file=fo)

System information

(Delete all the statements that don't apply.)

My system is Arch Linux

My Python version is 3.7.0

My Pip version is 18.0.

My SpeechRecognition library version is 3.8.1.

I don't have PyAudio installed.

I installed PocketSphinx from pip, with system swig, version is pocketsphinx==0.1.15

Run the supplied minimal example with examples/english.wav and compare recognize_sphinx using speech_recognition/pocketsphinx-data/en-US/pronounciation-dictionary.dict, cmudict.dict, and the stripped dictionary. Read the language and acoustic model paths in get_language, then verify how the numbered symbols in cmudict are handled. Done means establishing whether the library should support this dictionary format or clearly document the required compatibility behavior.