No Recognition Period when starting recognizer

There seems to be a short time period about 3 seconds where there is no recognition ability between stoping the mic and starting it. For example as soon as you press start mic, and speak into the asr, if you speak within the first 3 seconds, there is no decoding. Any insight on what to change to improve this delay?