microsoft / microsoft/BotFramework-WebChat
[Test reliability] Eliminate false positive and reduce wait time on verifying speech recognition is not restarted
Nobody has claimed this yet.
- Dominant language
- HTML
- Stars
- 1.8k
- Forks
- 1.6k
- Avg merge
- 22h 58m
- Merged PRs (30d)
- 2
Description
Feature Request
tl;dr we are waiting for "that signal should NOT be fired". This can introduce false positives and unnecessary wait time for positives.
There is a test reliability issue for turning on microphone after input hints:
- After receiving input hint of
expecting, we will turn on microphone - We use Redux +
render()to turn on microphone
Combining both in a busy environment, the microphone may not turn on immediately. We need to wait for the condition, more-or-less:
became(() => speechRecognitionStartCalled(), ...)
As we also test for negative case: after receiving input hint of ignoring, we should NOT turn on the microphone. But unfortunately, we don't have any signal to detect if Web Chat decided NOT to turn on microphone.
So, we are waiting for X seconds to make sure the speech recognition is not restarted. This is not great for a few reasons
- False positives: X seconds may not be long enough to make sure speech recognition is not started
- In positive case, we lengthened the tests
To fix this bug, add/find a good signal: after receiving input hint of ignoring, wait for a signal from Web Chat that it decided not to fire the speech recognition signal.
The following is a code snippet we were updating our test. This needs to further updated to make sure we have a good signal to make the last line faster.
test('should not turn on microphone if initiated via typing', async () => {
const { driver, pageObjects } = await setupWebDriver({
props: {
webSpeechPonyfillFactory: () => window.WebSpeechMock
}
});
await driver.wait(uiConnected(), timeouts.directLine);
await pageObjects.sendMessageViaMicrophone('hint expecting');
await driver.wait(minNumActivitiesShown(2), timeouts.directLine);
- await expect(speechRecognitionStartCalled().fn(driver)).resolves.toBeFalsy();
+ await expect(() => driver.wait(speechRecognitionStartCalled(), timeouts.ui)).rejects.toThrow('Waiting SpeechRecognition.start to be called');
});
[feature-request]
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start with the test named "should not turn on microphone if initiated via typing" and inspect setupWebDriver, speechRecognitionStartCalled, and the input-hint flow. Determine which Web Chat signal can confirm that speech recognition will not restart, then update the test to wait on that signal instead of a fixed delay. Done means the negative case avoids false positives and unnecessary waiting.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript
- Domain
- frontend, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100