kedro-org / kedro-org/kedro-plugins
Investigate flaky tensorflow tests
- Dominant language
- Python
- Stars
- 119
- Forks
- 136
- Avg merge
- 4d 10h
- Merged PRs (30d)
- 5
Description
## Description
Follow up on #363 and https://github.com/kedro-org/kedro-plugins/pull/377
"Thinking about this, wonder if it makes sense to use something like https://github.com/box/flaky? It's used fairly widely, but it's not that actively-maintained (not that need new functionality). I'm just not sure if we even want the tests slowed down rerunning the test X times."
## Context
In https://github.com/kedro-org/kedro-plugins/pull/363 we added some logic to skip the tensorflow tests on windows, because they started to fail randomly. Re-running would occasionally "fix" it.
In https://github.com/kedro-org/kedro-plugins/pull/377, a change was added to run tensorflow tests separately from the other tests, because when run as part of the full test suite they failed. Running locally and on Gitpod was always successful.
Contributor guide
Research direction
Start by reviewing issue #363 and pull requests #363 and #377 to understand the existing Windows skip and separate TensorFlow test execution. Reproduce the flaky TensorFlow tests in the full suite and in isolation, then determine a stable testing approach and verify that the TensorFlow tests no longer fail intermittently.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python, tensorflow
- Domain
- machine-learning, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 30/100