kedro-org / kedro-org/kedro-plugins

Investigate flaky tensorflow tests

Open
#367 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
119
Forks
136
Avg merge
4d 10h
Merged PRs (30d)
5

Description

## Description
Follow up on #363 and https://github.com/kedro-org/kedro-plugins/pull/377

"Thinking about this, wonder if it makes sense to use something like https://github.com/box/flaky? It's used fairly widely, but it's not that actively-maintained (not that need new functionality). I'm just not sure if we even want the tests slowed down rerunning the test X times."

## Context
In https://github.com/kedro-org/kedro-plugins/pull/363 we added some logic to skip the tensorflow tests on windows, because they started to fail randomly. Re-running would occasionally "fix" it.

In https://github.com/kedro-org/kedro-plugins/pull/377, a change was added to run tensorflow tests separately from the other tests, because when run as part of the full test suite they failed. Running locally and on Gitpod was always successful.

Contributor guide

Open the contributing guide

Research direction

Start by reviewing issue #363 and pull requests #363 and #377 to understand the existing Windows skip and separate TensorFlow test execution. Reproduce the flaky TensorFlow tests in the full suite and in isolation, then determine a stable testing approach and verify that the TensorFlow tests no longer fail intermittently.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, tensorflow
Domain
machine-learning, testing-qa
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.