robotframework / robotframework/robotframework

Successfull completion of [Teardown] overwrites ${TEST_FAILED} built in variable

Open
#3,737 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
Python
Stars
11.9k
Forks
2.6k
Avg merge
1d 18h
Merged PRs (30d)
10

Description

If a testcase fails it sets the ${TEST_FAILED} built in variable to "true". If that same test case also has a [Teardown] tag and the keyword that the tag points to then runs successfully then it changes the value of the ${TEST_FAILED} from "true" to false.

The [Teardown] result should not change the fact that the test failed. I expect the following to happen:

Test | Teardown | ${TEST_FAILED}

Pass | Pass | false
Pass | Fail | true
Fail | Pass | true
Fail | Fail | true

Here is the code that demonstrates the problem

`Testcase 1
[Documentation] This will set ${TEST_FAILED} to true
should be equal as integers 0 1

Testcase 2
[Documentation] This should fail on the "Should be True" keyword and not get to the "Log" keyword
Should be True ${TEST_FAILED}==${false} msg=One or more tests failed. Don't run this test
Log ${TEST_FAILED}

Testcase 3
[Documentation] This will set ${TEST_FAILED} to true but then the Teardown overwrites the ${TEST_FAILED} to false (BAD!)
[Teardown] should be equal as intergers 0 0
should be equal as integers 0 1

Testcase 4
[Documentation] This should fail and not get to the "Log" keyword. However Testcase 3 overwrote the ${TEST_FAILED} so
now it passes when it should have failed on the first keyword
Should be True ${TEST_FAILED}==${false} msg=One or more tests failed. Don't run this test
Log ${TEST_FAILED}`

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the provided Testcase 1–4 reproduction and inspect how ${TEST_FAILED} is set after a test and its [Teardown] keyword. Done means the variable remains true whenever either the test or teardown fails, matching the expected result matrix.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
30/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.