pytest-dev / pytest-dev/pytest-bdd
parsers.parse failed to differentiate two "When" statements which start with the same few words
Open
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 1.5k
- Forks
- 251
- Avg merge
- 43m
- Merged PRs (30d)
- 2
Description
Environment:
WSL 2 x86_64 GNU/Linux: version = "20.04.5 LTS (Focal Fossa)"
pytest==7.3.0
pytest-bdd==6.1.1
Python version: 3.8.10
Reproducing the bug:
Notice how the two different "When" statements in the new_feature.feature file start with the same few words. The parsers failed to differentiate the two statements.
test_new_feature.py
import json
from pytest import fixture
from pytest_bdd import given, when, then, scenarios, parsers
scenarios("new_feature.feature")
CONVERTERS = {
"req_body": lambda x: x,
"response": lambda x: x,
"is_processed": lambda x: x,
"pings_service": lambda x: x,
}
@fixture()
@given(
parsers.parse("an API post request with a {req_body}"),
target_fixture="a_post_request",
converters=CONVERTERS,
)
def a_post_request(req_body):
request_params = {
"method": "POST",
"headers": {},
"body": json.dumps(req_body).encode(),
"url": "/v1/endpoint",
"params": {},
}
return request_params
@fixture
@when(
parsers.parse("the post request {is_processed}"),
target_fixture="handle_post_request",
converters=CONVERTERS,
)
def handle_post_request(a_post_request, is_processed):
return "mock_sqs_client", lambda x: x
@then(
parsers.parse("a {response} is returned"),
converters=CONVERTERS,
)
def assert_post_process_valid_response(handle_post_request, response):
assert True
####################################################################################################
@fixture
@when(
parsers.parse("the post request is sucessfully processed and the request {pings_service}"),
target_fixture="publish_service_message",
converters=CONVERTERS,
)
def publish_service_message(a_post_request, pings_service):
return "mock_sqs_client", lambda x: x
@then(
parsers.parse("in this case also a {response} is returned"),
converters=CONVERTERS,
)
def assert_valid_response(publish_service_message, response, a_post_request):
"""Test valid responses from api call"""
assert True
new_feature.feature
Feature: To reproduce bug of having two When clauses read as one even when they are different.
Scenario Outline: Outline 1
Given an API post request with a <req_body>
When the post request <is_processed>
Then a <response> is returned
Examples:
| response | is_processed | req_body |
| response_1 | request_1 | valid_domain |
| response_2 | request_2 | valid_domain |
Scenario Outline: Outline 2
Given an API post request with a <req_body>
When the post request is sucessfully processed and the request <pings_service>
Then in this case also a <response> is returned
Examples:
| pings_service | response | req_body |
| ping_succeeds | response_1 | request_1 |
| ping_failed | response_3 | request_1 |
Actual behaviour:
✗ pytest test_new_feature.py -v
==================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.8.10, pytest-7.3.0, pluggy-1.0.0 -- /home/bmak/.virtualenvs/misc-work/bin/python
cachedir: .pytest_cache
rootdir: /home/bmak/code/misc-work/brian/bdd_bug_github_20230411/paring_down_from_original_files
plugins: bdd-6.1.1
collected 4 items
test_new_feature.py::test_outline_1[response_1-request_1-valid_domain] PASSED [ 25%]
test_new_feature.py::test_outline_1[response_2-request_2-valid_domain] PASSED [ 50%]
test_new_feature.py::test_outline_2[ping_succeeds-response_1-request_1] FAILED [ 75%]
test_new_feature.py::test_outline_2[ping_failed-response_3-request_1] FAILED [100%]
========================================================================================================================= FAILURES ==========================================================================================================================
____________________________________________________________________________________________________ test_outline_2[ping_succeeds-response_1-request_1] _____________________________________________________________________________________________________
file /home/bmak/.virtualenvs/misc-work/lib/python3.8/site-packages/pytest_bdd/scenario.py, line 217
@pytest.mark.usefixtures(*func_args)
def scenario_wrapper(request: FixtureRequest, _pytest_bdd_example: dict[str, str]) -> Any:
file /home/bmak/code/misc-work/brian/bdd_bug_github_20230411/paring_down_from_original_files/test_new_feature.py, line 55
@fixture
@when(
parsers.parse("the post request is sucessfully processed and the request {pings_service}"),
target_fixture="publish_service_message",
converters=CONVERTERS,
)
def publish_service_message(a_post_request, pings_service):
E fixture 'pings_service' not found
> available fixtures: _pytest_bdd_example, a_post_request, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, handle_post_request, monkeypatch, publish_service_message, pytestbdd_stepdef_given_an API post request with a {req_body}, pytestbdd_stepdef_given_trace, pytestbdd_stepdef_then_a {response} is returned, pytestbdd_stepdef_then_in this case also a {response} is returned, pytestbdd_stepdef_then_trace, pytestbdd_stepdef_when_the post request is sucessfully processed and the request {pings_service}, pytestbdd_stepdef_when_the post request {is_processed}, pytestbdd_stepdef_when_trace, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
> use 'pytest --fixtures [testpath]' for help on them.
/home/bmak/code/misc-work/brian/bdd_bug_github_20230411/paring_down_from_original_files/test_new_feature.py:55
_____________________________________________________________________________________________________ test_outline_2[ping_failed-response_3-request_1] ______________________________________________________________________________________________________
file /home/bmak/.virtualenvs/misc-work/lib/python3.8/site-packages/pytest_bdd/scenario.py, line 217
@pytest.mark.usefixtures(*func_args)
def scenario_wrapper(request: FixtureRequest, _pytest_bdd_example: dict[str, str]) -> Any:
file /home/bmak/code/misc-work/brian/bdd_bug_github_20230411/paring_down_from_original_files/test_new_feature.py, line 55
@fixture
@when(
parsers.parse("the post request is sucessfully processed and the request {pings_service}"),
target_fixture="publish_service_message",
converters=CONVERTERS,
)
def publish_service_message(a_post_request, pings_service):
E fixture 'pings_service' not found
> available fixtures: _pytest_bdd_example, a_post_request, cache, capfd, capfdbinary, caplog, capsys, capsysbinary, doctest_namespace, handle_post_request, monkeypatch, publish_service_message, pytestbdd_stepdef_given_an API post request with a {req_body}, pytestbdd_stepdef_given_trace, pytestbdd_stepdef_then_a {response} is returned, pytestbdd_stepdef_then_in this case also a {response} is returned, pytestbdd_stepdef_then_trace, pytestbdd_stepdef_when_the post request is sucessfully processed and the request {pings_service}, pytestbdd_stepdef_when_the post request {is_processed}, pytestbdd_stepdef_when_trace, pytestconfig, record_property, record_testsuite_property, record_xml_attribute, recwarn, tmp_path, tmp_path_factory, tmpdir, tmpdir_factory
> use 'pytest --fixtures [testpath]' for help on them.
/home/bmak/code/misc-work/brian/bdd_bug_github_20230411/paring_down_from_original_files/test_new_feature.py:55
================================================================================================================== short test summary info ==================================================================================================================
FAILED test_new_feature.py::test_outline_2[ping_succeeds-response_1-request_1]
FAILED test_new_feature.py::test_outline_2[ping_failed-response_3-request_1]
================================================================================================================ 2 failed, 2 passed in 0.03s ================================================================================================================
(misc-work) ➜ paring_down_from_original_files git:(master) ✗
Expected behaviour:
Simply changing the first word of the second When statement from 'the' to 'that' makes all the tests pass.
✗ pytest test_new_feature.py -v
==================================================================================================================== test session starts ====================================================================================================================
platform linux -- Python 3.8.10, pytest-7.3.0, pluggy-1.0.0 -- /home/bmak/.virtualenvs/misc-work/bin/python
cachedir: .pytest_cache
rootdir: /home/bmak/code/misc-work/brian/bdd_bug_github_20230411/paring_down_from_original_files
plugins: bdd-6.1.1
collected 4 items
test_new_feature.py::test_outline_1[response_1-request_1-valid_domain] PASSED [ 25%]
test_new_feature.py::test_outline_1[response_2-request_2-valid_domain] PASSED [ 50%]
test_new_feature.py::test_outline_2[ping_succeeds-response_1-request_1] PASSED [ 75%]
test_new_feature.py::test_outline_2[ping_failed-response_3-request_1] PASSED
Contributor guide
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by running pytest test_new_feature.py -v against the reproducer in test_new_feature.py and new_feature.feature. Trace how the two overlapping When step definitions are matched, then add or update a regression test that keeps both statements distinct and passes without changing the feature wording.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- python
- Domain
- testing
- Issue type
- Bug
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Quiet
- Clarity
- Mostly clear
- Newbie friendliness
- 55/100