Config-file discovery gets confused by custom command-line arguments

pytest looks in the tests' directory, or one of its parents directory, for a configuration file such as pytest.ini or tox.ini and uses the first one it finds. But how does pytest know which directory is the test directory? The relevant logic is in _pytest/config/findpaths.py, starting in determine_setup(). So for example if I run

pytest /some/directory/my/test.py

That logic determines that the test directory is "/some/directory/my". So far so good.

The problem starts when the user adds custom pytest arguments in conftest.py. For example, my project has in conftest.py:

def pytest_addoption(parser):
    parser.addoption('--scylla-path', action='store', default='',
        help='Path to the scylla excutable the tests are running against')

And people start a test with

pytest --scylla-path /some/path /some/directory/my/test.py

The problem now is that the logic in _pytest/config/findpaths.py looks at the command-line arguments, skips (in get_dirs_from_args()) the argument starting with "-" (in this example, --scylla-path) but then does not skip its parameter - /some/path. It then looks for the configuration file in /some/path, which is wrong - and in my case led pytest to find a broken configuration file in that directory and using it.

The ideal fix would be for get_dirs_from_args() to be called after the command line is parsed and the non-positional arguments are removed. I don't know if we can do this, or we have a chicken and egg problem of what gets read first.

Another possible fix is perhaps to first just look for a conftest.py (as we already do in _pytest/config/__init__.py), and if we find one (in my example, it's in in /some/directory/my, not /some/path) also look for pytest.ini in the same directory - NOT look again at all the directories in command line. In other words, it doesn't make too much sense (I think) to pick up conftest.py from one directory, but tox.ini from a different one, and since conftest.py is more important and more pytest-specific, it should be discovered first.

By the way, there is a workaround to solving my problem without fixing pytest at all - instead of running

pytest --scylla-path /some/path /some/directory/my/test.py

The user just needs to run

pytest --scylla-path=/some/path /some/directory/my/test.py

With an equals sign instead of a space. This works well because now the findpaths.py code skips the entire option, not just half of it. But I still think this needs a better fix, because a user might not remember to use an equals sign instead of a space, and if they do use a space, the resulting error message is very unhelpful (I got strange warnings coming from the definitions in a wrong tox.ini file, but without telling me which ini file this is coming from, or why this ini file was chosen).

Read _pytest/config/findpaths.py, starting with determine_setup() and get_dirs_from_args(), then compare the conftest.py discovery logic in _pytest/config/init.py. Reproduce the issue with a spaced custom option and a test path, then verify that the option's parameter is not treated as a test directory and that configuration is selected from the test directory.