Syntax error in test file makes pytest exit with exit code 2

Ubuntu 18.04
pytest-4.0.2
python 2.7 and 3.6

Description

Having a test file with syntax error or import error results in pytest exiting with exitcode 2.

How to reproduce

1. Create a file named test_syntax_error.py with content

def x:
    pass

2. Run pytest and print exit code
   pytest test_syntax_error.py; echo $?

Expected

Probably worth a new exitcode as it doesn't really fit into those 5 documented

Actual

Exits with code 2, which stands for "Test execution was interrupted by the user"

Similarly, a file with content import non_existent will have the same effect.

This issue is similar to https://github.com/pytest-dev/pytest/issues/2832 and also
https://github.com/pytest-dev/pytest/issues/2950, but still I hope this will be useful for reference and example.

Start by reproducing the issue with the provided test_syntax_error.py example and the pytest command, then compare the resulting exit code with the documented exit codes. Trace how syntax and import errors are classified and determine the intended distinct outcome. Done means the behavior and documentation clearly distinguish these errors from user interruption, with tests covering both examples.