coveragepy / coveragepy/coveragepy

Absolute paths with [run]relative_files

Open
#1,647 7 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

bug
Dominant language
Python
Stars
3.4k
Forks
525
Avg merge
18h 27m
Merged PRs (30d)
6

Description

Describe the bug
The [run]relative_files settings makes coverage store relative file paths in the data file. However this is only valid for files in or below the folder from which coverage is executed, i.e. the coverage current working folder.
Absolute paths are still used for files in or below adjacent folders.

To Reproduce
Please use the bash and Windows batch files below to reproduce the problem. Note that GitHub prevents attaching batch file.

#~ Cleanup.
rm -rf DEd4sf45
rm -rf .venv

#~ Create a virtual environment.
#~ Same behavior for 3.9-64, 3.10-64 and 3.11-64.
python -m venv .venv
. ./.venv/bin/activate
#~ Same behavior for version 5.5 and 6.5.0 and 7.2.7.
python -m pip install coverage
python --version --version
coverage --version

RUN_JOB () {
  #~ Create folders for code under test and test.
  mkdir -p $1/src/foo
  mkdir -p $1/test
  #~ Create environment.
  export PYTHONPATH=$PWD/$1/src
  #~ Create code under test.
  echo 'def foo():' > $1/src/foo/__init__.py
  echo '  pass' >> $1/src/foo/__init__.py
  #~ Create test.
  echo 'import foo' > $1/test/test.py
  echo 'foo.foo()' >> $1/test/test.py
  #~ Create the coverage configuration.
  echo '[run]' > $1/.coveragerc
  echo 'relative_files = True' >> $1/.coveragerc

  #~ Note that test code is normally not included in coverage measurement.
  echo source = $PWD/$1/src,$PWD/$1/test >> $1/.coveragerc
  #~ Setting source or include result in the same behavior.
  #~ echo include = $PWD/$1/src/foo/__init__.py,$PWD/$1/test/test.py >> $1/.coveragerc

  #~ Move in to the test folder, run the test and create the coverage report.
  cd $1/test
  coverage run --rcfile=../.coveragerc test.py
  #~ Note that the path to the code under test is absolute whereas the path to
  #~ the test code is relative.
  #~ The relative_files setting only applies for code in or below the folder from
  #~ which coverage is executed.
  #~ Note that relative_files seems to make no difference.
  coverage report --rcfile=../.coveragerc
  cd -
}

RUN_JOB 3df5grDR
RUN_JOB DEd4sf45

#~ Aggregate the coverage data.
mv 3df5grDR/test/.coverage DEd4sf45/test/.coverage.3df5grDR
mv DEd4sf45/test/.coverage .coverage.DEd4sf45
#~ CI Clears job data.
rm -rf 3df5grDR
#~ Combine the coverage data and create the report.
cd DEd4sf45/test
coverage combine --keep --rcfile=../.coveragerc
#~ Report fails on missing sources.
coverage report --rcfile=../.coveragerc
cd -

deactivate
@ECHO OFF

REM ~ Cleanup.
RMDIR /S /Q DEd4sf45
RMDIR /S /Q .venv

REM ~ Create a virtual environment.
REM ~ Same behavior for 3.9-64, 3.10-64.
py -3.11-64 -m venv .venv
CALL .venv\Scripts\activate
REM ~ Same behavior for version 5.5 and 6.5.0.
python -m pip install coverage
python --version --version
coverage --version

CALL :RUN_JOB 3df5grDR
CALL :RUN_JOB DEd4sf45

REM ~ Aggregate the coverage data.
COPY 3df5grDR\test\.coverage DEd4sf45\test\.coverage.3df5grDR
RENAME DEd4sf45\test\.coverage .coverage.DEd4sf45
REM ~ CI Clears job data.
RMDIR /S /Q 3df5grDR
REM ~ Combine the coverage data and create the report.
PUSHD DEd4sf45\test
coverage combine --keep --rcfile=..\.coveragerc
REM ~ Report fails on missing sources.
coverage report --rcfile=..\.coveragerc
POPD

GOTO:EOF

:RUN_JOB
REM ~ Create folders for code under test and test.
MKDIR %1\src\foo
MKDIR %1\test
REM ~ Create environment.
SET PYTHONPATH=%CD%\%1\src
REM ~ Create code under test.
ECHO def foo(): > %1\src\foo\__init__.py
ECHO   pass >> %1\src\foo\__init__.py
REM ~ Create test.
ECHO import foo > %1\test\test.py
ECHO foo.foo() >> %1\test\test.py
REM ~ Create the coverage configuration.
ECHO [run] > %1\.coveragerc
ECHO relative_files = True >> %1\.coveragerc

REM ~ Note that test code is normally not included in coverage measurement.
ECHO source = %CD%\%1\src,%CD%\%1\test >> %1\.coveragerc
REM ~ Setting source or include result in the same behavior.
REM ~ ECHO include = %CD%\%1\src\foo\__init__.py,%CD%\%1\test\test.py >> %1\.coveragerc

REM ~ Move in to the test folder, run the test and create the coverage report.
PUSHD %1\test
coverage run --rcfile=..\.coveragerc test.py
REM ~ Note that the path to the code under test is absolute whereas the path to
REM ~ the test code is relative.
REM ~ The relative_files setting only applies for code in or below the folder from
REM ~ which coverage is executed.
REM ~ Note that relative_files seems to make no difference.
coverage report --rcfile=..\.coveragerc
POPD
GOTO:EOF

Expected behavior
coverage must use relative paths for all files, not only files which are in or below the folder from which coverage is executed, i.e. coverage must use .. for relative paths to files which are in or below folders adjacent to the folder from which coverage is executed.

Additional context
NA

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by running the supplied Bash or Windows reproduction with [run] relative_files enabled, then compare paths produced by coverage run, report, and combine. Trace how paths outside the current working folder are stored and resolved. Done means adjacent-folder files use relative paths such as .. and the combined coverage report succeeds.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
testing
Issue type
Bug
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.