coveragepy / coveragepy/coveragepy
Expected performance of coveragepy?
Nobody has claimed this yet.
- Dominant language
- Python
- Stars
- 3.4k
- Forks
- 525
- Avg merge
- 18h 27m
- Merged PRs (30d)
- 6
Description
I wonder what is the expected performance of coverage? In my case it looks like some tests run 2x slower than without coverage
I am using python 3.12 with standard unittest framework.
Also, I noticed there is kind of big overhead before anything starts running, when you enable coverage. My app starts in about 1s, but if I enable coverage it starts running tests after about ~13s.
Here is a small sample of tests that are run without coverage:
2024-09-26 08:48:12,662 1 INFO odoodb odoo.addons.base.models.ir_attachment: filestore gc 0 checked, 0 removed
2024-09-26 08:48:12,662 1 INFO odoodb odoo.service.server: 8 post-tests in 3.11s, 1804 queries
2024-09-26 08:48:12,662 1 INFO odoodb odoo.tests.stats: base_env_meta: 12 tests 0.12s 118 queries
2024-09-26 08:48:12,662 1 INFO odoodb odoo.tests.stats: reports_project: 16 tests 3.11s 1804 queries
2024-09-26 08:48:12,662 1 INFO odoodb odoo.tests.stats: sale_importer_project: 9 tests 0.73s 286 queries
2024-09-26 08:48:12,662 1 INFO odoodb odoo.tests.stats: versatil: 43 tests 0.36s 230 queries
2024-09-26 08:48:12,662 1 INFO odoodb odoo.tests.result: 0 failed, 0 error(s) of 50 tests when loading database 'odoodb'
2024-09-26 08:48:12,662 1 INFO odoodb odoo.service.server: Initiating shutdown
2024-09-26 08:48:12,662 1 INFO odoodb odoo.service.server: Hit CTRL-C again or send a second signal to force the shutdown.
2024-09-26 08:48:12,706 1 INFO odoodb odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 2 connections
Here same tests with coverage:
2024-09-26 08:50:09,710 1 INFO odoodb odoo.service.server: 8 post-tests in 5.98s, 1804 queries
2024-09-26 08:50:09,711 1 INFO odoodb odoo.tests.stats: base_env_meta: 12 tests 0.22s 118 queries
2024-09-26 08:50:09,711 1 INFO odoodb odoo.tests.stats: reports_project: 16 tests 5.97s 1804 queries
2024-09-26 08:50:09,711 1 INFO odoodb odoo.tests.stats: sale_importer_project: 9 tests 1.01s 286 queries
2024-09-26 08:50:09,711 1 INFO odoodb odoo.tests.stats: versatil: 43 tests 0.42s 230 queries
2024-09-26 08:50:09,711 1 INFO odoodb odoo.tests.result: 0 failed, 0 error(s) of 50 tests when loading database 'odoodb'
2024-09-26 08:50:09,711 1 INFO odoodb odoo.service.server: Initiating shutdown
2024-09-26 08:50:09,711 1 INFO odoodb odoo.service.server: Hit CTRL-C again or send a second signal to force the shutdown.
2024-09-26 08:50:10,044 1 INFO odoodb odoo.sql_db: ConnectionPool(used=0/count=0/max=64): Closed 2 connections
All tests run slower especially ones with lots of queries. Note, odoo is tightly coupled with database engine (postgres), so there are lots of tests that depend on database. Not sure if its relevant here.
My .coveragerc:
[run]
include =
/opt/odoo/projects/monodoo/*
/opt/odoo/projects/custom/*
omit =
**/tests/*
**/migrations/*
[report]
# Regexes for lines to exclude from consideration
exclude_lines =
# Have to re-enable the standard pragma
pragma: no cover
# Don't complain if non-runnable code isn't run:
if __name__ == .__main__.:
I run coverage like this:
os.execvp(cmd, cmd_args)
Where cmd is coverage and cmd_args = ['coverage', 'run', '/opt/odoo/odoo/odoo-bin', '-d', 'odoodb', '--stop-after-init', '--test-enable', '-u', 'my-module']
P.S. I also run it via docker container.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by reproducing the timings with Python 3.12, the shown .coveragerc, and the coverage run command against the reported unittest/Odoo sample. Compare startup and query-heavy test durations with and without coverage, then determine whether the observed overhead is expected or indicates a coveragepy performance issue; the issue provides no source file or test to modify.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- postgresql, python
- Domain
- databases, performance, testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100