Blue sky: separate coverage reports for unit test + benchmarks
Nessuno ha ancora preso questa issue.
- Lingua principale
- C++
- Stelle
- 74
- Fork
- 9
- Metriche di merge delle PR
- Nessuna PR unita negli ultimi 30g
Descrizione
Context
We use https://codecov.io for online reports of code coverage. We consider code coverage a critical tool in ensuring robust code since it:
- Allows you to quickly see what code is actually run in your tests (to help find either dead code or code needing tests)
- Helps you be motivated to write tests
- Coverage shows how many times code is executed, by line: this can be a really helpful indication of where performance bottlenecks might be.
Currently we have coverage reporting available, by file and by line, that reflects just one metric: what code was executed when running the unit tests. This is viewable at https://codecov.io/gh/mapbox/node-cpp-skel/tree/620a268920ca47ca3bc49dd96fd8839e80774411/src.
Opportunity
@GretaCB is working on benchmarking scripts in #61. While the current code coverage answers the question of "what code do I have unit tests for?" it could also be useful, after #61 lands, to answer the question of "what code do I have benchmarks for?".
If the answer were: "only for some of the performance critical code and not all the performance critical code" then we have a problem. One of the easiest mistakes to make in performance optimization is to spend time optimizing the wrong thing. Performance optimization is hard enough when you are focused on the right code. So we should use all the tools we have to try to avoid this issue.
Solution
Codecov has a feature called flags. This allows you to mark a specific coverage upload by name. We could use this to provide display of coverage isolated to our unit tests vs our benchmark scripts. See more at https://docs.codecov.io/docs/flags
Guida per i contributori
Apri la guida per i contributori
Come iniziare
- Leggi tutta la issue e poi la guida ai contributi del progetto.
- Commenta sulla issue per dire che te ne occupi tu — evita che due persone facciano lo stesso lavoro.
- Fai un fork del repository e lavora su un branch.
- Apri una pull request che faccia riferimento al numero della issue.
Direzione di ricerca
Esamina il lavoro sui benchmark descritto in issue #61 insieme alla reportistica esistente sulla coverage degli unit test e alla documentazione dei flags di Codecov. Definisci upload di coverage separati per gli unit test e gli script di benchmark, quindi verifica che Codecov visualizzi i due report indipendentemente.
Scritto dal modello di indicizzazione a partire dal testo della issue.
Valutazione
- Stack tecnologico
- cpp
- Ambito
- performance, testing
- Tipo di issue
- Funzionalità
- Difficoltà
- 4/5
- Tempo stimato
- 3-5 giorni
- Stato di attività
- Ferma
- Chiarezza
- Abbastanza chiara
- Idoneità per principianti
- 35/100