mapbox / mapbox/node-cpp-skel

Blue sky: separate coverage reports for unit test + benchmarks

Abierto
#62 0 comentarios 1 reacción 0 asignados Ver en GitHub

Nadie ha tomado este issue todavía.

Lenguaje dominante
C++
Estrellas
74
Forks
9
Métricas de merge de PR
Sin PR fusionados en 30 d

Descripción

## Context

We use https://codecov.io for online reports of code coverage. We consider code coverage a critical tool in ensuring robust code since it:

- Allows you to quickly see what code is actually run in your tests (to help find either dead code or code needing tests)
- Helps you be motivated to write tests
- Coverage shows how many times code is executed, by line: this can be a really helpful indication of where performance bottlenecks might be.

Currently we have coverage reporting available, by file and by line, that reflects just one metric: what code was executed when running the unit tests. This is viewable at https://codecov.io/gh/mapbox/node-cpp-skel/tree/620a268920ca47ca3bc49dd96fd8839e80774411/src.

## Opportunity

@GretaCB is working on benchmarking scripts in #61. While the current code coverage answers the question of "what code do I have unit tests for?" it could also be useful, after #61 lands, to answer the question of "what code do I have benchmarks for?".

If the answer were: "only for some of the performance critical code and not all the performance critical code" **_then we have a problem_**. One of the easiest mistakes to make in performance optimization is to spend time optimizing the wrong thing. Performance optimization is hard enough when you are focused on the right code. So we should use all the tools we have to try to avoid this issue.

## Solution

Codecov has a feature called `flags`. This allows you to mark a specific coverage upload by name. We could use this to provide display of coverage isolated to our unit tests vs our benchmark scripts. See more at https://docs.codecov.io/docs/flags

Guía de contribución

Abrir la guía de contribución

Primeros pasos

  1. Lee el issue completo y luego la guía de contribución del proyecto.
  2. Comenta en el issue que vas a ocuparte — evita que dos personas hagan lo mismo.
  3. Haz un fork del repositorio y trabaja en una rama.
  4. Abre un pull request que haga referencia al número del issue.

Línea de trabajo

Revisa el trabajo de benchmark descrito en issue #61 junto con la cobertura existente de los unit tests y la documentación de los flags de Codecov. Define cargas de cobertura separadas para los unit tests y los scripts de benchmark, y verifica después que Codecov muestre los dos informes de forma independiente.

Escrito por el modelo de indexación a partir del texto del issue.

Evaluación

Stack tecnológico
cpp
Área
performance, testing
Tipo de issue
Nueva funcionalidad
Dificultad
4/5
Tiempo estimado
3-5 días
Estado de actividad
Estancado
Claridad
Bastante claro
Aptitud para principiantes
35/100

Recibe los nuevos issues en tu correo

Un resumen breve de issues de GitHub para principiantes.