benchmark-action / benchmark-action/github-action-benchmark

Fork opitmised for pytest-benchmark (upstream improvements?)

Aperta
#44 0 commenti 2 reazioni 0 assegnatari Vedi su GitHub
Lingua principale
TypeScript
Stelle
1.3k
Fork
183
Metriche di merge delle PR
Nessuna PR unita negli ultimi 30g

Descrizione

Hey, thanks for the great package! There was a number of additional data aspects I wanted to capture/utilise from pytest-benchmark (in particular test grouping) and so I created a fork optimised for this: https://github.com/aiidateam/github-action-benchmark, and you can see an example output here: https://aiidateam.github.io/aiida-core/dev/bench/ubuntu-18.04/django/

Now that I've just about finalised my implementation, I thought it would be good to feedback, since I sure at least some of this can be upstreamed to here 😄

I've outlined the changes in the README (and also the commit history is pretty readable), but I'll copy it here:

1. More metadata is saved about the benchmark run, for later comparison:
- CPU data (cores, processors, speed) is specifically saved, per commit, in the `cpu` key (uses [systeminformation](https://www.npmjs.com/package/systeminformation) npm package).
- The python version is extracted from the pytest-benchmark data in an `extra` key
- A `metadata` action option is available to save additional data
- This data is all shown in the data point's tooltip

2. The group name is saved for each test (see [pytest-benchmark markers](https://pytest-benchmark.readthedocs.io/en/latest/usage.html#markers)), or `null` if no group is given.
In the web-page rendering, tests are then arranged by the group they are in, which can be given a sub-heading, description, etc, and also a group can be "consolidated" into a single chart (with handling of differing data points).

3. A new `render-json-path` allows for a JSON file to be copied, which is used to configure the rendered web-page, e.g. for certain test suites and groups:

```json
{
"suites": {
"name": {
"header": "Test Suite Title",
"description": "Description of test suite."
}
},
"groups": {
"group1": {
"header": "Group 1 Title",
"description": "Description of group 1.",
"single_chart": true,
"xAxis": "date",
"backgroundFill": false
}
}
}
```

4. Split original `index.html` into multiple HTML/JS files in the `src/assets` folder. This allows for easier testing and development of the output web-page.
This has additionally allowed for:
- Adding `npm run serve`, for local development of output web-page (using [light-server](https://www.npmjs.com/package/light-server))
- Adding `overwrite-assets` option, to specify whether any existing assets should be overridden during a commit to `gh-pages`.

5. Improve formatting of charts:
- Color cycling for consolidated charts
- For legend, extract common test name prefix as title
- Data point tooltips: rounding values to 5 significant figures and better formatting of dates etc.
- Addition of the `chartjs-plugin-zoom` and a "Reset Zoom" button.

6. Add `commit-msg-append` option , useful for adding e.g. `[ci skip]` to commit message, but not having it as part of the test suite key in the data JSON.
7. Removed capture of commit author/committer, since it can be obtained from the commit id/url, and just bloats the data JSON.
8. Renamed `max-items-in-chart` -> `max-data-items`, to better describe its function of truncating the saved data during a commit.

Guida per i contributori

Apri la guida per i contributori

Direzione di ricerca

Inizia dal README, dall’index.html esistente e da src/assets, quindi confronta il fork collegato e la sua cronologia dei commit leggibile con l’action attuale. Identifica innanzitutto quali degli otto cambiamenti proposti rientrano nell’ambito e come dovrebbero essere testati. Il lavoro è concluso quando esiste un ambito upstream concordato, con la convalida dei dati di benchmark, delle pagine renderizzate e delle opzioni dell’action.

Scritto dal modello di indicizzazione a partire dal testo della issue.

Valutazione

Stack tecnologico
github-actions, python, typescript
Ambito
ci-cd, devops, web-dev
Tipo di issue
Funzionalità
Difficoltà
5/5
Tempo stimato
Più di una settimana
Stato di attività
Ferma
Chiarezza
Da chiarire
Idoneità per principianti
20/100

Ricevi le nuove issue nella tua casella

Un breve riepilogo di issue GitHub adatte ai principianti.