greta-dev / greta-dev/greta

Retire touchstone, and keep benchmark results in greta.benchmarks

Open
#836 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
C++
Stars
607
Forks
67
Avg merge
3d 8h
Merged PRs (30d)
1

Description

We don't really use the [`touchstone/`](https://github.com/greta-dev/greta/tree/e8563dae0ef2b8be384fac023f62df1febf17ae6/touchstone) approach to doing benchmarks.

We should:

- Delete `touchstone/`, `.github/workflows/touchstone-receive.yaml` and `.github/workflows/touchstone-comment.yaml`.
- Move benchmarking to [greta.benchmarks](https://github.com/greta-dev/greta.benchmarks), which alreadyhas:
- a [standing suite](https://github.com/greta-dev/greta.benchmarks/tree/main/suite) run locally across branches with [{cross}](https://github.com/DavisVaughan/cross), over models from greta's own `inst/examples/`
- dated run directories for one-off questions, each holding the script, the raw results and a rendered write-up that a `NEWS.md` entry can link to

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the touchstone/ directory and the two named workflow files: .github/workflows/touchstone-receive.yaml and .github/workflows/touchstone-comment.yaml. Then inspect the greta.benchmarks suite and its dated run directories, including the greta inst/examples models. Done means the old touchstone files are removed and benchmark scripts, raw results, and write-ups are kept in greta.benchmarks for linking from NEWS.md.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, r
Domain
ci-cd, performance
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
58/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.