E3SM-Project / E3SM-Project/cacts

Remove "baseline summary file" from baseline generation

Open
#40 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
0
Forks
0
Avg merge
9h 42m
Merged PRs (30d)
2

Description

This feature was added following the way eamxx does testing. The idea was to run all tests, and if everything passes, make the driving script copy all baseline files to the baseline folder.

In EAMxx we are moving toward putting the cmake code in charge of that, so that one can re-run the config/build/test commands manually and get the same behavior. In particular, one can use `add_test` to create a test that copies files over (if needed). This puts the responsibility of copying files on the project devs, rather than the testing system.

It also removes the hardcoded assumption (in cacts) that the baselines files are located in `${baseline_dir}/data`, since it will be up to the proj devs to copy the files. This way, they could even keep the same dir structure of the build folder into the baseline dir. CACTS should not assume that the baselines files are in `${baseline_dir}/data`. The structure of baselines is up to the project.

Contributor guide

No contributing guide indexed for this repository

Research direction

Start with the driving script responsible for baseline generation and the project’s CMake test definitions, especially the add_test usage described in the issue. Trace where the baseline summary file is created and where CACTS assumes files are under `${baseline_dir}/data`. Done means baseline copying is handled by project tests and CACTS no longer imposes that directory structure.

Written by the indexing model from the issue text.

Assessment

Tech stack
cmake, python
Domain
build-system, testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
48/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.