platformio / platformio/platformio-core

Option to delete intermediate output when running tests (and use less storage for CI builds in general)

Open
#4,883 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

feature unit testing
Dominant language
Python
Stars
9.5k
Forks
905
Avg merge
2d 13h
Merged PRs (30d)
2

Description

What kind of issue is this?

  • Feature Request.
    Start by telling us what problem you’re trying to solve. Often a solution
    already exists! Don’t send pull requests to implement new features without first getting our
    support. Sometimes we leave features out on purpose to keep the project small.

Configuration

Operating system:

PlatformIO Version (platformio --version): 6.1.14

Description of problem

Unit test builds produce statically linked output files of my application for every test:

13M	.pio/build/native_test/test/output/test_bytearray_rgb/main.o
13M	.pio/build/native_test/test/output/test_floatarray_exp_h/main.o
13M	.pio/build/native_test/test/output/test_floatarray_h/main.o
...
13M	.pio/build/native_test/test/output/test_tuple_tuple_hsv_float/main.o
13M	.pio/build/native_test/test/output/test_tuple_tuple_hsv_int/main.o
13M	.pio/build/native_test/test/output/test_tuple_tuple_rgb/main.o
676K	.pio/build/native_test/test/test_colours/main.o
1.6M	.pio/build/native_test/test/test_config/main.o
812K	.pio/build/native_test/test/test_ledprofile/main.o
1.4M	.pio/build/native_test/test/test_micropython/main.o
631M	total

This makes CI builds difficult because they use a lot of storage unnecessarily.

I'd like an option to delete intermediate output when running tests (and use less storage for CI builds in general).

Steps to Reproduce
  1. Download https://github.com/nomis/aurora-coriolis/tree/1.0.0 and run git submodule update --init --depth 1 (not --recursive)
  2. Comment out test_testing_command from app/pio/config.ini
  3. Run platformio test -e native_test
Actual Results

The total build output of .pio/build/native_test/ is 750MB+ because it has the main.o for every test program build.

Expected Results

The build output of .pio/build/native_test/ is ~120MB because it only has the most recent test program and not intermediate files.

Additional info

I'm running CI builds in RAM because it makes builds faster and avoids disk I/O for build output that will be discarded. There's not enough RAM dedicated to my CI host for this quantity of build output, especially when it's unnecessary.

It is difficult to do this with a wrapper script because obtaining the environment name for a custom test_testing_command with non-environment sections and sections that extend other sections is not trivial: #4480 #4531 #4882. Without the environment name the script wouldn't know where the current build output is to delete it, or where the test program is to run it.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Run platformio test -e native_test using the reproduction in the issue, then inspect app/pio/config.ini, the test_testing_command entry point, and the linked pio/testing-command.sh wrapper. Review issues #4480, #4531, and #4882 for environment-name handling. Done means an option removes unnecessary intermediate test output while retaining test execution and substantially reducing the .pio/build/native_test/ size.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
build-system, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
38/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.