rust-lang / rust-lang/libc

Improve integration testing in ctest.

Open
#5,226 2 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

ctest
Dominant language
Rust
Stars
2.6k
Forks
1.3k
Avg merge
1d 22h
Merged PRs (30d)
69

Description

Currently, the tests folder in ctest is very cluttered. The input folder contains not only the inputs to the tests, but also the pregenerated expected outputs as well, making it difficult to work with.

Additionally, all the tests are in one file, which was fine when we started but now that we have more tests and specifically tests that do different things (like fail), it is not a good idea.

Finally, the current integration testing infrastructure uses a lot of unwraps, which makes it difficult to reuse the existing functions like check_entrypoint for making tests that expect failure.

The most recent test added for example, test_mismatched_union_field, would benefit greatly if we could just call check_entrypoint, assert that it failed, and also assert that it specifically failed due to cc errors.

This issue is made to get an idea about how the tests folder should be laid out.

To me, the obvious things are to seperate the pregenerated outputs into a seperate output/ folder, move all of the testing infrastructure to a common directory, and for now at least, separate the tests into two files, basic.rs an failures.rs, where the latter only has tests that are expected to fail.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start in the tests folder and review how input files, pregenerated expected outputs, and the shared testing infrastructure are currently organized. Use test_mismatched_union_field, check_entrypoint, and the existing unwrap-based setup as the main examples. Done means separating outputs into output/, moving shared infrastructure to a common directory, and splitting tests into basic.rs and failures.rs with failure-specific assertions.

Written by the indexing model from the issue text.

Assessment

Tech stack
rust
Domain
testing-qa
Issue type
Refactor
Difficulty
4/5
Estimated time
3-5 days
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.