ocaml / ocaml/dune

[coq] [coq-lang-1.0] Ensure test suite is comprehensive.

Open
#7,598 5 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

rocq
Dominant language
OCaml
Stars
1.9k
Forks
500
Avg merge
15h 21m
Merged PRs (30d)
277

Description

Before (coq lang 1.0) we need to ensure that we are testing all possible configurations of Coq Dune projects.

It seems there are quite a few, let's study the variables:

  • composition: single theory, multiple same-scope ones, multiple theories in multiple workspaces
  • Coq: not present, in workspace, installed
  • stdlib present: no, in workspace, installed
  • user-contrib: no, installed
  • coqpath: yes / no
  • stdlib flag: yes / no
  • theories visibility: public and private, can depend in 3 different ways x 2 different configurations (same / different scope)
  • native: enabled / disabled in config, manually overriden.
  • plugins: no / yes single module / yes depending on other libs
  • plugins: living in coqpath without findlib registration / with findlib registration
  • transitive deps: both in worksspace and in user-contrib
  • Coq versions: maybe 8.13 / 8.16 / 8.17 ?
  • library names: different kind of overlappings

That's a lot of combos to test, but can be reduced with a bit of care.

Another point to consider is how to have a naming scheme that help us identify what part of the test matrix the test is covering.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by surveying the existing Coq Dune test suite and its current organization. Map the configuration dimensions listed in the issue, identify which combinations are already covered, and propose a naming scheme that shows the covered matrix. Done means the relevant configurations are deliberately covered without unnecessary combinations.

Written by the indexing model from the issue text.

Assessment

Tech stack
ocaml
Domain
build-system, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.