Macaulay2 / Macaulay2/M2

Speeding up running the tests

Open
#1,708 4 comments 0 reactions 1 assignee View on GitHub

@mahrud is already working on this.

Since Dec 17, 2020.

contributions welcome Core Documentation
Dominant language
Macaulay2
Stars
435
Forks
297
Avg merge
4d 20h
Merged PRs (30d)
11

Description

Here is a test:
```m2
i1 : elapsedTime check_1 "Macaulay2Doc"
-* running test 1 of package Macaulay2Doc in file:
../../Macaulay2/packages/Macaulay2Doc/doc9.m2:230:1:
rerun with: check_1 "Macaulay2Doc" *-
-- 9.86182 seconds elapsed
```
And the content of the test that took almost 10 seconds? Here it is:
https://github.com/Macaulay2/M2/blob/afaa83b5c6062dcbfc7f383bf4786d17ed0a0c3d/M2/Macaulay2/packages/Macaulay2Doc/doc9.m2#L227-L229
This is _objectively_ bad.

Here is the reason: when `(check, ZZ, String)` is called, two things happen:
1. `needsPackage (pkg, LoadDocumentation => true)`
https://github.com/Macaulay2/M2/blob/afaa83b5c6062dcbfc7f383bf4786d17ed0a0c3d/M2/Macaulay2/m2/testing.m2#L46
2. `loadPackage(pkg#"pkgname", LoadDocumentation => true, Reload => true)`
https://github.com/Macaulay2/M2/blob/afaa83b5c6062dcbfc7f383bf4786d17ed0a0c3d/M2/Macaulay2/m2/testing.m2#L31

Loading the documentation is necessary only because some authors have added package tests after `beginDocumentation()`. I see the appeal of this, especially since you can clearly put the tests near their respective documentation:
```m2
document{ key => "2 + 2",
EXAMPLE "2 + 2",
TEST "assert(2 + 2 = 4)"}
```
However, at least for `Macaulay2Doc`, this is slowing down running the tests **very** significantly, as the example above demonstrates.

Here are the two solutions I see:
1. use `capture` for tests instead (see #1689)
2. remove `LoadDocumentation => true` in `check` and enforce that all `TEST`s must appear before `beginDocumentation`
3. do one of the suggestions above, but only for Macaulay2Doc

@mikestillman @DanGrayson any thoughts on either of these?

This is somewhat a companion to #1480, as it speeds up testing of Macaulay2Doc very very significantly!

Related: #992.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.