Possibly incorrect tests
- Dominant language
- Java
- Stars
- 4
- Forks
- 9
- PR merge metrics
- No merged PRs in 30d
Description
There are currently 7 ignored tests.
[testRefinementByNiels](https://github.com/Ecdar/j-Ecdar/blob/984419562b7e4ef693abc6ea883edb9674e57527/test/features/BoolTest.java#L533): Is non-deterministic. The CI nearly always have this test pass, but on my local development PCs it fail 90% of the time. The assertions which fails are ``assert(new SimpleTransitionSystem(auts[1]).isFullyConsistent());`` and ``assert(new SimpleTransitionSystem(auts[1]).isLeastConsistent());``. When removing the ``assert(new SimpleTransitionSystem(auts[1]).isDeterministic());`` they still fail meaning that the ``isDeterminsitic`` check does not alter the state of the automaton.
[Z2RefinesZ2Z3Z4](https://github.com/Ecdar/j-Ecdar/blob/984419562b7e4ef693abc6ea883edb9674e57527/test/features/DelayRefinementTest.java#L324): ``Z2 <= Z2 \\ Z3``. Fails refinement because ``Z2`` has output ``o``and ``Z2 \\ Z3`` has the same as input. Both ``Z2`` and ``Z3`` have no inputs and only ``o`` as an output. As the quotient inputs are ``Act_i^T ∪ Act_o^S`` ``o`` will always be an input of the quotient. Either this test is incorrect or the channel ``o`` should be distinct between ``Z2`` and ``Z3``.
[testBoolSafeLoadXML](https://github.com/Ecdar/j-Ecdar/blob/984419562b7e4ef693abc6ea883edb9674e57527/test/features/BoolTest.java#L612): Relies on files from the ``testOutput`` directory which cannot be found when running the tests.
[T0RefinesT3T1T2](https://github.com/Ecdar/j-Ecdar/blob/984419562b7e4ef693abc6ea883edb9674e57527/test/features/DelayRefinementTest.java#L416): Fails on two refinements ``(T3 \\ T4) \\ T2 <= (T3 \\ T4) \\ T2`` and ``T1 <= (T3 \\ T4) \\ T2``. The first assertion fails because of ``Output violation`` and the second because of a ``Delay violation``.
[newQuotientTest5](https://github.com/Ecdar/j-Ecdar/blob/984419562b7e4ef693abc6ea883edb9674e57527/test/features/UniversitySimpleTest.java#L172): Fails because the file ``comp.xml`` cannot be found.
[testFromTestFramework1](https://github.com/Ecdar/j-Ecdar/blob/984419562b7e4ef693abc6ea883edb9674e57527/test/features/UniversityTest.java#L245): Fails refinement on ``Machine <= ((((Adm2 && HalfAdm1) || Machine || Researcher) \\ (Adm2 && HalfAdm2)) \\ Researcher)`` but a test passes on ``Machine <= (Machine || ((Adm2 && HalfAdm1) || Researcher)) \\ ((Adm2 && HalfAdm2) || Researcher)``.
[testFromTestFramework2](https://github.com/Ecdar/j-Ecdar/blob/984419562b7e4ef693abc6ea883edb9674e57527/test/features/UniversityTest.java#L275): Fails refinement on ``Administration <= (Spec \\ Machine) \\ Researcher`` but a test passes on ``Administration <= (Spec \\ (Machine || Researcher))``.
Contributor guide
No contributing guide indexed for this repository
Research direction
Start by running the ignored tests listed in test/features/BoolTest.java, DelayRefinementTest.java, UniversitySimpleTest.java, and UniversityTest.java, then compare the reported failures with the referenced assertions and missing testOutput or comp.xml files. Determine for each test whether the test, fixture, or implementation is incorrect, and finish with the seven tests no longer incorrectly ignored or with their expected behavior documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- java
- Domain
- testing-qa
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100