exercism / exercism/java-analyzer

Document guidelines on testing

Open
#123 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Java
Stars
13
Forks
27
Avg merge
4d 17h
Merged PRs (30d)
7

Description

I think we should come up with some guidelines on how analyzers should be tested. My proposal would be:

- Unit tests should exist to obtain as much code coverage as possible. Correctness of each exercise analyzer should be tested that way.
- Each exercise should have at least two smoke tests: one with an optimal solution that receives no feedback, and one with a solution that receives at least one exercise-specific comment.
- Next to that, a few smoke tests to cover exercises for which no analyzer is implemented should be present too, to make sure the analyzer works properly for every exercise. By that I mean that it shouldn't crash or something.

Once we come up with some concrete guidelines, we should probably write them down in the docs.

_Originally posted by @sanderploegsma in https://github.com/exercism/java-analyzer/issues/122#issuecomment-1924524027_

Contributor guide

Open the contributing guide

Research direction

Start by reviewing the proposal copied from issue #122 and inspect the existing analyzer unit and smoke tests to identify current coverage and conventions. Define concrete testing guidelines for analyzed and unanalyzed exercises, then document them and verify that the guidance matches the repository's existing tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
java
Domain
documentation, testing-qa
Issue type
Documentation
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.