google / google/codeworld

Requirements: Runtime testing

Open
#778 0 comments 0 reactions 0 assignees View on GitHub
discussion
Dominant language
Haskell
Stars
1.3k
Forks
201
PR merge metrics
No merged PRs in 30d

Description

It would be great to be able to define runtime tests that must be passed as part of the requirements checker. This would involve a severe change in the way the requirements checker works, but my plan is this:

1. Implement the requirements checker as a GHC source plugin rather than a separate CodeWorld build step.
2. For static rules (all of those currently implemented), the result is computed at compile time and hard-coded into the generated code. For dynamic rules (like runtime tests), code is generated to check the result at runtime.
3. Code for reporting (and evaluation, in the dynamic case) of rules is injected at the beginning of `main`, to run as the first action of the compiled code before anything else.

Contributor guide

Open the contributing guide

Research direction

Start by locating the requirements checker and the separate CodeWorld build step, then read how the current static rules are evaluated. The proposed work is complete when the checker is a GHC source plugin that handles static rules at compile time, dynamic runtime tests in generated code, and reporting before main performs other actions.

Written by the indexing model from the issue text.

Assessment

Tech stack
haskell
Domain
compilers, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.