runtimeverification / runtimeverification/llvm-backend

Automate regeneration of test cases

Open
#987 3 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

Dominant language
C++
Stars
43
Forks
22
PR merge metrics
No merged PRs in 30d

Description

Currently, we have a directory of checked-in KORE files that encode test cases using lit. The corresponding K source code also exists in the repo, but the process of actually generating the KORE from the K is not automatic and requires a reasonable amount of manual work.

We should implement a script that can take a K source definition and automatically regenerate the corresponding KORE that the backend actually runs. This should remain a manual process run by developers when something changes in the frontend; we don't really want to introduce a dependency on K in CI.

The script will need to:

  • kompile a definition and copy the definition.kore to the test directory in an appropriate place.
  • Extract the appropriate // RUN: ... lines from the K file and paste them at the front of the KORE file.
  • Perhaps also automatically generate the test inputs and outputs from surface syntax?

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by locating the checked-in KORE test directory and the corresponding K source definitions. Read how lit handles the // RUN: ... lines, then verify how kompile produces definition.kore; done means developers can manually regenerate the appropriate KORE and preserve the required test commands without adding K as a CI dependency.

Written by the indexing model from the issue text.

Assessment

Domain
testing, tooling
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Mostly clear
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.