OpenRakis / OpenRakis/Spice86

Feature request: Generate comprehensive unit tests for generated code...?

Open
#185 1 comment 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

question reverse engineering
Dominant language
C#
Stars
658
Forks
48
Avg merge
8h 33m
Merged PRs (30d)
5

Description

Is your feature request related to a problem? Please describe.

In order to detect any regression when translating low level code into high level code, Spice86 is very valuable.

However, more subtle changes in gameplay logic (for example) might get unnoticed (ie. it doesn't crash or generate a graphical or audio glitch, but the Emperor does not ask for the usual amount of Spice by a off-by-one margin, for example)

Describe the solution you'd like

A comprehensive set of unit test. Possibly, auto-generated

Describe alternatives you've considered

Any open-source alternative, including writing our own for our own needs (that is, DUNE).

But there is none at the moment.

Additional context

The DUNE code is a mess, so a good safety net seems appropriate.

Also, IntelliTest is locked behind the very high price of VS Enterprise Edition, is not open-source, only works with the old .NET Framework for Windows, and might be not very valuable in the end. Unless, maybe, if it is guided towards a good solution via attributes :

https://devblogs.microsoft.com/visualstudio/intellitest-for-net-test-more-with-less-effort/

https://learn.microsoft.com/fr-fr/previous-versions/visualstudio/visual-studio-2015/test/generate-unit-tests-for-your-code-with-intellitest?view=vs-2015&redirectedfrom=MSDN

https://devblogs.microsoft.com/devops/intellitest-one-test-to-rule-them-all/

https://devblogs.microsoft.com/devops/smart-unit-tests-test-to-code-binding-test-case-management/

https://www.reddit.com/r/dotnet/comments/3nmq1o/automatically_generate_unit_tests_for_your_code/

It is based on this research project:

https://www.microsoft.com/en-us/research/publication/pex-white-box-test-generation-for-net/

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

The issue does not name files, tests, or an entry point. Start by surveying how Spice86 translates low-level code into high-level code and where existing tests live; done would be a defined, comprehensive unit-test approach that detects subtle gameplay regressions, potentially including generated tests.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
reverse-engineering, testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
20/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.