TASEmulators / TASEmulators/BizHawk

Test suite for console-verified movies

Open
#2,294 0 comments 1 reaction 0 assignees View on GitHub

Nobody has claimed this yet.

Meta Request: Feature/Enhancement
Dominant language
C#
Stars
2.8k
Forks
468
PR merge metrics
No merged PRs in 30d

Description

With movies getting console-verified, it might be nice to create a test framework for ensuring cores continue to playback movies correctly.

There is a mountain of caveats here. For one, I'm specifically talking about movies where the input logs didn't need to be modified to work on console. Additionally, I am not suggesting a test failing would mean the core necessarily even has a problem; it would just be an indicator of something that might need to be looked into. If some behavior on console turns out to be nondeterministic and it just happened to work for the one tester, then it could obviously pose complications for that particular game.

A very basic and core-agnostic method of comparison would be to playback the movie and compare the framebuffer at the end. This approach is not without flaws; just because a movie's inputs were console-verified doesn't mean it rendered all of the graphics accurately in the emulator the original playback. Also, palettes would probably be a concern.

More sophisticated methods could take a full memory dump and compare, but this approach has even more subtle ways comparison can fail, and may be too complicated to be feasible.

see also #2813

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reviewing the related issue #2813 and the existing movie playback flow. Compare the proposed framebuffer approach with full memory-dump comparison, including the stated caveats around palettes and nondeterministic behavior. Done would be a test framework for console-verified movies with a clear comparison result.

Written by the indexing model from the issue text.

Assessment

Tech stack
csharp
Domain
testing-qa
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.