LibreSign / LibreSign/libresign

Validate the request signature screenshot in pull requests

Open
#8,091 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

backend feature-request frontend github_actions
Dominant language
PHP
Stars
818
Forks
146
Avg merge
11h 31m
Merged PRs (30d)
326

Description

The screenshot generation added in #8090 should also be validated automatically in pull requests.

The goal is to detect when changes to the LibreSign interface make img/screenshot/request-signature.png outdated.

This issue depends on #8090.

What should be done

Add a GitHub Actions workflow that:

  1. runs the screenshot generation command from #8090;
  2. compares the generated image with the committed img/screenshot/request-signature.png;
  3. uses pixelmatch or an equivalent small visual comparison library;
  4. ignores insignificant rendering differences using a reasonable tolerance;
  5. succeeds when the screenshots match within the allowed tolerance;
  6. fails when a relevant visual difference is detected.

When the comparison fails, the workflow should:

  • report the number or percentage of different pixels;
  • generate a visual diff;
  • upload the generated screenshot and diff as workflow artifacts;
  • add or update a comment in the pull request.

Reference implementation

A similar implementation can be used as reference from LSTelegramNotify:

  • [comparePngBuffers()](https://github.com/LibreCodeCoop/LSTelegramNotify/blob/c2f68cbdbe6318d134a5f2e79ccf27cded9106c0/scripts/screenshots/checker.mjs#L8-L47) shows how pixelmatch can compare two PNG buffers and report the number and percentage of different pixels;
  • [assertScreenshotsUpToDate()](https://github.com/LibreCodeCoop/LSTelegramNotify/blob/c2f68cbdbe6318d134a5f2e79ccf27cded9106c0/scripts/screenshots/checker.mjs#L49-L124) shows how a failed comparison can generate a visual diff;
  • [documentation-screenshots.yml](https://github.com/LibreCodeCoop/LSTelegramNotify/blob/c2f68cbdbe6318d134a5f2e79ccf27cded9106c0/.github/workflows/documentation-screenshots.yml) shows how screenshot validation and diff artifacts can be handled in GitHub Actions.

The LibreSign implementation does not need to copy this structure exactly.

Pull request comment

When a relevant difference is found, the workflow should leave a short comment explaining that the request signature screenshot may be outdated.

The comment should include:

  • the detected difference;
  • a note telling the contributor to run the screenshot generation command from #8090 if the visual change is expected;
  • a reference to the workflow artifacts with the generated screenshot and visual diff.

The workflow should avoid creating duplicate comments. If a screenshot validation comment already exists, update it instead.

A hidden marker can be used to identify the comment, for example:

<!-- libresign-request-signature-screenshot-check -->

No comment is needed when the screenshot matches.

The workflow must not automatically commit the generated screenshot. Updating img/screenshot/request-signature.png should remain an explicit change made after reviewing the detected visual difference.

Acceptance criteria

  • The screenshot generated by #8090 is compared with the committed img/screenshot/request-signature.png.
  • pixelmatch or an equivalent approach is used for visual comparison.
  • A documented tolerance prevents failures caused only by insignificant rendering differences.
  • The workflow succeeds when the images match within the allowed tolerance.
  • The workflow fails when a relevant visual difference is detected.
  • The comparison reports the number or percentage of different pixels.
  • A visual diff is generated when the comparison fails.
  • The generated screenshot and visual diff are uploaded as GitHub Actions artifacts.
  • A pull request comment explains when the screenshot may need to be updated.
  • Repeated runs update the existing validation comment instead of creating duplicate comments.
  • The workflow does not automatically commit screenshot changes.
Additional context
  • If you have questions, feel free to ask in this issue.
  • Give a ⭐️ star to this repository if you find LibreSign useful and would like to support the project.
  • You can also join our community: https://t.me/LibreSign

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with the screenshot generation command from #8090, the committed img/screenshot/request-signature.png, and the referenced checker.mjs and documentation-screenshots.yml examples. Build the GitHub Actions validation around the stated comparison, tolerance, diff artifacts, and pull request comment behavior; done means all acceptance criteria pass without automatically committing the screenshot.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, javascript
Domain
ci-cd, devops, testing-qa
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Active
Clarity
Mostly clear
Newbie friendliness
55/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.