MetaMask / MetaMask/metamask-extension

Support api-specs test outputs in the test summary

Open
#32,796 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
TypeScript
Stars
13.2k
Forks
5.6k
Avg merge
2d 5h
Merged PRs (30d)
451

Description

### What is this about?

Prerequisite: https://github.com/MetaMask/metamask-extension/issues/31461?issue=MetaMask%7Cmetamask-extension%7C32787

Out of the box, the api-specs tests only produce an html report. These html reports are only shown as a comment under the PR when an api-specs job fails. This is a problem, because we cannot show failures or successes in the workflow summary. These tests also cannot be part of the flaky test report, as failures are not recorded.

After migrating to the mocha json reporter, we should develop a custom open-rpc json reporter that produces the same output that the mocha json reporter does. This way, the test results would seamlessly work with the workflow summary and the flaky test report.

In addition, it would be nice to extend the workflow summary with a link to the html report (if the job has one, display it)

### Scenario

_No response_

### Design

_No response_

### Technical Details

_No response_

### Threat Modeling Framework

_No response_

### Acceptance Criteria

_No response_

### Stakeholder review needed before the work gets merged

- [x] Engineering (needed in most cases)
- [ ] Design
- [ ] Product
- [x] QA (automation tests are required to pass before merging PRs but not all changes are covered by automation tests - please review if QA is needed beyond automation tests)
- [ ] Security
- [ ] Legal
- [ ] Marketing
- [ ] Management (please specify)
- [ ] Other (please specify)

### References

_No response_

Contributor guide

Open the contributing guide

Research direction

Start with prerequisite issue #31461 and inspect the existing api-specs HTML report, Mocha JSON reporter migration, workflow summary, and flaky test report integration. Determine the expected JSON output and how an HTML report link is exposed; done means api-specs successes and failures appear in both reports and the workflow summary links to the HTML report when available.

Written by the indexing model from the issue text.

Assessment

Tech stack
github-actions, typescript
Domain
ci-cd, testing
Issue type
Feature
Difficulty
4/5
Estimated time
3-5 days
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
35/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.