ipfs / ipfs/gateway-conformance

Generate Implementation Dashboard

Open
#123 2 comments 0 reactions 0 assignees View on GitHub
Dominant language
Go
Stars
19
Forks
15
Avg merge
1d 4h
Merged PRs (30d)
5

Description

## Base Bricks

- [x] https://github.com/ipfs/gateway-conformance/pull/125
- [x] https://github.com/ipfs/gateway-conformance/pull/145 | 1 day
- [x] https://github.com/ipfs/gateway-conformance/pull/146
- [x] Update workflows to expose their JSON outputs & retrieve them from gateway-conformance | 1 day
- [x] Try in experiment repositories
- [x] Implement the artifact-retrieval script
- [x] Update client repositories
- https://github.com/ipfs/kubo/pull/10092
- https://github.com/ipfs/boxo/pull/442
- https://github.com/ipfs/bifrost-gateway/pull/195

Phase 1: [Example output](https://github.com/ipfs/gateway-conformance/actions/runs/5924237972/attempts/1#summary-16061404658)

## Generate Dashboard - Base Structure

We'll generate a dashboard in gh-pages, under a sub-path like project.io/dashboard.

- [x] Define the Dashboard sitemap | 1 day
- [x] https://github.com/ipfs/gateway-conformance/pull/150
- [x] Output to `gh-pages` and setup a single URL | 1 day
- [x] Generate the dashboard daily using outputs from other repos | 1 day
- [x] https://github.com/ipfs/gateway-conformance/pull/152 | 3 days
- [x] /(index)
- [x] /dashboard
- [x] render gateway version
- [x] render implementation
- [x] render results
- [x] https://github.com/ipfs/gateway-conformance/pull/156 to be used in gateway checker | 1 day

## Generate Dashboard - Drill Down

We'll let the user drill down test results, and link to test outputs / github action results

- [x] Create pages to drill-down tests results | 2 day
- [x] per version
- [x] per implementation
- [x] per test / sub test / etc
- [x] Render the execution logs in tests | 1 day
- [x] https://github.com/ipfs/gateway-conformance/pull/166#issue-1898102599 | 1 day

## Generate Dashboard - Specs Relationships

- [x] https://github.com/ipfs/gateway-conformance/pull/159
- [x] https://github.com/ipfs/gateway-conformance/pull/163
- [x] Add `https://` to the specs URL + add `Specs` field for multiple specs
- [x] Create spec -> tests & tests -> spec indexes | 2 days

## [Implementation Dashboard - Going Public](https://github.com/ipfs/gateway-conformance/issues/169) | 5 days

## Generate Dashboard - 💅

- [ ] Export Hints in the test outputs as well
- [ ] If possible, add links to the test definition sources (it should be possible to extract the test line number and file, etc and create permalink to github's conformance source)
- [ ] Nicer Design | 1 day
- [ ] Edit Content | 2 day

## Follow Ups

- [ ] fix the redirects for /tests and /results (not critical if we host at the root)
- [ ] Deal with matrix in client jobs (like https://github.com/ipfs/bifrost-gateway/actions/runs/5925256845)
- [ ] Validate specs (check the page and anchors exists)
- [ ] Generate a spec coverage table (detect how many specs pages + anchors we have and how much we test)

## Related Discussions

https://github.com/ipfs/gateway-conformance/pull/116
https://github.com/ipfs/gateway-conformance/pull/116#pullrequestreview-1550977269

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.