oasisprotocol / oasisprotocol/cli
cli: Make example tests smarter
Nobody has claimed this yet.
- Dominant language
- Go
- Stars
- 80
- Forks
- 23
- Avg merge
- 9h 43m
- Merged PRs (30d)
- 3
Description
We have way too much .static examples meaning the output is not actually tested inside CI. I suggest we get rid of using .static for content that changes often and try to make a "smarter" diff.
For example, recent regression caused that ParaTime names got lost:
$ oasis network status
=== NETWORK STATUS ===
Network: localhost_testnet
Node's ID: 5MsgQwijUlpH9+0Hbyors5jwmx7tTmKMA4c9leV3prI=
Core version: 24.3
==== Consensus ====
Status: syncing
Version: 7.0.0
Chain context: 0b91b8e4e44b2003a7c5e23ddadb5e14ef5345c0ebcb3ddcae07fa2f244cab76
Latest height: 24115729 (2024-11-28 08:29:38 +0100 CET)
Latest block hash: b7a9ad2fe687b9b2887512b03707347b0ec743020a175bc45c09979e3cc90d82
Latest epoch: 40176
Is validator: false
Registration: false
==== ParaTimes ====
Status: waiting for consensus sync
Latest round: 7933058 (2024-11-28 08:29:38 +0100 CET)
Last finalized round: 18446744073709551615
Storage status: initializing
Number of peers: 23
Status: waiting for consensus sync
Latest round: 8930789 (2024-11-28 08:29:38 +0100 CET)
Last finalized round: 18446744073709551615
Storage status: initializing
Number of peers: 22
Status: waiting for consensus sync
Latest round: 9219372 (2024-11-28 08:29:38 +0100 CET)
Last finalized round: 18446744073709551615
Storage status: initializing
Number of peers: 23
We should ignore the latest height, block hash and epoch in this case, but still test everything else.
Perhaps a kind of a regex "mask" per example that should define how the output should look like. So when you run "make examples" and the regex is matched, then don't generate the new .out files.
Contributor guide
No contributing guide indexed for this repository
First steps
- Read the whole issue, then the project's contributing guide.
- Comment on the issue to say you are picking it up — it saves two people doing the same work.
- Fork the repository and make your change on a branch.
- Open a pull request that references the issue number.
Research direction
Start by inspecting the example definitions used by make examples, along with the .static and .out files. Determine how example output is currently compared, then verify that dynamic fields such as latest height, block hash, and epoch can be excluded while the remaining output is still checked and matching examples do not regenerate .out files.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- go
- Domain
- cli, testing
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100