livepeer / livepeer/protocol

Invariant testing across many rounds

Open
#237 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement
Dominant language
JavaScript
Stars
155
Forks
50
PR merge metrics
No merged PRs in 30d

Description

There are a few invariants within the protocol that must always be upheld.

Example:

A transcoder with a reward cut of X% gets X% of rewards minted during a round. All delegators delegated to the transcoder during that round receive a share of (100 - X)% of minted reward proportional to their bonded stake relative to the transcoder's total delegated stake.

This type of invariant should be upheld for every round. We can improve the rigor of our test suite by fully exploring the different types of actor sets, the ordering/timing of transactions.

One way to do this is to create a model using seed data that tracks the expected reward shares for multiple parties across many rounds with various parties bonding/unbonding during different rounds. A data fixture would be produced. The test would then mimic the actions that the model assumes and checks that the values at each round match the data fixture.

Another way to do this that could be complementary is to use automated tools for exploring state transitions within a contract:
- [Echidna](https://github.com/trailofbits/echidna) is a property based fuzz tester that can be used to generate a wide variety of inputs for transactions in an attempt to see if a specific property can be violated.
- [Manticore](https://github.com/trailofbits/manticore) is a symbolic execution tool that can be used to generate scenarios with a variety of actors that fully explore the possible state of a contract via transaction submission using symbolic execution.

Symbolic execution would technically be more complete than fuzz testing since it explores the entire state space, but it can also be slow (to the point of not being able to complete) depending on the use case. We should keep this in mind when evaluating whether to use one tool or the other. Either way these tools might be useful for invariant testing across many rounds when the type of actors, ordering/timing of transactions matters.

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

No files or tests are named. Start by reviewing the protocol's reward invariants and comparing the proposed model-based fixture approach with Echidna and Manticore; done means a repeatable test strategy that checks invariant behavior across many rounds, actor sets, and transaction orderings.

Written by the indexing model from the issue text.

Assessment

Domain
blockchain, testing
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.