lightninglabs / lightninglabs/taproot-assets

[refactor]: unify local proof source handling

Open
#1,398 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

database enhancement proofs
Dominant language
Go
Stars
525
Forks
150
Avg merge
2d 15h
Merged PRs (30d)
31

Description

Today we have three "sources of truth" when it comes to proofs (they all implement the proof.NotifyArchiver and parts of the proof.Archiver interface):

  • File based archive (proof.FileArchiver)
    • Content: Always full proof files (overlap/duplication across items if they share same origin)
    • Purpose: Persistence of transfer related proofs, freighter will store all proofs for all outputs here, even for outputs that don't belong to the local node
  • DB based archive (tapdb.AssetStore)
    • Content: Always full proof files (overlap/duplication across items if they share same origin)
    • Purpose: Store proofs for assets that are owned by the local node. Should really only reflect proofs for assets that are also in the assets table (or were at some point)
  • Local universe/multiverse (tapdb.MultiverseStore)
    • Content: Individual issuance or transfer proofs. Fully de-duplicated. If a transfer proof for state X exists, we expect all proofs in the provenance chain to also be present (from issuance up to state X-1).
    • Purpose: Store synced issuance proofs to identify assets and asset groups, store synced transfer proofs for own or foreign transfers

This situation mostly exists due to the universe being added much later as a feature, compared to the other two archives, so things grew organically.

This leads to certain assumptions about the state of the proof sources to be hard coded in some areas.
For example:

  • The freighter stores all proofs for outputs it creates in the file based store (and with an upcoming PR also in the universe). For a non-interactive (address based) transfer, the custodian will expect and look it up there. But for an interactive transfer, the RegisterTransfer RPC needs to be called, which expects the proof to be found in the multiverse instead.
  • The custodian will listen for new proof notifications from the asset store and the multiverse. But it will then import the proofs matched to inbound transfers into the file store and the asset store.
  • The re-org watcher only watches and updates proofs in case of a re-org in the file and asset store.
  • Probably more examples not listed here

We should think hard about what source should contain what proofs, when they are stored and updated in each source and when they notify about new proofs.
Each subsystem (mainly the freighter, custodian and re-org watcher) should either be completely oblivious to what source of proofs it uses (meaning it shouldn't need to handle any edge cases), or, if that's not possible, the assumptions need to be made more clear.

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start by reading the implementations of proof.FileArchiver, tapdb.AssetStore, and tapdb.MultiverseStore, then trace how the freighter, custodian, and re-org watcher store, update, and notify about proofs. Done means the ownership and lifecycle of each proof source are unified or clearly documented, with subsystem assumptions made explicit.

Written by the indexing model from the issue text.

Assessment

Tech stack
go
Domain
backend, databases, distributed-systems
Issue type
Refactor
Difficulty
5/5
Estimated time
Over a week
Activity status
Stale
Clarity
Needs clarification
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.