multiformats / multiformats/multicodec

Qualifications for identification as a "codec"

Open
#204 10 comments 2 reactions 4 assignees View on GitHub

@aschmahmann is already working on this.

Since Jan 22, 2021.

Dominant language
Python
Stars
383
Forks
221
Avg merge
24d 6h
Merged PRs (30d)
1

Description

This question comes up very regularly, essentially what are the qualifications for being included in this table, but I'm particularly focusing on the "codec"s in here, IPLD and related.

Some recent discussions that point toward challenges in definitions:

  • Likecoin uses dag-cbor but wants to do content-routing with a new codec https://github.com/multiformats/multicodec/pull/200
  • SoftWare Heritage mostly uses git-raw but interop with their systems might benefit from new codecs https://github.com/multiformats/multicodec/pull/203 (also related to content routing)
  • Filecoin wanting to represent CommP/CommR/CommD as CIDs where there exist practical and logical difficulties in treating the codecs as "encoding formats" https://github.com/multiformats/multicodec/pull/172 (lengthy discussion and includes a few other linked PRs)
  • Bitcoin Witness Commitment https://github.com/multiformats/multicodec/pull/176 is a union of two 32-byte byte arrays, which is the arguably the same binary format as bmt although the logical format is slightly different because only one of these things is technically a hash, but it's still a 256-bit number. Similar is bitcoin-tx which either describes an actual transaction (including or excluding witness data) or a node in a binary merkle tree (bmt again!), but you can't disambiguate so there's some degree of nominative typing going on in using this. Same pattern is repeated for most of the other blockchains we have in there (well the Bitcoin forks at least).

How about the fact that all of the codecs we present could also be interpreted as raw? I liked @Ericson2314's thoughts related to this kind of question: https://github.com/ipld/specs/pull/349#issuecomment-763867809

Some excellent and clear thinking from @aschmahmann about nominative typing using codecs: https://github.com/ipld/specs/pull/349#discussion_r559963156

There clearly exists a grey area here, and while we should avoid strong gatekeeping of the table where a contributor has greater expertise in their particular system than us, there's an educational role to play too because many people show up with requests that clearly don't fit the purpose of this table and the definitions of "codec" that we broadly share. It's valid to say "this is an incorrect use of multicodec / CID" where it clearly is. But what we need is better shared understanding of those "clear" boundaries.

Thoughts please!

(/cc @warpfork who isn't in the Assignees list)

Contributor guide

No contributing guide indexed for this repository

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.