Canonical serialization format for JSON metadata v2+
- Dominant language
- C++
- Stars
- 25.7k
- Forks
- 6.2k
- Avg merge
- 1d 11h
- Merged PRs (30d)
- 21
Description
Depends on #16152.
See also https://github.com/ethereum/solidity/issues/14389#issuecomment-3148552429.
## Abstract
Define precise rules for how the [JSON metadata](https://docs.soliditylang.org/en/develop/metadata.html#contract-metadata) is to be serialized for the purpose of derivation of the metadata hash that the compiler embeds in the bytecode. The goal is to make it possible to arrive at the same hash even if the JSON is formatted differently than the original compiler output.
## Motivation
Currently the hashes are derived from the metadata in the form returned by the `--metadata` output. This form is not officially documented and depends on the implementation. While in practice it follows certain formatting rules and we have been careful to keep it unchanged even when switching JSON libraries, this is not something that is guaranteed. Deriving the same hash requires tools to effectively treat that output as binary and preserve its exact form. It is not certain that after deserializing it one will be able to arrive back at that exact form.
Another consequence of this is that solc has to be very careful about always presenting that output in the same form. When embedded in Standard JSON, the metadata is represented with a string, rather than a nested JSON object. It is also unaffected by the `--pretty-json` option, which prevents it from being presented to the user in a readable form.
## Specification
1. Define the canonicalization rules. Those should specify at the very least:
- How to handle the whitespace (minimizing it is probably the best choice).
- How to order the keys.
- Whether to preserve or remove `null` keys.
- Whether to allow and preserve comments.
- Whether to include unicode characters as-is in strings or replace them with escapes.
1. Officially document the canonicalization rules.
1. Introduce `--metadata-version` as specified in #16152.
1. Make the format of the `--metadata`/`metadata` output dependent on the selected version.
- For version 1, keep it as it is now.
- For version 2, make it return a proper JSON object rather than a string containing serialized JSON.
- It should now be affected by `--pretty-json` and `--json-indent` options as the rest of the JSON structure.
1. Introduce `--metadata-canonical`/`metadataCanonical` output, returning the exact form of metadata that is used for calculating the metadata hashes.
- For version 1 this is simply the same output as `--metadata`.
- For version 2 this is the canonicalized metadata JSON.
## Backwards Compatibility
The change is fully backwards compatible, because we keep the current behavior by default and the new one has to be explicitly opted into via `--metadata-version 2`. We will not change the default until the next breaking version.
We will also only apply the new rules to versions 2 or later to make it unambiguous whether it is safe to reformat the metadata or not. **Note that this depends on the the feature being included in the same release as #16152**. Otherwise we will have to bump metadata version again to make sure that each metadata version is only ever used with one set of canonicalization rules.
Contributor guide
Research direction
Start with dependency #16152 and the linked metadata documentation, then inspect the current --metadata/metadata outputs and the --pretty-json/--json-indent options. Done means canonicalization rules are defined and documented, version-dependent output behavior is specified, and the metadata-canonical output is covered for versions 1 and 2.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, solidity
- Domain
- blockchain, compilers
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 28/100