CLI and Standard JSON do not generate the same outputs in case of errors at code generation stage
- Dominant language
- C++
- Stars
- 25.7k
- Forks
- 6.2k
- Avg merge
- 2d 19h
- Merged PRs (30d)
- 29
Description
### Description
When compilation fails due to an ICE or error at the code generation stage or later, Standard JSON still gives user some outputs, e.g. metadata, while the CLI does not. For consistency both should provide the same outputs.
One consequence of this inconsistent behavior is that ICEs [have to be ignored by `prepare_report.py`](https://github.com/ethereum/solidity/blob/v0.8.18/scripts/bytecodecompare/prepare_report.py#L134-L140) and [by `prepare_report.js`](https://github.com/ethereum/solidity/blob/v0.8.18/scripts/bytecodecompare/prepare_report.js#L75-L82) in our bytecode comparison. This potentially hides some ICEs that may happen only in the emscripten binary (e.g. #13924) - we won't detect such inconsistent ICEs otherwise because we don't run full test suite against the emscripten binary. We'll be able to remove this workaround when the problem is fixed.
Ensuring that all those inputs are getting printed correctly on the CLI is going to take some effort so my suggestion would be to fix it by just hiding all the outputs in case of an ICE or other such errors. They're unlikely to be of much use in that case anyway.
### How to reproduce
This applies at least to `UnimplementedFeatureError` (ICE), `CompilerError` (ICE), `CodeGenerationError` (error) but may also affect other types that just don't happen in any of our test cases currently.
#### Input
`test.sol`
```solidity
pragma solidity *;
// SPDX-License-Identifier: GPL-3.0
contract C {
uint immutable x;
constructor() {
x = 0;
revert("");
}
function f() external view returns(uint) {
return x;
}
}
```
`test.json`
```json
{
"language": "Solidity",
"sources": {
"test.sol": {"urls": ["test.sol"]}
},
"settings": {
"outputSelection": {"*": { "*": ["metadata", "evm.bytecode"]}}
}
}
```
#### CLI
```bash
solc test.sol --optimize --metadata --bin
```
Current output:
```
Error: Some immutables were read from but never assigned, possibly because of optimization.
```
#### Standard JSON
```bash
solc --standard-json test.json --json-indent 4 --allow-paths .
```
Current output:
```json
{
"contracts":
{
"test.sol":
{
"C":
{
"metadata": "{\"compiler\":{\"version\":\"0.8.17+commit.8df45f5f\"},\"language\":\"Solidity\",\"output\":{\"abi\":[{\"inputs\":[],\"stateMutability\":\"nonpayable\",\"type\":\"constructor\"},{\"inputs\":[],\"name\":\"f\",\"outputs\":[{\"internalType\":\"uint256\",\"name\":\"\",\"type\":\"uint256\"}],\"stateMutability\":\"view\",\"type\":\"function\"}],\"devdoc\":{\"kind\":\"dev\",\"methods\":{},\"version\":1},\"userdoc\":{\"kind\":\"user\",\"methods\":{},\"version\":1}},\"settings\":{\"compilationTarget\":{\"test.sol\":\"C\"},\"evmVersion\":\"london\",\"libraries\":{},\"metadata\":{\"bytecodeHash\":\"ipfs\"},\"optimizer\":{\"enabled\":false,\"runs\":200},\"remappings\":[]},\"sources\":{\"test.sol\":{\"keccak256\":\"0x082c821cef2fd83774f15988052d30d20b86dcf465cdf8675717e7fd43685128\",\"license\":\"GPL-3.0\",\"urls\":[\"bzz-raw://a7ddee9e3fe3639b7e8d0133e3b29806926877831790345aeff11f308e2cd345\",\"dweb:/ipfs/QmPKyguCF7nwnDD9ahfiRJRVx5H9aNJ5ajCehwUm52Nkzd\"]}},\"version\":1}"
}
}
},
"errors":
[
{
"component": "general",
"errorCode": "1284",
"formattedMessage": "CodeGenerationError: Some immutables were read from but never assigned, possibly because of optimization.\n\n",
"message": "Some immutables were read from but never assigned, possibly because of optimization.",
"severity": "error",
"type": "CodeGenerationError"
}
],
"sources":
{
"test.sol":
{
"id": 0
}
}
}
```
Contributor guide
Research direction
Start by reproducing the provided test.sol and test.json cases with the CLI and Standard JSON commands, then trace how each interface handles UnimplementedFeatureError, CompilerError, and CodeGenerationError. Compare the output paths and the prepare_report.py and prepare_report.js workarounds; done means both interfaces consistently hide outputs for these failures so the workarounds can be removed.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, solidity
- Domain
- cli, compilers
- Issue type
- Bug
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100