crytic / crytic/crytic-compile
Add support for solcjs (npm-installed Solidity compiler) for cross-architecture compatibility
- Dominant language
- Python
- Stars
- 197
- Forks
- 97
- PR merge metrics
- No merged PRs in 30d
Description
# Support for solcjs (npm-installed Solidity compiler) in crytic-compile
## Background
The Solidity compiler ecosystem has two main distribution methods:
1. **Native binaries** (`solc`) - Platform-specific executables distributed by the Ethereum Foundation
2. **JavaScript/WASM version** (`solcjs`) - Cross-platform compiler distributed via npm
Currently, crytic-compile only supports native `solc` binaries and fails when encountering `solcjs`, limiting its use in cross-architecture environments where native binaries are unavailable.
## The Problem
### Architecture Compatibility Challenge
Native Solidity binaries have limited architecture support:
- **x86_64**: Full support for all versions
- **ARM64 (Apple Silicon, AWS Graviton, etc.)**: Limited or no support for older versions
- **Other architectures**: Generally unsupported
This creates problems for:
- Developers on Apple Silicon Macs
- CI/CD pipelines running on ARM64 instances
- Docker containers targeting multiple architectures
- Projects requiring older Solidity versions (e.g., 0.4.x) on non-x86_64 platforms
### Current Failure Mode
When crytic-compile encounters `solcjs` instead of native `solc`, it fails with:
```
crytic_compile.platform.exceptions.InvalidCompilation: Invalid solc compilation /usr/lib/node_modules/solc/soljson.js:1
var Module;if(!Module)Module=...
```
This happens because:
1. `solcjs` is actually a Node.js script, not a native binary
2. crytic-compile uses `subprocess.Popen` with `executable=shutil.which(cmd[0])` to invoke `solc`
3. Without a shell, the Node.js shebang (`#!/usr/bin/env node`) isn't processed
4. The JavaScript content is returned as stdout instead of being executed
### Affected Code Locations
The issue manifests in multiple places:
- `crytic_compile/platform/solc.py:385`: `subprocess.Popen(..., executable=shutil.which(cmd[0]))`
- `crytic_compile/platform/solc.py:556-565`: Similar subprocess invocations
- `crytic_compile/utils/subprocess.py:35-50`: Common subprocess utility
## Proposed Solution
### Option 1: Add Native solcjs Support (Recommended)
Create a new platform type `SolcJS` that:
1. Detects when `solcjs` is available instead of `solc`
2. Handles the different CLI interface (solcjs uses different flags than solc)
3. Translates between command-line formats
**Implementation approach:**
```python
class SolcJS(AbstractPlatform):
NAME = "solcjs"
TYPE = Type.SOLCJS
@staticmethod
def is_supported(target: str, **kwargs: str) -> bool:
# Check if solcjs is available and target is .sol file
solcjs_path = shutil.which("solcjs")
if solcjs_path and target.endswith(".sol"):
# Verify it's actually solcjs not native solc
return _is_nodejs_script(solcjs_path)
return False
def compile(self, crytic_compile: "CryticCompile", **kwargs: str) -> None:
# Translate solc arguments to solcjs format
# e.g., --combined-json abi,bin → --abi --bin + manual JSON assembly
# Invoke with proper Node.js execution
```
**Benefits:**
- Clean abstraction aligned with crytic-compile's architecture
- Automatic fallback when native solc unavailable
- Transparent to downstream tools (Slither, Manticore, etc.)
### Option 2: Modify Existing Solc Platform
Enhance the existing `Solc` platform to detect and handle both native and JS versions:
```python
def _run_solc(...):
# Check if solc is actually solcjs
if _is_solcjs(solc_path):
return _run_solcjs(...)
else:
# Existing native solc logic
```
**Benefits:**
- Smaller change footprint
- Single platform for all direct Solidity compilation
**Drawbacks:**
- Mixes two different compiler interfaces in one class
- More complex error handling
### Option 3: Fix subprocess invocation
Modify subprocess calls to use shell=True or explicitly invoke through Node.js:
```python
# Instead of:
subprocess.Popen(cmd, executable=shutil.which(cmd[0]))
# Use:
subprocess.Popen(cmd, shell=True) # Let shell handle shebang
# OR
subprocess.Popen(["node", solc_path] + cmd[1:]) # Explicit Node.js
```
**Drawbacks:**
- Security implications of shell=True
- Doesn't address CLI incompatibilities between solc and solcjs
## CLI Differences to Handle
The main differences between `solc` and `solcjs`:
| Feature | solc | solcjs |
|---------|------|--------|
| Combined JSON | `--combined-json abi,bin` | Not supported (must use `--abi --bin` separately) |
| Output format | Direct JSON output | Separate files or stdout |
| Remappings | `--remap` | Different format |
| Import paths | `--allow-paths` | `--base-path` and `--include-path` |
## Testing Requirements
1. **Unit tests**: Mock solcjs responses for different Solidity versions
2. **Integration tests**: Test with actual npm-installed solcjs
3. **Cross-platform CI**: Test on x86_64 and ARM64 architectures
4. **Version compatibility**: Test with solcjs versions matching solc 0.4.x through 0.8.x
## Migration Path
1. **Phase 1**: Implement solcjs support behind a feature flag
2. **Phase 2**: Auto-detect and use solcjs when native solc unavailable
3. **Phase 3**: Document usage and limitations
4. **Phase 4**: Consider deprecating workarounds in downstream projects
## Alternative Workarounds (Current State)
Projects currently work around this limitation by:
1. Creating wrapper scripts that translate between solc and solcjs
2. Using Docker images with pre-installed native binaries (x86_64 only)
3. Skipping tests on unsupported architectures
4. Using remote compilation services
None of these are ideal solutions.
## Related Issues
- solc-select doesn't support ARM64 Linux binaries
- Foundry increasingly used in projects, which has better cross-platform support
- Growing adoption of ARM64 in cloud (AWS Graviton) and development (Apple Silicon)
## Impact
Adding solcjs support would:
- Enable crytic-compile on all architectures
- Remove a significant barrier for Apple Silicon developers
- Reduce Docker image complexity
- Enable older Solidity version support on modern hardware
- Improve CI/CD flexibility with ARM64 instances
## Recommended Approach
We recommend **Option 1** (Native solcjs Support) because:
1. It aligns with crytic-compile's existing architecture
2. Provides clean separation of concerns
3. Allows for proper testing and maintenance
4. Enables gradual rollout and testing
## Next Steps
1. Validate the proposed approach with maintainers
2. Implement prototype of SolcJS platform
3. Test with real-world projects using various Solidity versions
4. Document limitations and usage
5. Coordinate with downstream projects for testing
## Code References
Key files that need modification:
- `/crytic_compile/platform/solcjs.py` (new file)
- `/crytic_compile/platform/all_platforms.py:19` (add import)
- `/crytic_compile/platform/types.py` (add SOLCJS type)
- `/crytic_compile/crytic_compile.py:600-618` (platform selection logic)
## Use Case Example
This issue was discovered while trying to run Manticore tests in Docker on ARM64. The Dockerfile installs solcjs via npm since native solc 0.4.25 isn't available for ARM64, but crytic-compile fails to recognize and properly execute solcjs, making cross-architecture testing impossible.
## Conclusion
Supporting solcjs would significantly improve crytic-compile's usability across different architectures and environments. While there are multiple approaches, adding proper solcjs support as a new platform provides the cleanest and most maintainable solution.
Contributor guide
Research direction
Start by reading crytic_compile/platform/solc.py, crytic_compile/utils/subprocess.py, and the platform registration and selection paths in all_platforms.py, types.py, and crytic_compile.py. Run the existing compiler tests before comparing native solc and npm-installed solcjs behavior. Done means solcjs is detected and invoked with compatible arguments, with unit, integration, version, and architecture coverage.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- javascript, node.js, python, solidity
- Domain
- build-system, compilers, devtools
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100