compiler-explorer / compiler-explorer/infra
[julia] Use package for the wrapper to speed up compilation
- Dominant language
- Python
- Stars
- 434
- Forks
- 429
- Avg merge
- 4d 21h
- Merged PRs (30d)
- 38
Description
I created a small Julia package, [`CompilerExplorer.jl`](https://github.com/giordano/CompilerExplorer.jl), which simply replicates what the [Julia wrapper in Compiler Explorer](https://github.com/compiler-explorer/compiler-explorer/blob/cbce8dfb8b15a44959fe8d3f35f8a9c5050e137a/etc/scripts/julia_wrapper.jl) currently does. The code in that wrapper would simply become
```julia
using CompilerExplorer
CompilerExplorer.main()
```
The significant advantage is that when Julia code is saved into a package, it can cache some precompile code (and from Julia v1.9 onwards also native code), which can significantly improve runtime performance for the first run, as the methods called don't need to be JIT-compiled (or not as much extensively). Note that here I'm talking about the compilation of the functions calling the compiler infrastructure to eventually compile the code the user inputs in Compiler Explorer, that code will of course still have to be compiled regularly, but at least that'd be where most of the time will be spent.
As an example, for the default example code for the Julia language, in my local Compiler Explorer instance I get runtime going down from ~1800 ms with the current version of the wrapper

to ~400 ms with using the package

Using this package would also make testing easier for us for all versions, and hopefully avoid silly mistakes like https://github.com/compiler-explorer/compiler-explorer/pull/5927#issuecomment-1882396045.
I'm happy to make a PR to make this happen, but I want to understand whether this would be effective in the production version of Compiler Explorer. If we install this package during the installation of the Julia toolchain here, can we cache its precompiled code? By default Julia stores it in the `~/.julia/compiled` directory, but the location can be changed with the environment variable [`JULIA_DEPOT_PATH`](https://docs.julialang.org/en/v1/manual/environment-variables/#JULIA_DEPOT_PATH), to accommodate whatever caching infrastructure is used here.
CC: @vchuravy
Contributor guide
Research direction
Start by comparing etc/scripts/julia_wrapper.jl with the CompilerExplorer.jl package and inspect how the Julia toolchain is installed. Determine whether precompiled code can persist through the production setup using JULIA_DEPOT_PATH; done means the package is integrated with effective caching and the runtime improvement is verified.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- julia
- Domain
- infrastructure, performance
- Issue type
- Feature
- Difficulty
- 4/5
- Estimated time
- 3-5 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100