conda-forge / conda-forge/conda-forge.github.io
Use ccache
- Dominant language
- JavaScript
- Stars
- 170
- Forks
- 320
- Avg merge
- 2d 10h
- Merged PRs (30d)
- 5
Description
This issue is intended to discuss the use of ```ccache``` when building compiled packages and the findings I've got on a [experimental branch](https://github.com/gqmelo/llvmdev-feedstock/pull/1).
# Introduction
> ccache is a compiler cache. It speeds up recompilation by caching previous compilations and detecting when the same compilation is being done again. Supported languages are C, C++, Objective-C and Objective-C++.
From my personal experience ```ccache``` is very reliable and after a couple of years using it at work we never found any issue. Additionally adopting ccache was a huge improvement.
# Why
Using ```ccache``` on ```conda-forge``` could bring a lot of benefits:
- Faster feedback for mantainers of large packages. I personally find it very demotivating trying to make a large package build on CI.
- Possibly decrease CI's queue size
- Make it possible to build packages that exceed timeout limits
# Obtained results
When experimenting with [llvmdev-feedstock](https://github.com/gqmelo/llvmdev-feedstock/pull/1) the results were very promising.
- On CircleCI a build with ```99.92%``` cache hit took ```15m``` as opposed to the usual ```1h30m```
- On CircleCI a first-time build took about ```1h``` instead of the usual ```1h30m```. This happens because on CircleCI all package variations are built on the same job.
- On Travis a build with ```99.92%``` cache hit took about ```14m``` instead of the usual ```44m``` (very close to the timeout limit).
# Necessary changes
## TravisCI
```diff
language: generic
os: osx
osx_image: xcode6.4
+cache:
+ branch: no-md5deep
+ directories:
+ - $HOME/.ccache
+
+
env:
matrix:
```
The ```branch: no-md5deep``` is a workaround for https://github.com/travis-ci/travis-ci/issues/7456
## CircleCI
```diff
# just to pull each time. #rollondockercaching
override:
- docker pull condaforge/linux-anvil
+ - ./ci_support/run_docker_build.sh || touch ./build_failed
+ cache_directories:
+ - "./build_artefacts/.ccache"
test:
override:
# Run, test and (if we have a BINSTAR_TOKEN) upload the distributions.
- - ./ci_support/run_docker_build.sh
+ - if test -f ./build_failed; then exit 1; fi
```
CircleCI cache is only done for the dependencies phase. So to cache the compilation we need to move the build command. Also, the build command cannot fail during the dependencies phase, otherwise the cache will not be saved. That is why the verification is done later.
## ccache setup
This would be added preferably to [toolchain activate script](https://github.com/conda-forge/toolchain-feedstock/blob/master/recipe/activate.sh):
```bash
if [ $(uname) == Linux ]; then
export CC="ccache gcc"
export CXX="ccache g++"
export CCACHE_DIR=/feedstock_root/build_artefacts/.ccache
# Set max cache size so we don't carry old objects for too long
ccache -M 400M
else
export CC="ccache clang"
export CXX="ccache clang++"
# Set max cache size so we don't carry old objects for too long
ccache -M 200M
fi
export CCACHE_BASEDIR="${SRC_DIR}"
ccache -z
```
Note the ```ccache -M``` commands to limit the cache size. This is important because the cache is never automatically deleted by TravisCI and CircleCI. Also we would want to set different sizes for both CI's as CircleCI builds more than one package per job.
Also ```ccache``` needs to be added as dependency, preferably on [toolchain package](https://github.com/conda-forge/toolchain-feedstock/blob/master/recipe/meta.yaml)
## Recipes
There would be no necessary changes on recipes whose build systems respect ```CC``` and ```CXX``` variables. However for large packages that could exceed the timeout limit it would be desirable to avoid the timeout killing the build command. On Linux this can be achieved with something like:
```bash
timeout 5400 make -j{CPU_COUNT}
```
Avoiding the timeout limits is important because otherwise the cache is not saved.
# Windows
On Windows there is ```clcache``` which was inspired by ccache and so is very similar. The problem is that AppVeyor cache mechanism is different. The cache is only enabled for regular branchs, not PR's. Also it limits the cache size to 1GB per account, which in my opinion make the cache useless for a large organization as ```conda-forge```.
# References
- https://circleci.com/docs/1.0/how-cache-works/
- https://docs.travis-ci.com/user/caching/
- https://www.appveyor.com/docs/build-cache/
# Pending
- [ ] Make it work when someone build locally without ```run_docker_build.sh``` script
- [ ] Use ```conda build --no-build-id``` and avoid setting ```CCACHE_BASEDIR```
Contributor guide
Assessment
This issue has not been assessed yet.