Ensuring consistent compiler version for linux binaries
- Dominant language
- Python
- Stars
- 258
- Forks
- 68
- PR merge metrics
- No merged PRs in 30d
Description
Mason should be more strict about using consistent compiler versions for all mason binaries.
This ticket describes some context of why this is important, background, proposed ideas for a solving this and related issues.
## Context
We've been mixing clang 3.5 and 3.8 binaries for a while now in mason for Linux packages. Old packages are built with 3.5, and new ones published are built with 3.8. This is not okay going forward. Recently an issue arose in mapnik-vector-tile on Linux that indicates rare bugs can surface when clang++ 3.5 and 3.8 binaries are mixed: https://github.com/mapbox/mapnik-vector-tile/issues/216. In this case 1000s of unit tests were passing and only one was failing. But that failing test was only solvable by rebuilding all binaries to consistently use clang 3.8. So the tentative takehome is that mixing clang 3.5 and 3.8 binaries works 99% of the time. That is not good enough. For a binary deployment solution to be solid the binaries need to work 100% of the time.
## Background on compiler defaults per platform
### linux
Currently we are targeting clang 3.8. But 3.8.1 and 3.9.0 are now out and we should get a plan in place to upgrade to those compilers safely. Per the `Context` above we have evidence to suggest that mixing binaries between clang++ versions (at least 3.5 <->3.8) is not safe (see `Context` above for more info). At this point we don't have any evidence that mixing binaries between clang (any version) and gcc 5 is problematic (as long as `-D_GLIBCXX_USE_CXX11_ABI=0` is set when using gcc5 to ensure linking works).
### macOS and iOS
Currently most .travis.yml target `osx_image: xcode7.3`. We have no evidence to suggest there is are problems mixing binaries between xcode versions, but ideally we come up with a good mechanism to track the latest stable, which is now `osx_image: xcode8`
### Android
We use the cross compiler so this is already explicit and controlled, no action needed.
## Ideas for improving mason
- Add dev mode to mason core to enable easily rebuilding all packages against a new compiler without overwriting any existing binaries. Applications using Mason should be able to opt-in to the freshly compiled binaries when upgrading their mason version.
- For builds: add environment assertion that runs before packages are built that assert:
- CXX and CC environment settings make sense
- CXX --version and CC --version make sense
- Add post-build assertions that test for GLIBC and GLIBCXX requirements and ensure no > than expected
- Vet every .travis.yml for consistency of format and toolchain bootstrap.
/cc @kkaefer @jfirebaugh @jakepruitt @tmpsantos @karenzshea
Contributor guide
Research direction
Start by reviewing Mason core's build path and every .travis.yml, then inspect how CXX and CC are selected and how compiler versions are reported. The issue names no specific tests or files; done would require an agreed implementation plan covering consistent toolchains, rebuild behavior, environment and post-build assertions, and CI consistency.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- cpp, linux
- Domain
- build-system, devops, operating-systems
- Issue type
- Feature
- Difficulty
- 5/5
- Estimated time
- Over a week
- Activity status
- Stale
- Clarity
- Needs clarification
- Newbie friendliness
- 25/100