Builds can fail depending on host resources, some control is desireable

[See original issue on GitLab](https://gitlab.com/BuildStream/buildstream/-/issues/185)
In GitLab by [[Gitlab user @jjardon]](https://gitlab.com/jjardon) on Jan 10, 2018, 15:14

In some scenarios, builds can fail depending on how much resources they have available. This is worsened when coupled with fault intolerant tooling, such as compilers which abort at the first sight of an OOM condition.

A recently observed case is when building LLVM (which manages to link many large binaries in parallel). In this case the build was working fine with a reasonable amount of available RAM, but begins to fail when given more CPUs, triggering more parallel linking and reaching the OOM condition.

Various approaches are possible to address the load here. A declarative approach might include some additional user configuration to describe the capabilities of the host, and / or some extension to the `.bst` format to allow one to describe the resource consumption characteristics of a given build (for instance, one might be able to express the nature of the LLVM build in a `llvm.bst` file so as to inform BuildStream about the per-job memory requirements of the element).

None of the approaches I've come up with so far are in any way attractive, though; open to suggestions.

Start with the original GitLab issue and review how BuildStream currently controls parallel builds and host-resource use. Compare the proposed host capability configuration with describing resource consumption in `.bst` files, including the `llvm.bst` example. Done means an agreed approach that prevents resource-dependent build failures and is reflected in the relevant configuration or format.

Written by the indexing model from the issue text.