Explain larger implications of using SCM feature with revisions
- Dominant language
- C++
- Stars
- 125
- Forks
- 382
- Avg merge
- 22h 39m
- Merged PRs (30d)
- 21
Description
The SCM feature has surprised a number of people with some of it's behavior. Users have made a number of feature requests relating to this behavior which seem reasonable. However, many of these have turned out to be in conflict with the advanced ideas and strong guarantees which were conceived the original design of the feature, or outright impossible due to implementation details of Conan 1.0.
The most recent case which has been requested a few times is here.
https://github.com/conan-io/conan/issues/8214
Many users want to use SCM for it's nice declarative syntax, and the ability to run a recipe "out of source", but they want to filter out some files from affecting the Conan recipe revision, just like with `exports_sources`. This filtering is currently impossible and is in conflict with the strong guarantees SCM intends to make.
Nonetheless, this feature will keep getting requested unless the documentation explains clearly:
1. That this is not currently allowed by design
2. That the design is intended to provide guarantees about reproducibility, and how those are beneficial
3. The resulting tradeoff that users then must accept when using SCM
4. That even if we wanted to add the filtering feature, it's impossible in Conan 1.x to record and upload the SCM revision information with the package without generating a new Conan `RREV`, because we can only record it to a file, and thus it will always affect the manifest, and thus always affect the hash of `conan_exports.tgz`. We would need some a new server API to support any other behavior.
I think we should definitely document something about the above situation and add it to the docs ASAP.
We may also want to add more. For example, there is another major case of unfortunate surprise, was the discovery of an unexpected and detrimental implication relating to GIT workflows and package promotion between Conan repositories when using SCM. Here's an example from earlier experimentation from our CI training course.
1. feature branch is created -> SCM revision ABC -> Conan calculates RREV 001 and creates it -> upload to temp repo
2. pull request -> SCM revision ABC -> Conan calculates and finds RREV 001 -> Use to rebuild downstream consumers
3. merge PR -> SCM revision XYZ -> Conan calculates RREV 002, cannot find it, cannot promote it
In step 3 above, the logical and straightforward thing to do for most cases would be to promote the Conan package of RREV 001 to a "production" repository. It was already built and thoroughly tested with the sources which were changed, and then it was used to build and test all downstream consumers. There is no general/intrinsic need to rebuild the package and all it's dependencies one more time based on the merge commit.
However, because the merge commit changed the Conan revision to RREV 002, the result of promoting RREV 001 would be really awkward and unfortunate. There would be no Conan RREV built which corresponds to the `HEAD` of the main branch. The entire dependency tree would need to be rebuilt based on the merge commit to create ones. Both options are unacceptable for many organizations. Thus, the advice we should give to organizations who want to do package promotion with GIT workflows is to use `exports_sources` instead of `SCM`.
So, we should clearly explain in the documentation that SCM is not the best fit for all use-cases, and tat it specifically leads to this situation with respect to package promotion, and that `exports_sources` is still a viable option which avoids this situation.
Contributor guide
No contributing guide indexed for this repository
Research direction
No documentation file or test is named. Start by locating the existing SCM and exports_sources documentation, then use issue 8214 and the described promotion workflow to explain SCM's reproducibility guarantees, filtering limitation, Conan 1.x constraint, tradeoffs, and when exports_sources is preferable; done means these implications and guidance are clearly documented.
Written by the indexing model from the issue text.
Assessment
- Tech stack
- git
- Domain
- documentation
- Issue type
- Documentation
- Difficulty
- 3/5
- Estimated time
- 1-2 days
- Activity status
- Stale
- Clarity
- Mostly clear
- Newbie friendliness
- 35/100