[MBUILDCACHE-27] Improve efficiency of cache with plugin and source type aware processing

**[Alexander Ashitkin](https://issues.apache.org/jira/secure/ViewProfile.jspa?name=alex_ashitkin)** opened **[MBUILDCACHE-27](https://issues.apache.org/jira/browse/MBUILDCACHE-27?redirect=false)** and commented

Currently build cache has implementation which checks source code and plugins and if it detects mismatch, whole project is rerun. This is is redundant and could be optimized in following cases:
* Test code changed. In that case optimal behavior is to rerun test in an affected module and reuse cached artifacts in case of success. Actual behavior: change in a test in upstream module invalidates all downstream projects
* Variation of the situation above is a test library project (aka test-jar) - though by convention modification is in production code, because the project is used only in test scope - modification of upstream test jar should cause just tests rerun 
* Upstream dependency changed. In that case any plugin which execution is not affected by upstream dependency, could be skip. Typically, it could be any of linter plugins - like checkstyle, which checks project source code only. Current behavior: plugin will rerun

With such granular plugin and sources demarcation, it is possible to significantly optimize cached build time.

To address this problem need following changes:
* changes in build configuration to demarcate plugins as private (no need rebuild on dependencies change)
* split checksum into at least 3 parts - test sources checksum, dependencies checksum and  production checksum
* Take into account changes in every checksum when decision of plugin execution is taken. If only dependencies changed, `source-code-only` plugins could be skipped
* When calculating dependencies between reactor projects, take into account "production" checksum only. In that case changes in test code will not affect downstream projects

Additional challenge is to rerun-tests-only mode - it obviously relies on compiled project, but probably that could be worked around by restoring cached jar

Summarizing the above and applying ideas of ABI to maven build, need to distinguish 3 categories of project inputs:
* implementation sources - part of the code resembling functionality shared within reactor (src/main and similar)
* private sources - part which is not shared (unit test and similar)
* dependencies - they change functionality but do not change code

Having this 3 categories correctly represented by a separate checksums, it is possibly to implement optimization mentioned above.

In target state, following behavior is desirable:

 
|Change|Changed project|Downstream Project|
|:---|:---|:---|
|Test sources|rerun tests + later plugins|Reuse cache|
|Test dependency |rerun tests + later plugins except "source-only" plugins|Reuse cache|
|Test library project|rerun everything|rerun tests|
|Prod sources|rerun everything|Rerun everything except "source only plugins"|
|Prod dependency|rerun everything except "source-only" plugins|Rerun everything except "source only plugins"|

This behavior could be made opt-in

 

---

**Sub-tasks:**
- [MBUILDCACHE-28](https://issues.apache.org/jira/browse/MBUILDCACHE-28) Change buildinfo model to support separate checksums

- [MBUILDCACHE-29](https://issues.apache.org/jira/browse/MBUILDCACHE-29) Capture separate checksums of test source, dependencies and other code

- [MBUILDCACHE-30](https://issues.apache.org/jira/browse/MBUILDCACHE-30) Track reactor projects using "implementation + dependencies" checksum instead of full

- [MBUILDCACHE-31](https://issues.apache.org/jira/browse/MBUILDCACHE-31) Implement skipping "source-only" plugin executions

1 votes, 3 watchers

Start with the linked sub-tasks MBUILDCACHE-28 through MBUILDCACHE-31, which cover the buildinfo model, checksum capture, reactor tracking, and source-only plugin skipping. Compare their target behavior with the change matrix in this issue; done means separate implementation, private, and dependency inputs drive the described cache and plugin decisions.

Written by the indexing model from the issue text.