[orchestration pipeline] Stop exporting OpenShift resources

I have circled these thoughts with a few people already, but I realised I never wrote them down in a GitHub issue. So here it goes.

At the moment, the orchestration pipeline checks for each component whether an openshift folder exists. If it does, the pipeline assumes that this folder contains templates defining the OpenShift resources. If the folder does not exist, an export of all resources labelled with app=${projectId}-${componentId} is performed, and the resulting template is stored in openshift-exported. This feature was introduced to allow users to deploy their component without needing to think about / declare the resources in code. Instead, users can use the console UI to click their app together in the DEV environment, and then get this state replicated in QA/PROD. This approach poses a problem though: typically some parameters (e.g. DB access credentials) should be different between the environments, and to overcome this problem, there is a "reverse param" magic, which replaces hardcoded values in the exported template with parameter names, so that other values can be set for those parameters in QA/PROD.

Let's make this very clear - I think this whole idea of an "ad-hoc automated export" is a terrible mistake, for several reasons:

• Inferring the required resources from the current resources in DEV s error-prone: e.g. you might have created resources for experimental reasons only, or, you might have resources which you did not label correctly. To put it more strongly, if you do not define in source code what the resources should be, you are working with an undefined state.
• Using the UI to make changes to the resources kills traceability: you don't know who did something, you don't know when they did it, and you don't know for which reasons they did it. There is no link to any software design spec / functional requirement spec, and there is no responsible / author of the change. To my understanding, this is a no-go in a GxP context.
• The "reverse param" magic works only somewhat. There can be bugs (e.g. replacements happen that you did not want, as it is a crude search-and-replace operation), it's limited in that you can't make modifications in the template that stick around (every export overwrites the existing template), and most importantly, it is quite hard to explain to users (I had many questions about it). Having a feature that only works in the most basic case but doesn't work in a typical scenario is not worthwhile IMHO.
• Creating an export requires the tailor binary to be present. Unfortunately, the orchestration pipeline was designed to run on the Jenkins Master, but the tailor binary was only installed on the Jenkins agent in the beginning. Therefore, the orchestration pipeline does this "switch to the agent and back" dance for the export operation (note that now the Jenkins Master also has the tailor binary because there was a bug with one component type and this "dance" ... but having a binary in two places is a bad place to be and I'd rather not have this in the first place ...)
• The component pipeline now supports Helm next to Tailor, and the orchestration pipeline should support Helm too. Long-term, Helm will likely replace Tailor, as OpenShift has added tight integration with Helm since v4.3. Helm has a much more complex and powerful templating engine compared to OpenShift Templates (which is what Tailor uses). I have not found a tool yet that can properly export current resources to Helm templates. Even if there is such a tool (or we would build one), it would again suffer from the problem that no "template logic" can be conveyed in an export. A live export is an even worse fit for Helm templates.
• Creating an export in the orchestration pipeline requires us to commit and push the newly created/updated template. This means that during a pipeline run, we have a commit for which the pipeline is triggered, and we have a commit which is the result of the pipeline run. This is bad for several reasons:
  • It pollutes the repository with commits from a technical user
  • As each release version has its own branch, the commit ends up in one release branch, but not in the main branch (e.g. master). This can be confusing for users not too familiar with Git. As a consequence, the release branches are merged back into the main branch during an orchestration pipeline run. I deeply regret giving in to this feature request. I believe this feature has nothing to do with orchestration, is error prone (e.g. merge conflicts), and needlessly complicates the code. Instead, the orchestration should NOT create any commits. Consequently nothing needs to be merged back (note that user-made changes in a release branch should be done through pull requests, and pull requests can be merged automatically to the main branch using Bitbucket).

To sum it up, I strongly believe this feature should be removed. Using the auto-export is not something I would recommend to anyone, especially not in a GxP context. Removing the feature would simplify the documentation, it would simplify the code, it would allow us to easier integrate Helm and would allow us to undo other mistakes (like the merge-back) in future steps.

If we agree on removing it, we should however define how we want to provision new quickstarters. I believe the best would be to ship them with some templates so that there is no initial, manual effort. We'd need to pick if we want to ship with OpenShift templates or Helm templates (or allow the user to pick? or make that dependant on the OpenShift version?). I would love to take this opportunity to ship with Helm templates out-of-the-box, at least for OpenShift v4.

@clemensutschig @metmajer @martsec @segator @henrjk @gerardcl @oalyman Please provide feedback.