Support larger workflows

# Summary

It seems there's a limit on the size of Argo workflows (in terms of number of tasks and task specification size). Argo compresses the workflow `status.nodes` field into `status.compressedNodes`, but the `spec.templates` field can still become very large if there are a lot of tasks and/or if the tasks have large specs.

https://argoproj.github.io/argo-workflows/offloading-large-workflows/ seems to only apply to the `status.nodes` field, not to `spec.templates`.

Would it be possible for the `spec.templates` field to also be gzip+b64 encoded, like `status.nodes`? I think this would allow for much larger workflows. Offloading to Postgres would be another option to scale even bigger, though that may not be necessary.

# Use Cases

I'm using Argo via Kubeflow Pipelines, and a workflow with ~200 tasks exceeds the limit of what KFP/Argo can run: I get the error `offload node status is not supported`, [more details here](https://github.com/kubeflow/pipelines/issues/4170#issuecomment-954994233). KFP adds some metadata annotations which makes each task template spec kind of large (the KFP devs [suggested a hack here](https://github.com/kubeflow/pipelines/issues/4170#issuecomment-655764762) to make this a bit smaller).

Other users have run into this problem as well. Example from the CNCF slack on 2021-10-29: https://cloud-native.slack.com/archives/C01QW9QSSSK/p1635517397302400

---

**Message from the maintainers**:

Impacted by this bug? Give it a 👍. We prioritise the issues with the most 👍.

No source files, tests, or entry points are named. Start by tracing Argo's existing status.nodes gzip/base64 compression and how spec.templates is persisted, then compare template encoding with PostgreSQL offloading. Done means large workflows no longer hit the reported size limit through a supported approach.

Written by the indexing model from the issue text.