We store too many large secrets in etcd

The primary scale limiter on large dense clusters is secrets (namespaces * 3_service_accounts_per_namespace * 3_secrets_per_service_account * ~2-10k, i.e. on a large 10k namespace cluster it's 90% of the cluster size). We can try to reduce the constant scale elements now (gzip, ca.crts secrets per SA, SAs) and target structural changes that fit our long term direction (to remove service account tokens from storage and embrace dynamic secret generation).

Drafted some thoughts in kubernetes/kubernetes#48408 for the latter. Need to think through the former. This gates scale growth on any of our existing clusters because we can't afford a 500mb download everytime we want to list secrets.

This is an online-first item (newly created!)