Revisit production infrastructure architecture / tooling

Currently, our infrastructure is split in two:
- some machines are oriented towards k8s, with a control plane provided by Scaleway and machines provided only by Scaleway
- other machines are provided by Scaleway and other providers, and managed manually

I see multiple drawbacks which could be opportunities for discussions / improvements:
- we use the heavy lifted k8s which is rather designed for big clusters of 100s of machines with workloads moving everywhere while in our case we have assigned workloads and only 4 machines
- this allows us to benefit from inventions designed for the masses
- this comes with a significant burden in terms of maintenance
- many feature we do not mind about but are not tailored to our use case so we need to tweak them
- simple things like not overwhelming a machine with many old software versions become something complex and risky to tackle
- this comes with a significant risk for production (significant accidental complexity for rather very simple services)
- we have limited opportunities to implement things we are supposed to be more straightforward (storage HA, IPv6)
- we pay a lot (1260€/year) for a control plane without any SLA, no HA and some serious limitations (IPv6 not supported, limited etcd size, no backup)
- only k8s machines are monitored in Grafana because it was the most urgent ones, if the technology was homogeneous everything would be monitored

One perspective I have in mind is the question of using something maybe simpler or more tailored to our "one machine - one role" use case, like k3s. Or even simplify the stack further down with even simpler tools.

The issue names no files, tests, or entry points. Start by mapping the current Kubernetes and manually managed machines, monitoring coverage, control-plane costs and limitations, then compare the existing setup with the proposed simpler alternatives; done means reaching and documenting an agreed production architecture.

Written by the indexing model from the issue text.