Improved Firewall handling

Feature Request

Is your feature request related to a problem? Please describe:

1. tiup cluster check --apply only temporarily disables firewalld.service. It does a systemctl stop, but not a systemctl disable/systemctl mask. This leaves this as a booby trap that will explode on next reboot.
2. Disabling the firewall doesn't seem to be the right action. Changing the zone from public to trusted seems to be a better option.
3. Some high security deployments might not allow one to disable the firewall
4. The docs aren't clear that a host based firewall isn't recommended, but a network based firewall around the cluster is.

Describe the feature you'd like:

1. Make sure whatever tiup cluster check --apply is done in a persistent way that survives reboots
2. Don't disable the firewall. Change the zone instead.
3. Extend documentation for what ports need to be open, both for host based firewalls and for network based firewalls around the cluster.
4. Allow one to operate with a firewall enabled.
5. Create service definitions for the firewall service which then can be added to the zone.

Why the featue is needed:

Describe alternatives you've considered:

Teachability, Documentation, Adoption, Migration Strategy:

Related:

• https://github.com/pingcap/tiup/issues/1329
• https://github.com/pingcap/docs/pull/20081
• https://github.com/pingcap/tiup/blob/8e15a038a9fcd7c4562fc9839263609fb9eafe37/pkg/cluster/task/check.go#L105
• https://docs.pingcap.com/tidb/stable/best-practices-for-security-configuration#protect-internal-ports
• https://docs.pingcap.com/tidb/stable/hardware-and-software-requirements#network-requirements
• https://docs.pingcap.com/tidb/stable/best-practices-for-security-configuration#restrict-access-control

Start with pkg/cluster/task/check.go at the linked line and review related issue #1329, then compare the existing firewall behavior with the linked security and network documentation. Done means the requested firewall-enabled operation, persistent configuration, service definitions, and documentation for host- and network-based firewall ports are addressed.