Optimization on Kubernetes

Though we already support deploying TiFlash on Kubernetes with TiDB Operator, there are still some optimizations that need to be done to run TiFlash on Kubernetes more gracefully:

1. The default configurations handling.
   Currently, there are many configuration parameters that need to be set default values explicitly by both TiDB Operator and TiUP (check this big file to set default config), which is tedious and may not sync up with the behavior change in TiFlash, for example, someday TiFlash may change the default value of one parameter but its default value is not updated in TiDB Operator or TiUP in time.
2. Many processes are combined in one container
   This is not a good practice to run applications in the container, usually, one process per container.
   Currently, there are 3 processes and 4 log files in the TiFlash container, which brings the following restrictions:

• The health check of Kubernetes cannot work as expected because it can only check one process, although we can write a script to check all of the 3 processes, it's not the best practice anyway.
• The 4 log files are written to the disk (we can print the proxy logs to stderr and we still have 3), however, usually, the applications are printed to stdout/stderr and users will deploy central log service, e.g. EFK/ELK, to collect all of the logs in the Kubernetes cluster, and they only collect the logs from stdout/stderr by default so the logs of TiFlash will not be collected, we now have to add 4 sidecar containers to tail the logs to work around this issue.

Is it possible to deploy the 3 processes in 3 containers in one Pod?

Start by reviewing the linked TiDB Operator file, pkg/manager/member/tiflash_util.go, along with the TiUP configuration handling mentioned in the issue. Define the design for Kubernetes deployment before changing code; done means defaults stay aligned with TiFlash and the three processes, health checks, and logs can be handled through separate containers and stdout/stderr.