rate-limited pod log stream

Describe the Feature You Want

There should be a way to set a rate limit on PodLogStream.

e.g., if you get more than, X bytes per second or more than X lines per second, PodLogStream backs off

Current Behavior

If you're trying to develop a very chatty container, PodLogStream will try to live-stream all your logs from the APIServer, and send them down to the Tilt UI.

Why Do You Want This?

At some thresholds, this could not only overload the Tilt UI. it could also overload the apiserver and Docker log streamer.

In infra tooling in general (e.g., CI), it's pretty common to put limits on the amount of logs stored. But I've seen many different UI solutions for solving the problem:

• Stream all the logs but do backoff, so we only stream them in chunks of every 15s
• Add a notice that we've truncated the logs, and a link to how to get them from the command-line
• Add a notice that we've stopped fetching logs, and a button to start streaming them again

Additional context
Related issues:

• https://github.com/tilt-dev/tilt/issues/3814 (we currently support a CLI to dynamically add and remove containers from the pod log streaming, but it's not exposed from the Tiltfile)
• https://github.com/tilt-dev/tilt/issues/4579 (specifically for CI environments)
• https://github.com/tilt-dev/tilt/issues/4220 (we could optimize how Tilt stores and streams logs)
• https://github.com/tilt-dev/tilt/issues/3916 (this has some links to how the underlying kubernetes log streamer works on the apiserver side, and the performance characteristics of it)

Start by locating the PodLogStream entry point and tracing how logs move from the Kubernetes APIServer to the Tilt UI. Review the related issues 3814, 4579, 4220, and 3916, then clarify which rate-limit behavior and user-facing outcome should define done.