[docs/bug?] New no-volumes implementation deletes the pod after every step, old implementation keeps the completed job and doesn't clean it

Previously the pod would only be deleted after all the steps are run. This did create an issue since the entrypoint is not reaping, it is just tail; and I would recommend switching to a reaping entrypoint if possible.

However, currently the pod is removed and rescheduled after every step. I don't think this is sound; it creates a lot of extra friction with the scheduler, When you are trying to fit containers efficiently, rescheduling the same pod constantly is not very efficient. If the point of that is maximum isolation, it might be ok to enable that through some environment variable.

This really works against the usage of services. Since I would expect services to stay up throughout the entire execution of the container job, not restarted after every step

(Misunderstood)

In the new implementation (0.8), for containerStep it uses a pod, while on the previous implementation (0.7) it was using a batch job, and not cleaning the jobs after it finishes. At least if there is an error they appear to remain there forever... Is there anything meant to clean these up over time?

Can we provide some documentation explaining how the container step hooks are actually placed

Start by reading the containerStep hook behavior described in the issue and compare the new 0.8 pod implementation with the previous 0.7 batch-job behavior. Document how the container step hooks are placed, including pod or job cleanup expectations and service lifetime, and make the resulting workflow understandable to users.