Support for ipcMode in AWS batch

New feature

I would like to see the ipcMode=Host setting supported in AWS batch to enable shared memory between processes.

Usage scenario

This came up when trying to run a workflow with many parallel STAR align processes. STAR has a shared memory option that can be controlled with the "--genomeLoad" flag. For example by setting "--genomeLoad LoadAndKeep", the process will only load the genome into memory if it has not already been loaded by another process, and upon completion it will remove it from memory only if it is not in use by another process. This works fine with Docker containers on the same host by using "docker run --ipc host ...", and AWS Batch supports an 'ipcMode: "host"' setting in the Batch job definition. It would be nice to enable this option via nextflow. Using a pre-loaded genome reduces startup time by about 1 minute and reduces the ram needed for each process from about 32 GB to about 4 GB, so the impact is significant when there are many STAR align processes.

Suggest implementation

It appears there are two versions of Batch job definitions, using either ContainerProperties or EcsProperties as described here: Creating job definitions using EcsProperties - AWS Batch. Nextflow is using the legacy "ContainerProperties" job definition, and this does not support ipcMode or other options described in the link above (not sure if any of the other impacted options are significant for nextflow users: dependsOn, essential, name, and pidMode).

I believe enabling ipcMode would require adding support for the new EcsProperties job definition. This could continue to support the current containerOptions directive, and ipcMode could be added to the supported containerOptions.

In addition to updating the job definition format, it appears there are overrides applied to the job definition at runtime whose structure would also need to change, from ContainerOverrides to TaskContainerOverrides. (This last point prevents the workaround of manually updating the job definition to EcsProperties format with "ipcMode=host" and continuing to run NextFlow, as this raises the error "Container overrides should not be set for ecsProperties jobs.")

Start by tracing Nextflow's AWS Batch job-definition creation and runtime override handling, focusing on ContainerProperties versus EcsProperties and ContainerOverrides versus TaskContainerOverrides. The work is done when ipcMode=host can be supplied through supported container options and AWS Batch jobs run with the requested shared IPC mode without override errors.