Site Worker not cleaning up after itself (competition bundle)

Sometimes, the site worker leaves behind competition bundles files inside its container.

There are multiple ways we can fix this :

Limit competition bundle upload size

We can limit the competition bundle upload size, making users upload the bigger files (usually datasets and the like) directly from the resources tab, which won't be handled by the site worker.

This implies checking the size of a bundle before it's uploaded to refuse it if it's too big and show an error describing the necessary steps to take to fix this error (remove the big files from the bundle and upload them from another tab).

Fix whatever is crashing the site worker for big competitions bundles

The first problem is that the memory limit is pretty low ( 256 MB as defined in the docker-compose.yml ). Raising this limit should fix some of the problems. However, if the site worker crashes for another reason while processing the unpacking, the leftover files won't be cleaned up and will cause problems afterward.

• Memory raised to 15 GB

Cleaning up at start

• We should run a clean up script at the (re)start of the site worker

Either way, we need to improve the site worker so that it can better manage the space it takes and better clean up after itself when it encounters problems.

Start by locating the site worker startup and restart path, then review docker-compose.yml and the existing competition-bundle handling. Determine how cleanup should behave after a restart or processing failure, and verify that leftover bundle files are removed without disrupting valid resources.