[5.x]: "Generating Image Transform" Queue Constantly Running

Description

I am managing a site with ~100,000 image assets on a craft install.

Recently, I upgraded to Craft 5 and moved the transform process from AWS Lambda to the site's server. Ever since, the queue has been full of "Generating image transform" jobs.

Of course, I expect this after the initial migration, and the server gradually worked through a queue of ~500,000 jobs. But even after getting the queue down to 0, it is still regularly filling up with thousands of jobs (weeks later).

Moreover, the process stalls frequently, and I have to manually release jobs in order for it to continue.

The queue log is very opaque so this has been hard to debug. The only errors/warnings i am seeing have to do with reading .gif images, and are as follows:

2024-10-02 14:05:25 [queue.WARNING] [/home/ploi/patbates.com/vendor/pixelandtonic/imagine/src/Imagick/Imagine.php] Unable to open image /home/ploi/patbates.com/storage/runtime/assets/sources/321985.gif {"memory":8107552} 
2024-10-02 14:05:25 [queue.WARNING] [/home/ploi/patbates.com/vendor/pixelandtonic/imagine/src/Imagick/Imagine.php:72] memory allocation failed `/home/ploi/patbates.com/storage/runtime/assets/sources/321985.gif' @ error/gif.c/ReadGIFImage/1306 {"memory":8107880} 

Craft CMS version

5.4.5.1

PHP version

8.3.8

Operating system and version

Linux 5.15.0-118-generic

Database type and version

MySQL 8.0.39

Image driver and version

Imagick 3.7.0 (ImageMagick 6.9.11-60)

Installed plugins and versions

• CKEditor 4.2.0
• DigitalOcean Spaces Filesystem 3.0.0
• Feed Me 6.4.0
• SEOmatic dev-develop-v5

Start with the queue warnings and the referenced vendor/pixelandtonic/imagine/src/Imagick/Imagine.php:72 path, focusing on the failing 321985.gif asset and the reported memory allocation error. Trace why image-transform jobs continue to be enqueued and why processing stalls; done means identifying the cause and confirming that the queue no longer continually refills or requires manual job release.