[Enhancement suggestion] Improve API integration experience
- Dominant language
- Python
- Stars
- 133k
- Forks
- 15.7k
- Avg merge
- 1d 6h
- Merged PRs (30d)
- 155
Description
When using the ComfyUI API to process multiple images with multiple ComfyUI servers (imagine processing 100k images with 100 ComfyUI instances).
It will face lots of challenges with the API.
The key obstacles that I'm facing are:
+ When deploying multiple ComfyUI instances with Docker environment and load balancer. It's hard to identify the prompt job's status. Because the /history API is unable to identify the prompt job, it is waiting for processing or doesn't exist.
+ When processing large numbers of images. There is no API that can help to know about ComfyUI job queue status. If one ComfyUI instance has lots of jobs waiting to be processed, the loadbalancer can find another instance to do the job.
+ There are no limitations to restricting the ComfyUI job queue size if you submit too many jobs. The queue will OOM to kill the ComfyUI.
So based on the above obstacles, I suggest providing some enhancements regarding the ComfyUI API.
+ Add a new unique ComfyUI instance ID:
When you call the API to submit a new prompt job. and query the history to check if the job was finished or not. If you can't find the job in history, if we can add an instance ID in the /history API then you can have an extra flag to check if the server is the same instance or not. to avoid when the server reboots or is replaced in multiple instances.
+ Add ComfyUI job queue size limitations:
To avoid submitting lots of prompt jobs, let the job queue out of the memory. If we submit lots of jobs, ComfyUI can reject new prompt jobs to avoid this and make it configable.
+ Add instance ID and job queue status to the /system_stats API:
So we can know how many jobs are waiting in the queue to avoid submitting new jobs to busy instance. And identify the instance.
Contributor guide
Assessment
This issue has not been assessed yet.