lablup / lablup/backend.ai

Add an option to limit the max number of desired session counts for inference endpoints

Open
#1,883 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

If the user increases it to a very large number, it may exceeds the system limit to handle the pending session queue.

This would be also further constrained by #1120 as the actual limit should be the minimum of both settings: (the pending session limit, the desired session count limit).

JIRA Issue: BA-155

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.