Improve image pulling process
- Dominant language
- Python
- Stars
- 670
- Forks
- 183
- Avg merge
- 17h 7m
- Merged PRs (30d)
- 358
Description
# Steps
> [!NOTE]Step 1 targets 24.03, the remaining steps target 24.12.
## Step 1
- Allow superadmins to update status of kernels from `PULLING` to `CANCELLED` forcibly
## Step 2
- Implement `check-and-pull` APIIt checks whether an agent has a specific image or not.If the agent does not have the image, it starts a background task that pulls the image and returns the background task id.After the background task finishes, the agent should dispatch `ImagePulled` event.
- Check images early when creating kernelsWhen a kernel is scheduled, manager first calls the `check-and-pull` API. This agent-side API spawns a background task that verifies whether the agent has the required image. If the agent does not have the image, the task pulls it. The API produces an `ImagePullStartedEvent` when the agent begins pulling the image and an `ImagePullFinishedEvent` when the task completes.
- Consume Image eventsManager consumes `ImagePullStartedEvent` and `ImagePullFinishedEvent` events.When `ImagePullStartedEvent` consumed, transits kernel status to `PULLING`.When `ImagePullFinishedEvent` consumed, transits kernel status to `PREPARED`.
- `start-containers` loopManager runs a global asynchronous background loop that monitors all kernels with `PREPARED` status. It starts these kernels.
### Subtasks
- [x] #3128
- [ ] Update `USER_RESOURCE_OCCUPYING_KERNEL_STATUSES`Exclude `PULLING` kernel status from `USER_RESOURCE_OCCUPYING_KERNEL_STATUSES` since `PULLING` status kernels don't occupy any resources!
### As-is
```mermaid
sequenceDiagram
loop "prepare" loop
activate Manager
Manager->>Manager: Fetch "SCHEDULED" sessions
Manager->>+Agent: Create kernel
opt need to pull
Agent->>Event bus: Produce "Pulling" event
Agent->>Agent: Pull the image and wait till it finishes
Agent->>Event bus: Produce "Preparing" event
end
Agent->>Agent: Create kernel
Agent-->>-Manager: Return kernel creation info
deactivate Manager
end
```
### To-do
```mermaid
sequenceDiagram
loop "check-precond" loop
activate Manager
Manager->>Manager: Fetch "SCHEDULED" sessions and transit status to "PREPARING"
Manager->>Agent: check-and-pull
activate Agent
Note right of Agent: Run in background task
Agent-->>Manager: Return background task id
deactivate Manager
end
opt need to pull
Agent->>Event bus: Produce "Pulling" event
Agent->>Agent: Pull the image and wait till it finishes
end
Agent->>Event bus: Produce "Pull finished" event
deactivate Agent
loop "create-kernel" loop
activate Manager
Manager->>Manager: Fetch "PREPARED" sessions
Manager->>+Agent: Create kernel
Agent->>Agent: Create kernel
Agent-->>-Manager: Return kernel creation info
deactivate Manager
end
```
## Step 3
- Show image pulling progress (refs https://github.com/lablup/backend.ai/issues/227)
## Step 4
- Implement Image preload/unload API (refs https://github.com/lablup/backend.ai/issues/39)
- Create agents-images mapping table to let multiple agents preload images
JIRA Issue: BA-118
Contributor guide
Assessment
This issue has not been assessed yet.