lablup / lablup/backend.ai

Agent: Implement mountpoint monitoring and pre-creation condition checks

Open
#7,105 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Overview

Implement mountpoint availability monitoring and non-retryable pre-creation condition checks in the Agent component to prevent kernel creation failures due to unavailable storage mounts.

## Background

When vfolders are mounted to containers, the storage mountpoints must be available and accessible. Currently, there's no proactive check mechanism to verify mountpoint availability before attempting to create a kernel, which can lead to runtime failures.

## Goals

1. **Mountpoint Health Monitoring**: Implement a status checker that continuously monitors whether storage mountpoints are available and accessible
1. **Pre-creation Condition Validation**: Add non-retryable validation in kernel creation flow that checks mountpoint status and fails fast with informative error messages

## Scope

- Agent component (`src/ai/backend/agent/`)
- Health check system (`src/ai/backend/agent/health/`)
- Kernel creation flow (`src/ai/backend/agent/agent.py`)

## Expected Benefits

- Early detection of storage mount issues
- Fast failure with clear error messages when mounts are unavailable
- Reduced failed kernel creation attempts
- Better operational visibility into storage health

## Related Components

- Storage Proxy: Manages actual storage and mount operations
- Manager: Receives error information for session scheduling decisions

JIRA Issue: BA-3244

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.