lablup / lablup/backend.ai

Fix Registry Scanning Tag Validation

Open
#5,986 0 comments 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
670
Forks
183
Avg merge
17h 7m
Merged PRs (30d)
358

Description

## Motivation

Registry scanning processes currently fail when encountering images with non-conforming tag formats, causing incomplete registry synchronization and preventing users from accessing valid container images. The scanning logic needs to be updated to handle the relaxed validation policy and continue processing despite tag format variations.

## Required Features

- **Robust scanning logic**: Update registry scanners to handle validation warnings gracefully
- **Complete synchronization**: Ensure all valid images are discovered and processed
- **Error resilience**: Continue scanning when individual tags fail validation
- **Comprehensive logging**: Log tag validation issues without stopping the process
- **Progress tracking**: Provide visibility into scanning progress and any skipped items

## Impact

### Primary Components

- `src/ai/backend/manager/registry/` - Registry scanning modules
- Registry connector implementations (Docker Hub, Harbor, ECR, etc.)
- Image metadata collection and storage logic

### Secondary Components

- Automated registry synchronization jobs
- Manual registry refresh operations
- Image availability reporting in admin interface

### Services Affected

- Manager service registry scanning components
- Scheduled registry synchronization tasks
- Real-time image discovery processes

## Testing Scenarios

### Registry Compatibility Tests

- Scan Docker Hub repositories with various tag formats successfully
- Handle Harbor private registries with custom tag patterns
- Process ECR repositories with AWS-specific tag conventions
- Test GitLab Container Registry compatibility

### Error Recovery Tests

- Continue scanning when individual images have validation issues
- Process mixed repositories with both conforming and non-conforming tags
- Handle network interruptions during scanning gracefully
- Verify partial scan results are preserved and can be resumed

### Performance Tests

- Ensure scanning performance doesn't degrade with relaxed validation
- Test scanning of large repositories (1000+ images)
- Verify memory usage remains stable during extended scanning
- Measure impact on scanning time with additional validation logic

### Integration Tests

- End-to-end registry addition and scanning workflow
- Verify scanned images appear correctly in GUI
- Test image selection and session creation with newly scanned images

JIRA Issue: BA-2461

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.