AOSSIE-Org / AOSSIE-Org/PictoPy
Performance: image_util_is_valid_image opens every image with PIL during folder scan, causing O(n) full reads on large libraries
- Langage dominant
- Python
- Étoiles
- 283
- Forks
- 679
- Merge moyen
- 7 j 2 h
- PR mergées (30 j)
- 3
Description
## Summary
The image_util_is_valid_image function in backend/app/utils/images.py (lines 444-459) performs a full PIL Image.open() + img.verify() call on every file during a folder scan. This causes each image to be read twice during initial ingestion - once for validation and once for thumbnail generation + metadata extraction - resulting in severe performance degradation on large photo libraries.
## Current Behavior
```python
# backend/app/utils/images.py - lines 444-459
def image_util_is_valid_image(file_path: str) -> bool:
allowed_extensions = {".jpg", ".jpeg", ".png"}
file_extension = Path(file_path).suffix.lower()
if file_extension not in allowed_extensions:
return False
# This does a full disk read for EVERY file in the folder:
try:
with Image.open(file_path) as img:
img.verify()
return True
except Exception:
return False
```
This is called inside image_util_get_images_from_folder -> image_util_process_folder_images. After validation, image_util_prepare_image_records then calls image_util_generate_thumbnail (another full Image.open()) and image_util_extract_metadata (another Image.open() + img.getexif()). Each image is opened 3 separate times on initial ingestion.
The video pipeline already handles this correctly - video_util_is_valid_video only checks the file extension + size, deferring decode validation to thumbnail generation where failure is non-fatal.
## Expected Behavior
image_util_is_valid_image should mirror video_util_is_valid_video: validate by file extension + non-zero size only, deferring PIL validation to thumbnail generation (already handles failure gracefully).
```python
def image_util_is_valid_image(file_path: str) -> bool:
allowed_extensions = {".jpg", ".jpeg", ".png"}
if Path(file_path).suffix.lower() not in allowed_extensions:
return False
try:
return os.path.getsize(file_path) > 0
except OSError:
return False
```
## Impact
- Large libraries (1,000-50,000+ images): Folder scan time scales linearly with full PIL image decodes. On an HDD or NAS, this can add minutes of extra latency before any images appear.
- Cold disk cache: Each Image.open + verify forces a full sequential file read before thumbnail generation forces the same read again.
- Re-sync: On every sync-folder call, all files are re-validated via PIL even if unchanged.
## Related Bug: db_toggle_image_favourite_status bypasses _connect() helper
A secondary bug in backend/app/database/images.py (line 498): db_toggle_image_favourite_status uses sqlite3.connect(DATABASE_PATH) directly instead of the _connect() helper. The _connect() helper enables PRAGMA foreign_keys = ON, which enforces ON DELETE CASCADE. Bypassing it means cascade deletes may silently fail on this code path.
```python
# Line 498 - uses raw connect, not _connect()
def db_toggle_image_favourite_status(image_id: str) -> bool:
conn = sqlite3.connect(DATABASE_PATH) # should be _connect()
```
## Suggested Fixes
Fix 1 - Replace PIL verify with extension + size check in image_util_is_valid_image (matches the video pipeline pattern).
Fix 2 - Replace sqlite3.connect(DATABASE_PATH) with _connect() in db_toggle_image_favourite_status.
## Environment
- Component: backend/app/utils/images.py, backend/app/database/images.py
- Related (already correct): backend/app/utils/videos.py - video_util_is_valid_video
Guide de contribution
Ouvrir le guide de contribution
Évaluation
Cette issue n'a pas encore été évaluée.