AOSSIE-Org / AOSSIE-Org/PictoPy

[Enhancement] Add batch processing and embedding caching to face clustering for better performance on large galleries

Open
#961 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
283
Forks
679
Avg merge
7d 2h
Merged PRs (30d)
3

Description

### Description
Face clustering with FaceNet and DBSCAN is great for grouping similar faces offline, but for users with large photo libraries, sequential embedding computation can lead to long wait times and high memory usage. Implementing batch processing (feed multiple images to FaceNet/ONNX at once) and caching (store computed embeddings in SQLite to skip recomputes on unchanged images) would optimize this, making PictoPy more scalable.

### Proposed Changes
- In the clustering service (e.g., backend/app/services/clustering.py or equivalent):
- Modify embedding extraction to use batching: Load images in chunks (e.g., via OpenCV imread in loop, then np.stack for ONNX input).
- Add caching: Before computing, check/query SQLite for existing embeddings by image hash/path; store new ones post-compute.
- Handle batch size as configurable (e.g., via .env, default 32 based on hardware).
- Update any related routers/endpoints to return progress (if async) or handle interruptions.
- Add unit tests in backend/tests/ for batch vs. single performance and cache hits.
- Ensure compatibility with CPU/GPU via ONNX Runtime.

### References
- Recent clustering fix: Commit 7649865 (improve accuracy with thresholds).
- ONNX batching docs: https://onnxruntime.ai/docs/ort/python/api_summary.html#inference
- Similar optimizations in other ML apps (e.g., via torch DataLoader analogs).

I'm experienced with Python/ML and would love to implement it via PR if assigned!

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.