agentscope-ai / agentscope-ai/ReMe

feat(multimodal): build multimodal capabilities for ReMe

Open
#345 0 comments 0 reactions 1 assignee Claimed by @imrewce View on GitHub
enhancement
Dominant language
Python
Stars
3.4k
Forks
298
Avg merge
19h 52m
Merged PRs (30d)
55

Description

**Background**

ReMe currently focuses primarily on text-based memory. To support agents that interact with images, audio, video, and mixed-content documents, ReMe needs a unified multimodal memory capability that preserves its local-first and file-native principles. Multimodal source files should remain user-owned and authoritative, while derived representations such as embeddings, indexes, captions, and metadata remain rebuildable.

**Changes**
- Define extensible schemas and interfaces for multimodal memory content and metadata
- Support ingesting and storing images, audio, video, and mixed-content documents without replacing the original user-owned files
- Add pluggable preprocessing and embedding pipelines for different modalities
- Enable multimodal indexing, retrieval, and cross-modal search
- Expose multimodal capabilities through the existing service, client, and job interfaces
- Add configuration, documentation, and focused tests for the new workflows

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.