alibaba / alibaba/loongsuite-python

[Feature] Refactor Mem0 instrumentation for OpenTelemetry GenAI Memory semantic conventions

Open
#243 2 comments 0 reactions 0 assignees View on GitHub
enhancement genai instrumentaion
Dominant language
Python
Stars
133
Forks
35
Avg merge
1d 12h
Merged PRs (30d)
5

Description

## Background

LoongSuite currently instruments Mem0 with legacy custom `gen_ai.memory.*` fields and a broad method-to-operation mapping. The current semantic-convention alignment adds a minimal compatibility adapter, but it intentionally does not claim complete Mem0 support.

OpenTelemetry GenAI semantic conventions now define Memory operations and attributes, including:

- operations: `create_memory_store`, `search_memory`, `create_memory`, `update_memory`, `upsert_memory`, `delete_memory`, `delete_memory_store`
- `gen_ai.memory.store.id`
- `gen_ai.memory.record.id`
- `gen_ai.memory.record.count`
- opt-in `gen_ai.memory.query.text`
- opt-in `gen_ai.memory.records`

## Current minimal adapter

- `add` -> `upsert_memory`
- `update` / `batch_update` -> `update_memory`
- `search` / `get` / `get_all` / `history` -> `search_memory`
- `delete` / `batch_delete` / `delete_all` -> `delete_memory`
- local `Memory` / `AsyncMemory` operations use `INTERNAL`; remote `MemoryClient` / `AsyncMemoryClient` operations use `CLIENT`
- only explicit store and record identifiers are mapped
- query and record content are emitted only when content capture is enabled

This mapping is intentionally conservative. In particular, `delete_all` does not necessarily mean `delete_memory_store`, and framework-scoped `user_id` / `agent_id` / `run_id` values are not assumed to be OpenTelemetry memory-store identifiers.

## Required full redesign

- [ ] Build a supported API matrix for Mem0 1.x and 2.x, including sync/async local classes and hosted clients.
- [ ] Cover method signatures and `options=` / nested request objects without relying only on top-level keyword arguments.
- [ ] Define stable `gen_ai.memory.store.id` derivation rules for local and hosted Mem0. Do not repurpose user/session/agent identifiers without an explicit contract.
- [ ] Normalize add/search/get/batch responses to the upstream Memory record schema (`content` required; `id`, `metadata`, and `score` when available).
- [ ] Define exact operation mapping for create/update/upsert/delete/store lifecycle APIs, including bulk operations and `delete_all`.
- [ ] Preserve the PII/content boundary: query text and records remain opt-in and must not leak through custom attributes or events.
- [ ] Validate `INTERNAL` vs `CLIENT` span kind for every supported class and deployment mode.
- [ ] Redesign the optional internal vector/graph/retrieval/rerank spans so the hierarchy is useful and does not duplicate the public Memory operation.
- [ ] Add sync, async, streaming-if-applicable, error, batch, and concurrency/context-isolation tests.
- [ ] Test both the oldest and latest supported Mem0 versions in CI; use recorded or mocked fixtures where provider credentials are not available.
- [ ] Add Weaver validation against the current upstream GenAI Memory registry and update `loongsuite-semantic-conventions` with the upstream Memory fields and operation enums.
- [ ] Verify Robin synchronization and commercial metric suppression/augmentation behavior before release.

## Acceptance criteria

- No legacy LoongSuite-only Memory attributes are emitted by default.
- Every supported Mem0 API has a documented operation name, span kind, store-id rule, record normalization rule, and content-capture behavior.
- Unsupported or ambiguous APIs omit fields instead of fabricating identifiers or operation semantics.
- Focused tests and real emitted-span Weaver samples cover local/hosted, sync/async, success/error, and content-capture on/off paths.
- LoongSuite and Robin preserve the same trace semantic contract; commercial metrics remain owned by Robin and are not duplicated by the open-source adapter.

Contributor guide

Open the contributing guide

Research direction

Start by locating the current Mem0 instrumentation adapter and the loongsuite-semantic-conventions integration, then map supported Mem0 1.x and 2.x APIs across local classes and hosted clients. Review existing instrumentation tests and CI fixtures before adding coverage for sync/async, batch, error, concurrency, and content-capture paths. Validate emitted spans with Weaver and confirm Robin synchronization and metric ownership before considering the redesign complete.

Written by the indexing model from the issue text.

Assessment

Tech stack
python
Domain
observability
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
25/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.