dimensionalOS / dimensionalOS/dimos

memory2 - per-stream metadata kv store

Open
#1,977 0 comments 0 reactions 0 assignees View on GitHub

Nobody has claimed this yet.

enhancement memory
Dominant language
Python
Stars
4.5k
Forks
808
Avg merge
3d 5h
Merged PRs (30d)
233

Description

want to attach arbitrary key/value config to a stream and have it persist alongside observations.

first usage of this is record reply https://github.com/dimensionalOS/dimos/pull/1976 storing original topic names for streams (but I assume this is just first usage, embedding model name for embedding stream would be another example)

store.streams.lidar.metadata.calibration_offset = 3
store.streams.lidar.metadata.set("frame_id", "base_link")
store.streams.lidar.metadata.update({"sensor_model": "mid360", "rate_hz": 10})
store.streams.lidar.metadata["calibration_offset"]  # 3

distinct from per-observation Observation.tags (dimos/memory2/type/observation.py:57), which is immutable-derive (obs.tag(k=v) returns a new observation). keeping the APIs separate on purpose — observations are immutable, stream metadata is mutable config that's set once and read often.

requirements:

  • attribute + dict-style get/set, plus set() / update() / delete()
  • persisted by the backend (sqlite store gets a kv table; null/memory stores keep a dict)
  • json-serializable values only at first; reject anything else loudly
  • survives store reopen

should we...?

  • make it Notifier-watchable so subscribers get told when metadata changes?
  • expose it on the Backend directly so transforms/embedders can read it (e.g. a vectorstore reads embedding_model from stream metadata)?

related: #1652 (postgres store would also need this).


Synced from DIM-845 by summer

Contributor guide

Open the contributing guide

First steps

  1. Read the whole issue, then the project's contributing guide.
  2. Comment on the issue to say you are picking it up — it saves two people doing the same work.
  3. Fork the repository and make your change on a branch.
  4. Open a pull request that references the issue number.

Research direction

Start with dimos/memory2/type/observation.py:57 to understand how metadata differs from observation tags, then review the memory2 store implementations for sqlite, null, and memory persistence. Done means attribute and dict-style access plus set(), update(), and delete() accept only JSON-serializable values, persist across store reopen, and satisfy the open decisions about Notifier and Backend exposure.

Written by the indexing model from the issue text.

Assessment

Tech stack
python, sqlite
Domain
backend, database
Issue type
Feature
Difficulty
5/5
Estimated time
Over a week
Activity status
Quiet
Clarity
Mostly clear
Newbie friendliness
45/100

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.