ByteDance-Seed / ByteDance-Seed/m3-agent

generate_memory_qwen is missing and memorization_intermediate_outputs stuck

Open
#2 6 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
1.4k
Forks
117
PR merge metrics
No merged PRs in 30d

Description

```
python data_preparation/generate_memory_qwen.py \
--data_file data/data.jsonl
```
Hey,I can't find generate_memory_qwen code in the repo.

Besides that when i run below command. I encounter bellow timeout issue that very large base64 encoded string was passed to the llm api call which lead to timeout after many tries. I think that may not be an issue but I am not using it in the right way.
```
python m3_agent/memorization_intermediate_outputs.py --data_file data/data.jsonl
```
```
'gpt-4o-2024-11-20': , 'text-embedding-3-large': }
/home/ubuntu/miniconda3/envs/rex/lib/python3.12/site-packages/onnxruntime/capi/onnxruntime_inference_collection.py:121: UserWarning: Specified provider 'CUDAExecutionProvider' is not in available provider names.Available providers: 'AzureExecutionProvider, CPUExecutionProvider'
warnings.warn(
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ubuntu/.insightface/models/buffalo_l/1k3d68.onnx landmark_3d_68 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ubuntu/.insightface/models/buffalo_l/2d106det.onnx landmark_2d_106 ['None', 3, 192, 192] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ubuntu/.insightface/models/buffalo_l/det_10g.onnx detection [1, 3, '?', '?'] 127.5 128.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ubuntu/.insightface/models/buffalo_l/genderage.onnx genderage ['None', 3, 96, 96] 0.0 1.0
Applied providers: ['CPUExecutionProvider'], with options: {'CPUExecutionProvider': {}}
find model: /home/ubuntu/.insightface/models/buffalo_l/w600k_r50.onnx recognition ['None', 3, 112, 112] 127.5 127.5
set det-size: (640, 640)
{'video_found': True, 'audio_found': True, 'metadata': {'major_brand': 'isom', 'minor_version': '512', 'compatible_brands': 'isomiso2mp41', 'encoder': 'Lavf58.76.100'}, 'inputs': [{'streams': [{'input_number': 0, 'stream_number': 0, 'stream_type': 'video', 'language': None, 'default': True, 'size': [1578, 720], 'bitrate': 6940, 'fps': 30.0, 'codec_name': 'hevc', 'profile': '(Main)', 'metadata': {'Metadata': '', 'handler_name': 'VideoHandler', 'vendor_id': '[0][0][0][0]'}}, {'input_number': 0, 'stream_number': 1, 'stream_type': 'audio', 'language': None, 'default': True, 'fps': 44100, 'bitrate': 131, 'metadata': {'Metadata': '', 'handler_name': 'SoundHandler', 'vendor_id': '[0][0][0][0]'}}], 'input_number': 0}], 'duration': 30.07, 'bitrate': 7081, 'start': 0.0, 'default_video_input_number': 0, 'default_video_stream_number': 0, 'video_codec_name': 'hevc', 'video_profile': '(Main)', 'video_size': [1578, 720], 'video_bitrate': 6940, 'video_fps': 30.0, 'default_audio_input_number': 0, 'default_audio_stream_number': 1, 'audio_fps': 44100, 'audio_bitrate': 131, 'video_duration': 30.07, 'video_n_frames': 902}
/home/ubuntu/miniconda3/envs/rex/lib/python3.12/site-packages/imageio_ffmpeg/binaries/ffmpeg-linux-x86_64-v7.0.2 -i data/clips/robot/bedroom_01/3.mp4 -loglevel error -f image2pipe -vf scale=1578:720 -sws_flags bicubic -pix_fmt rgb24 -vcodec rawvideo -
MoviePy - Writing audio in /tmp/tmprfzc94mi.wav
MoviePy - Done.
2025-08-18 20:37:54,893 - mmagent.utils.chat_api - INFO - Calling API for model: gpt-4o-2024-11-20, timeout: 10s, messages length: 2, total chars: 35487331
INFO:mmagent.utils.chat_api:Calling API for model: gpt-4o-2024-11-20, timeout: 10s, messages length: 2, total chars: 35487331
2025-08-18 20:38:28,377 - mmagent.utils.chat_api - INFO - Calling API for model: gpt-4o-2024-11-20, timeout: 10s, messages length: 2, total chars: 35487331
INFO:mmagent.utils.chat_api:Calling API for model: gpt-4o-2024-11-20, timeout: 10s, messages length: 2, total chars: 35487331
2025-08-18 20:39:03,755 - mmagent.utils.chat_api - INFO - Calling API for model: gpt-4o-2024-11-20, timeout: 10s, messages length: 2, total chars: 35487331
INFO:mmagent.utils.chat_api:Calling API for model: gpt-4o-2024-11-20, timeout: 10s, messages length: 2, total chars: 35487331
```
```
{"id": "bedroom_01", "video_path": "data/clips/robot/bedroom_01/72.mp4", "clip_path": "data/clips/robot/bedroom_01", "mem_path": "data/videos/memory_graphs/bedroom_01.pkl", "intermediate_outputs": "data/videos/intermediate_outputs/robot/bedroom_01"}
```

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.