deepseek-ai / deepseek-ai/DeepSeek-OCR

profiling警告:提示词长度超过 max_num_batched_tokens / max_num_seqs

Open
#260 1 comment 0 reactions 0 assignees View on GitHub
Dominant language
Python
Stars
23.9k
Forks
2.2k
PR merge metrics
No merged PRs in 30d

Description

### 问题描述
运行DeepSeek-OCR-master/DeepSeek-OCR-vllm/run_dpsk_ocr_image.py脚本,产生警告信息:
WARNING 11-13 07:10:29 [profiling.py:245] The sequence length used for profiling (max_num_batched_tokens / max_num_seqs = 745) is too short to hold the multi-modal embeddings in the worst case (796 tokens in total, out of which {'image': 693} are reserved for multi-modal embeddings). This may cause certain multi-modal inputs to fail during inference, even when the input text is short. To avoid this, you should increase `max_model_len`, reduce `max_num_seqs`, and/or reduce `mm_counts`.
WARNING 11-13 07:11:22 [profiling.py:245] The sequence length used for profiling (max_num_batched_tokens / max_num_seqs = 744) is too short to hold the multi-modal embeddings in the worst case (796 tokens in total, out of which {'image': 693} are reserved for multi-modal embeddings). This may cause certain multi-modal inputs to fail during inference, even when the input text is short. To avoid this, you should increase `max_model_len`, reduce `max_num_seqs`, and/or reduce `mm_counts`.

### 猜测问题产生原因
我使用的提示词prompt比较长,加上输入的图像之后,token长度超过了max_num_batched_tokens(8192) / max_num_seqs = 745 这个长度。

### 问题
1. 这个问题如何解决?
2. 请问max_num_seqs这个参数是什么? 是否建议修改? 如何修改?
3. max_num_batched_tokens=8192是最大允许的token数吗?
4. 我单独修改了max_num_batched_tokens=103824,max_num_batched_tokens/ max_num_seqs的值并没有翻倍,这是为什么?

谢谢!

PS:
我使用的prompt如下:
PROMPT = "\n<|grounding|>In this image, the locations where text appears include, but are not limited to: \
1. Title: The title text at the top or prominent position of the image; \
2. Labels: Axis labels (X/Y axes), module names (in the architecture diagram), legend items (in the chart), table headers (in Table); \
3. Notes: The explanatory text in the image, formulas, and references cited (such as [1]); \
4. Numbers/Scale: The numerical values on the axes, the contents of table cells, the height markings of the bars in a histogram; \
5. Others: Annotations in the image, grammar tree nodes (in NLP rules), place names on the map (in Maps), etc. \
Identify all the text information in the image and return the result in Markdown format, ensuring that all visible text is accurately extracted."

Contributor guide

No contributing guide indexed for this repository

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.