InternLM / InternLM/lmdeploy

[Feature] 希望加入生成参数allowed_token_ids

Open
#2,905 0 comments 1 reaction 0 assignees View on GitHub
Dominant language
Python
Stars
8.1k
Forks
748
Avg merge
6d 2h
Merged PRs (30d)
54

Description

### Motivation

vLLM中存在生成参数allowed_token_ids,可以控制LLM在推理时仅从给定的token中生成。
类似于LMDeploy中bad_token_ids的反选?

我在LMDeploy的bad_token_ids加入了除我希望生成的token以外所有token的ids,但这会导致程序崩溃。

希望能在LMDeploy的生成参数中加入类似allowed_token_ids功能的参数,或者能否告诉我现有的版本能够如何实现这样的功能。

### Related resources

vLLM: [https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py#L87](https://github.com/vllm-project/vllm/blob/main/vllm/sampling_params.py#L87)

### Additional context

_No response_

Contributor guide

Open the contributing guide

Assessment

This issue has not been assessed yet.

Get new issues in your inbox

A short digest of beginner-friendly GitHub issues.