Skip to content

Remove deprecated prompt_token_ids wrapping in vLLM backend#1191

Open
sihyeonn wants to merge 1 commit intohuggingface:mainfrom
sihyeonn:fix/vllm-deprecated-prompt-token-ids
Open

Remove deprecated prompt_token_ids wrapping in vLLM backend#1191
sihyeonn wants to merge 1 commit intohuggingface:mainfrom
sihyeonn:fix/vllm-deprecated-prompt-token-ids

Conversation

@sihyeonn
Copy link
Copy Markdown

Context

Follow-up to #1173 which upgraded vLLM to >=0.11.0. A few call sites still used the old prompt_token_ids wrapping patterns that are no longer needed.

Relates to #1002

What changed

vLLM's PromptType includes list[int] directly in DecoderOnlyPrompt, so we can pass token ID lists to LLM.generate() without any wrapping:

PromptType = DecoderOnlyPrompt | EncoderDecoderPrompt
DecoderOnlyPrompt = str | TextPrompt | list[int] | TokensPrompt | ExplicitEncoderDecoderPrompt

Removed:

  • {"prompt_token_ids": ...} dict wrapping in the ray data-parallel path (vllm_model.py)
  • TokenInputs(prompt_token_ids=...) wrapping + its import in the single-GPU path (vllm_model.py)
  • {"prompt_token_ids": ...} dict wrapping in the LLM-as-judge vLLM path (llm_as_judge.py)

Now all three call sites just pass list[list[int]] directly, which is simpler and works across vLLM 0.11 through 0.17+.

Test plan

  • Verified against vLLM docs that list[int] is a valid PromptType from 0.10.2 onwards
  • Existing vLLM tests in the repo should continue to pass (no behavioral change, just removing unnecessary wrapping)

vLLM's PromptType natively accepts list[int] as a valid input since
v0.10.2 (see vllm-project/vllm DecoderOnlyPrompt type definition),
so wrapping token IDs in {"prompt_token_ids": ...} dicts or internal
TokenInputs objects is unnecessary.

This is a follow-up cleanup to PR huggingface#1173 which upgraded vllm to >=0.11.0
but left some deprecated patterns behind.

Relates to huggingface#1002

Signed-off-by: Sihyeon Jang <sihyeon.jang@navercorp.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant