Skip to content

Conversation

@yinpeiqi
Copy link

@yinpeiqi yinpeiqi commented Dec 1, 2025

PLEASE FILL IN THE PR DESCRIPTION HERE ENSURING ALL CHECKLIST ITEMS (AT THE BOTTOM) HAVE BEEN CONSIDERED.

Purpose

The related PR infos could be referred to [#92], the issue could be referred to [#74].

We add the custom process support for both qwen2.5omni model and the qwen3omni model.

Specifically, we allow user to custom a custom_preprocess function and custom_postprocess function between the forward function. In qwen3omni, we put the whole MTP stage into the custom_preprocess.

By this mean, in model_kwargs, we can remove request_ids, request_span, and addition_information_by_req.

Test Plan

Test Result


Essential Elements of an Effective PR Description Checklist
  • The purpose of the PR, such as "Fix some issue (link existing issues this PR will resolve)".
  • The test plan, such as providing test command.
  • The test results, such as pasting the results comparison before and after, or e2e results
  • (Optional) The necessary documentation update, such as updating supported_models.md and examples for a new model.
  • (Optional) Release notes update. If your change is user facing, please update the release notes draft.

BEFORE SUBMITTING, PLEASE READ https://github.com/vllm-project/vllm-omni/blob/main/CONTRIBUTING.md (anything written below this line will be removed by GitHub Actions)

Copy link

@chatgpt-codex-connector chatgpt-codex-connector bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

ℹ️ About Codex in GitHub

Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".

Comment on lines +715 to +718
req_input_ids, req_embeds, update_dict = self.model.preprocess(
input_ids=input_ids[s:e],
input_embeds=inputs_embeds[s:e],
**req_infos

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

P1 Badge Guard preprocess kwargs when additional info is absent

When has_preprocess is true the runner now always calls self.model.preprocess(..., **req_infos), but req_infos is set via getattr(req_state, "additional_information_cpu", None) a few lines above. For new requests without any additional_information this value is None, so the **req_infos expansion raises TypeError: 'NoneType' object is not a mapping before the model forward ever runs. This breaks talker-stage execution for requests that do not provide extra metadata. Default the dict to {} before unpacking or skip the call when no info is available.

Useful? React with 👍 / 👎.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant