Qwen3 next

[grimoire]

Qwen3-next require kernels from:

https://github.com/Dao-AILab/causal-conv1d
https://github.com/fla-org/flash-linear-attention

We need env check for different model-device combinations.

[windreamer]

Do we need to consider to integrate ssm cache pool in PD migration request?

[lvhan028]

opencompass evaluation failed

2025-11-01 22:35:12,565 - lmdeploy - ERROR - engine.py:1234 - exception happened: <class 'IndexError'> index 1032 is out of bounds for axis 0 with size 1032
Traceback (most recent call last):
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 1229, in async_loop
    await self._async_loop_main(resp_que=resp_que,
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 1128, in _async_loop_main
    forward_inputs, next_running = await inputs_maker.prefetch_next_inputs()
                                   ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 320, in prefetch_next_inputs
    return await self._send_next_inputs_impl(prefill, True)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 285, in _send_next_inputs_impl
    forward_inputs = self._make_forward_inputs(prefill, enable_empty)
                     ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 227, in _make_forward_inputs
    return self.engine._make_forward_inputs(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/engine/engine.py", line 897, in _make_forward_inputs
    scheduler_output = scheduler.schedule(is_prefill=prefill, prealloc_size=prealloc_size)
                       ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/paging/scheduler.py", line 299, in schedule
    output = self._schedule_prefill(0)
             ^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/utils.py", line 271, in __func_warpper
    return func(*args, **kwargs)
           ^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/paging/scheduler.py", line 232, in _schedule_prefill
    if not __evict_for_seq(seq, waiting):
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/paging/scheduler.py", line 213, in __evict_for_seq
    return eviction_helper.evict_for_seq(seq, evictable, prealloc_size)
           ^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^^
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/paging/eviction_helper/recompute_eviction_helper.py", line 74, in _evict_for_ssm
    state_manager.free(evict_seq)
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/paging/state_manager.py", line 55, in free
    self.allocator.free(seq.logical_state)
  File "/nvme1/lvhan/lmdeploy/lmdeploy/pytorch/paging/state_manager.py", line 29, in free
    self._free_states[num_used] = state_id
    ~~~~~~~~~~~~~~~~~^^^^^^^^^^
IndexError: index 1032 is out of bounds for axis 0 with size 1032

serving:

 lmdeploy serve api_server Qwen/Qwen3-Next-80B-A3B-Instruct --tp 4

[grimoire]

Fixed

[lvhan028]

The OC evaluation test failed due to an issue where quite a few prompts from aime2025 dataset became trapped in repetition.

[grimoire]

Fixed