Skip to content

Commit 8415bf2

Browse files
authored
[kv_offload] Set offloading connector to prefer HND layout (vllm-project#41928)
Signed-off-by: Martin Hickey <martin.hickey@ie.ibm.com>
1 parent ac06214 commit 8415bf2

1 file changed

Lines changed: 4 additions & 0 deletions

File tree

vllm/distributed/kv_transfer/kv_connector/v1/offloading_connector.py

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -163,6 +163,10 @@ def take_events(self) -> Iterable[KVCacheEvent]:
163163
assert self.connector_scheduler is not None
164164
return self.connector_scheduler.take_events()
165165

166+
@classmethod
167+
def get_required_kvcache_layout(cls, vllm_config: VllmConfig) -> str | None:
168+
return "HND"
169+
166170
def get_kv_connector_stats(self) -> KVConnectorStats | None:
167171
if self.connector_worker is None:
168172
return None # We only emit stats from the worker-side

0 commit comments

Comments
 (0)