Skip to content

Import a device allocation when the params ask for one - #394

Closed
Geramy wants to merge 1 commit into
ROCm:mainfrom
Geramy:fix/import-device-allocation
Closed

Geramy wants to merge 1 commit into
ROCm:mainfrom
Geramy:fix/import-device-allocation

Conversation

@Geramy

@Geramy Geramy commented Aug 21, 2026

Copy link
Copy Markdown

hrx_allocator_import_buffer always builds a HOST_ALLOCATION external buffer:

iree_hal_external_buffer_t ext = {
    .type = IREE_HAL_EXTERNAL_BUFFER_TYPE_HOST_ALLOCATION,
    ...
    .handle.host_allocation.ptr = host_ptr,
};

So a device address cannot be imported at all — the allocator rejects a device pointer described as host memory. That leaves no way to bring one device's memory into another's allocator, which is exactly what a cross-device copy needs: hrx_queue_copy / hrx_stream_copy_buffer will not accept another device's hrx_buffer_t, and passing one faults the GPU rather than returning an error.

When the requested memory type is device-local and not host-visible, the import now describes an IREE_HAL_EXTERNAL_BUFFER_TYPE_DEVICE_ALLOCATION. Host imports take the same path they always did.

Paired with #393 (device-pointer export), a peer copy becomes expressible through the C API: get the source's device pointer, import it into the destination's allocator, copy on the destination's queue.

Built and exercised on 8x gfx1201 (Radeon AI PRO R9700, ROCm 7.13).

hrx_allocator_import_buffer always built a HOST_ALLOCATION external buffer, so
a device address could not be imported at all -- the allocator rejects a device
pointer described as host memory. That leaves no way to bring a peer's memory
into this allocator, which is what a cross-device copy needs: a queue will not
touch another device's buffer object, and handing it one faults the GPU.

When the requested memory type is device-local and not host-visible, the import
now describes a DEVICE_ALLOCATION instead. Host imports are unchanged.

With this and the device-pointer export, a peer copy is expressible through the
C API: get the source's device pointer, import it into the destination's
allocator, copy on the destination's queue.
@Geramy

Geramy commented Aug 21, 2026

Copy link
Copy Markdown
Author

Withdrawing this — testing it on 8x gfx1201 showed it does not achieve what the description claims, and the reason suggests it is the wrong shape entirely.

With the change applied, the import reaches iree_hal_amdgpu_allocator_import_device_allocation and is then refused there:

PERMISSION_DENIED; device allocation is owned by an HSA GPU agent
outside the AMDGPU HAL logical topology

Each hrx_device_t is its own logical device with a single-GPU topology, so one device's allocator will never accept another's memory no matter how the handle is described. The blocker is the topology, not the external-buffer type.

And once that is addressed the right way — a logical device whose iree_hal_amdgpu_topology_t spans several GPUs, which the HAL already supports — a buffer allocated on any of those GPUs is already a HAL buffer of that device, so hrx_stream_copy_buffer works directly and there is nothing to import. This patch would be dead code on the correct design.

Leaving the observation here in case it is useful: there is currently no way through the C API to create a device spanning more than one GPU (hrx_gpu_device_get is one device per GPU), which is what a cross-GPU copy on this stack appears to need.

#393 is unaffected and stands on its own — it is what lets a caller reach hsa_amd_memory_async_copy at all, worth 41 GB/s against ~8 for the blit path on the same link.

@Geramy Geramy closed this Aug 21, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant