[Question]: Device-api collectives and kernels

[whatdhack]

Question

1. Is there any test one can run to compare device-api allreduce ( e.g. example 06) vs host-api allreduce ?
2. Is there any example kernel that uses device-api collective ( e.g. allreduce + gemm) ?

[xiaofanl-nvidia]

Is there any test one can run to compare device-api allreduce ( e.g. example 06) vs host-api allreduce ?

See https://github.com/NVIDIA/nccl-tests/blob/master/src/all_reduce.cu#L21.
But please note that the point of this device API test is not to beat host-api. Some of the benefits of device APIs are: 1) allow users to write custom communication kernels to explore new algorithms or satisfy specific use cases; 2) compute-comms fusion.

Is there any example kernel that uses device-api collective ( e.g. allreduce + gemm) ?

We plan to release an example soon. ++ @teojgo for viz.

BTW, nccl_ep uses device APIs - https://github.com/NVIDIA/nccl/tree/master/contrib/nccl_ep It can give you an idea of what you can use device API for.