Build error ROCm GPU

[fmotta]

Prerequisites

Before submitting your issue, please ensure the following:

• [ X] I am running the latest version of PowerInfer. Development is rapid, and as of now, there are no tagged versions.
• [ X] I have carefully read and followed the instructions in the README.md.
• [ X] I searched using keywords relevant to my issue to make sure that I am creating a new issue that is not already open (or closed).

Expected Behavior

I expect it to build without type errors

Current Behavior

Fails to build

Please provide a detailed written description of what PowerInfer did, instead.
Failed to build due to type errors

Environment and Context

Please provide detailed information about your computer setup. This is important in case the issue is not reproducible except for under certain specific conditions.

• Physical (or virtual) hardware you are using, e.g. for Linux:

`$ llscpu
Architecture: x86_64
CPU op-mode(s): 32-bit, 64-bit
Address sizes: 48 bits physical, 48 bits virtual
Byte Order: Little Endian
CPU(s): 32
On-line CPU(s) list: 0-31
Vendor ID: AuthenticAMD
BIOS Vendor ID: Advanced Micro Devices, Inc.
Model name: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S
BIOS Model name: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S Unknown CPU @ 3.0GHz
BIOS CPU family: 107
CPU family: 26
Model: 112
Thread(s) per core: 2
Core(s) per socket: 16
Socket(s): 1
Stepping: 0
Frequency boost: enabled
CPU(s) scaling MHz: 72%
CPU max MHz: 5187.5000
CPU min MHz: 625.0000

• Operating System, e.g. for Linux:

$ uname -a
uname -a
Linux gtkAMDStrix 6.17.0-19-generic #19~24.04.2-Ubuntu SMP PREEMPT_DYNAMIC Fri Mar 6 23:08:46 UTC 2 x86_64 x86_64 x86_64 GNU/Linux

• SDK version, e.g. for Linux:

$ python3 --version
python3 --version
Python 3.12.3

$ make --version
make --version
GNU Make 4.3
Built for x86_64-pc-linux-gnu
Copyright (C) 1988-2020 Free Software Foundation, Inc.
License GPLv3+: GNU GPL version 3 or later <http://gnu.org/licenses/gpl.html>
This is free software: you are free to change and redistribute it.
There is NO WARRANTY, to the extent permitted by law.

$ g++ --version
g++ --version
g++ (Ubuntu 13.3.0-6ubuntu2~24.04.1) 13.3.0
Copyright (C) 2023 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.

 ldd --version
ldd (Ubuntu GLIBC 2.39-0ubuntu8.7) 2.39
Copyright (C) 2024 Free Software Foundation, Inc.
This is free software; see the source for copying conditions.  There is NO
warranty; not even for MERCHANTABILITY or FITNESS FOR A PARTICULAR PURPOSE.
Written by Roland McGrath and Ulrich Drepper.

Failure Information (for bugs)

Please help provide information about the failure / bug.

Steps to Reproduce

Please provide detailed steps for reproducing the issue. We are not sitting in front of your screen, so the more detail the better.

0. python-mvenv powerinfer && source powerinfer/bin/activate
1. git pull
2. run build (I made a sh file to do this
   cat make.sh

Replace '1100' to your card architecture name, you can get it by rocminfo

CC=/opt/rocm/llvm/bin/clang CXX=/opt/rocm/llvm/bin/clang++ cmake -S . -B build -DLLAMA_HIPBLAS=on -DAMDGPU_TARGETS=gfx1151
4. source make.sh script
5. Watch it fail due to strong type errors

Failure Logs

~/PowerInfer# git status
On branch main
Your branch is up to date with 'origin/main'.

Untracked files:
(use "git add ..." to include in what will be committed)
make.sh

nothing added to commit but untracked files present (use "git add" to track)
(powerinfer) root@gtkAMDStrix:~/PowerInfer# cat make.sh

Replace '1100' to your card architecture name, you can get it by rocminfo

CC=/opt/rocm/llvm/bin/clang CXX=/opt/rocm/llvm/bin/clang++ cmake -S . -B build -DLLAMA_HIPBLAS=on -DAMDGPU_TARGETS=gfx1151
cmake --build build --config Release
(powerinfer) root@gtkAMDStrix:/PowerInfer# rocminfo | grep gfx
Name: gfx1151
Name: amdgcn-amd-amdhsa--gfx1151
Name: amdgcn-amd-amdhsa--gfx11-generic
(powerinfer) root@gtkAMDStrix:/PowerInfer# source make.sh
-- The C compiler identification is Clang 20.0.0
-- The CXX compiler identification is Clang 20.0.0
-- Detecting C compiler ABI info
-- Detecting C compiler ABI info - done
-- Check for working C compiler: /opt/rocm/llvm/bin/clang - skipped
-- Detecting C compile features
-- Detecting C compile features - done
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /opt/rocm/llvm/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD
-- Performing Test CMAKE_HAVE_LIBC_PTHREAD - Success
-- Found Threads: TRUE
CMake Warning (dev) at /opt/rocm/lib/cmake/hip/hip-config-amd.cmake:70 (message):
AMDGPU_TARGETS is deprecated. Please use GPU_TARGETS instead.
Call Stack (most recent call first):
/opt/rocm/lib/cmake/hip/hip-config.cmake:138 (include)
CMakeLists.txt:361 (find_package)
This warning is for project developers. Use -Wno-dev to suppress it.

-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS
-- Performing Test HIP_CLANG_SUPPORTS_PARALLEL_JOBS - Success
-- HIP and hipBLAS found
AMD LLD 20.0.0 (/longer_pathname_so_that_rpms_can_support_packaging_the_debug_info_for_all_os_profiles/src/llvm-project/llvm 27682a16360e33e37c4f3cc6adf9a620733f8fe1) (compatible with GNU linkers)
-- CMAKE_SYSTEM_PROCESSOR: x86_64
-- x86 detected
-- Configuring done (0.8s)
-- Generating done (0.1s)
-- Build files have been written to: /root/PowerInfer/build
[ 1%] Building CXX object CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o
/root/PowerInfer/ggml-cuda.cu:7718:13: error: no matching function for call to 'hipblasGemmEx'
7718 | cublasGemmEx(g_cublas_handles[id], CUBLAS_OP_T, CUBLAS_OP_N,
| ^~~~~~~~~~~~
/root/PowerInfer/ggml-cuda.cu:31:22: note: expanded from macro 'cublasGemmEx'
31 | #define cublasGemmEx hipblasGemmEx
| ^~~~~~~~~~~~~
/root/PowerInfer/ggml-cuda.cu:222:32: note: expanded from macro 'CUBLAS_CHECK'
222 | cublasStatus_t err_ = (err);
| ^~~
/opt/rocm/include/hipblas/hipblas.h:23710:32: note: candidate function not viable: no known conversion from 'hipDataType' to 'hipblasComputeType_t' for 18th argument
23710 | HIPBLAS_EXPORT hipblasStatus_t hipblasGemmEx(hipblasHandle_t handle,
| ^
/root/PowerInfer/ggml-cuda.cu:8655:9: error: no matching function for call to 'hipblasGemmStridedBatchedEx'
8655 | cublasGemmStridedBatchedEx(g_cublas_handles[id], CUBLAS_OP_T, CUBLAS_OP_N,
| ^~~~~~~~~~~~~~~~~~~~~~~~~~
/root/PowerInfer/ggml-cuda.cu:33:36: note: expanded from macro 'cublasGemmStridedBatchedEx'
33 | #define cublasGemmStridedBatchedEx hipblasGemmStridedBatchedEx
| ^~~~~~~~~~~~~~~~~~~~~~~~~~~
/root/PowerInfer/ggml-cuda.cu:222:32: note: expanded from macro 'CUBLAS_CHECK'
222 | cublasStatus_t err_ = (err);
| ^~~
/opt/rocm/include/hipblas/hipblas.h:24072:32: note: candidate function not viable: no known conversion from 'hipDataType' to 'hipblasComputeType_t' for 22nd argument
24072 | HIPBLAS_EXPORT hipblasStatus_t hipblasGemmStridedBatchedEx(hipblasHandle_t handle,
| ^
/root/PowerInfer/ggml-cuda.cu:8689:9: error: no matching function for call to 'hipblasGemmBatchedEx'
8689 | cublasGemmBatchedEx(g_cublas_handles[id], CUBLAS_OP_T, CUBLAS_OP_N,
| ^~~~~~~~~~~~~~~~~~~
/root/PowerInfer/ggml-cuda.cu:32:29: note: expanded from macro 'cublasGemmBatchedEx'
32 | #define cublasGemmBatchedEx hipblasGemmBatchedEx
| ^~~~~~~~~~~~~~~~~~~~
/root/PowerInfer/ggml-cuda.cu:222:32: note: expanded from macro 'CUBLAS_CHECK'
222 | cublasStatus_t err_ = (err);
| ^~~
/opt/rocm/include/hipblas/hipblas.h:23881:32: note: candidate function not viable: no known conversion from 'hipDataType' to 'hipblasComputeType_t' for 19th argument
23881 | HIPBLAS_EXPORT hipblasStatus_t hipblasGemmBatchedEx(hipblasHandle_t handle,
| ^
3 errors generated when compiling for gfx1151.
gmake[2]: *** [CMakeFiles/ggml-rocm.dir/build.make:76: CMakeFiles/ggml-rocm.dir/ggml-cuda.cu.o] Error 1
gmake[1]: *** [CMakeFiles/Makefile2:626: CMakeFiles/ggml-rocm.dir/all] Error 2
gmake: *** [Makefile:146: all] Error 2
(powerinfer) root@gtkAMDStrix:/PowerInfer# cat /etc/lsb-release
DISTRIB_ID=Ubuntu
DISTRIB_RELEASE=24.04
DISTRIB_CODENAME=noble
DISTRIB_DESCRIPTION="Ubuntu 24.04.4 LTS"
(powerinfer) root@gtkAMDStrix:/PowerInfer# neofetch
.-/+oossssoo+/-. root@gtkAMDStrix
:+ssssssssssssssssss+: ----------------
-+ssssssssssssssssssyyssss+- OS: Ubuntu 24.04.4 LTS x86_64
.ossssssssssssssssssdMMMNysssso. Host: NucBox_EVO-X2 Version 1.0
/ssssssssssshdmmNNmmyNMMMMhssssss/ Kernel: 6.17.0-19-generic
+ssssssssshmydMMMMMMMNddddyssssssss+ Uptime: 4 days, 23 hours, 34 mins
/sssssssshNMMMyhhyyyyhmNMMMNhssssssss/ Packages: 2236 (dpkg), 12 (snap)
.ssssssssdMMMNhsssssssssshNMMMdssssssss. Shell: bash 5.2.21
+sssshhhyNMMNyssssssssssssyNMMMysssssss+ Theme: Adwaita [GTK3]
ossyNMMMNyMMhsssssssssssssshmmmhssssssso Icons: Adwaita [GTK3]
ossyNMMMNyMMhsssssssssssssshmmmhssssssso CPU: AMD RYZEN AI MAX+ 395 w/ Radeon 8060S (32) @ 5.187GHz
+sssshhhyNMMNyssssssssssssyNMMMysssssss+ GPU: AMD ATI c5:00.0 Device 1586
.ssssssssdMMMNhsssssssssshNMMMdssssssss. Memory: 26598MiB / 127435MiB
/sssssssshNMMMyhhyyyyhdNMMMNhssssssss/
+sssssssssdmydMMMMMMMMddddyssssssss+
/ssssssssssshdmNNNNmyNMMMMhssssss/
.ossssssssssssssssssdMMMNysssso.
-+sssssssssssssssssyyyssss+-
:+ssssssssssssssssss+:
.-/+oossssoo+/-.

(powerinfer) root@gtkAMDStrix:/PowerInfer# /opt/rocm/llvm/bin/clang++ --version
AMD clang version 20.0.0git (https://github.com/RadeonOpenCompute/llvm-project roc-7.1.1 25444 27682a16360e33e37c4f3cc6adf9a620733f8fe1)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm-7.1.1/lib/llvm/bin
Configuration file: /opt/rocm-7.1.1/lib/llvm/bin/clang++.cfg
(powerinfer) root@gtkAMDStrix:/PowerInfer# /opt/rocm/llvm/bin/clang --version
AMD clang version 20.0.0git (https://github.com/RadeonOpenCompute/llvm-project roc-7.1.1 25444 27682a16360e33e37c4f3cc6adf9a620733f8fe1)
Target: x86_64-unknown-linux-gnu
Thread model: posix
InstalledDir: /opt/rocm-7.1.1/lib/llvm/bin
Configuration file: /opt/rocm-7.1.1/lib/llvm/bin/clang.cfg