Has anyone gotten AMD Instinct MI50 (or MI60) to work -- under FreeBSD?

[UnitedMarsupials]

The new AMD Instinct MI50 32Gb is very cheap for 32Gb -- has anyone gotten llama.cpp work on it? My machines all run FreeBSD so, I imagine, I'll be limited to Vulkan -- but I don't have time at present to be the first to attempt it :-)

(I want high-quality responses -- which tend to require large models and FP16 -- but it does not have to be very fast... 5 tokens per second should be sufficient.)

[taronaeo]

Looks like someone already tested this and posted their benchmark here: #15021

Instinct MI50	32 GB / HBM2 / 4096 bit	1040.64 ± 1.20	87.40 ± 0.48	21c17b5	@davispuh

See #15021 (reply in thread)

[UnitedMarsupials]

I saw that, but that test used ROCm, rather than Vulkan -- which means, it was not tested on FreeBSD...

[IMbackK]

I have not tried it, but afaik freebsd ships a (old) port of amdgpu.ko, with mesa's radv on top so its very likely to just work.
while lamacpp dose use some vulkan features rarely used by graphics clients, its not terribly speical over all, so if you can find someone who has had success running any kind of vulkan workload (includeing games, vkcubem demos whatever) on a Radeon VII (same chip as mi50) you should be pretty much guaranteed that it will work

rocm will be entirely of the table as rocr/clr uses lots of linux specific syscalls.