SYCL Compiler Support, main branch (2024.11.05.)

[krasznaa]

Triggered by the release of oneAPI 2025.0.0, which deprecates the <CL/sycl.hpp> include, I set out to make sure that the project would work with both oneAPI 2025.0.0 and AdaptiveCpp 24.06.0.

I had to do the following:

• Replace all <CL/sycl.hpp> includes with <sycl/sycl.hpp>;
• Replace all uses of the cl::sycl:: namespace with ::sycl::;
• Teach the CMake code how to detect AdaptiveCpp, which now has a new command line flag (--acpp-version) for collecting the necessary info;
• Make sure that I would use consistent argument types in vecmem::device_atomic_ref function calls with the type that the atomic reference is created on top of;
  • The AdaptiveCpp code seem to be a little too templated in this respect. 🤔 As the following fails to compile (see sycl::atomic_ref implemented in a too rigid way (2024.11.06.) AdaptiveCpp/AdaptiveCpp#1591):

vecmem::device_atomic_ref<unsigned int> foo(...);
foo.atomic_add(1);

• Had to make it possible to turn off the use of UBSAN in the tests, as AdaptiveCpp seems to make the same mistake with exporting its symbols that we ourselves fixed in Symbol Visibility Update, main branch (2024.06.13.) #282;
  • Update: According to SYCL link error with -fsanitize=undefined when targeting offload backends AdaptiveCpp/AdaptiveCpp#1283, it's a different issue after all. But the ability to turn off the usage of UBSAN on our end is still necessary.
• Had to make sure that <sycl/sycl.hpp> would be included "early enough" in one of our tests, otherwise device_atomic_ref.hpp would select the wrong backend. Which I'm very surprised didn't hit us with oneAPI... 😕

With all of this in place, the code works relatively well like the following for instance:

> export SYCLCXX=/software/AdaptiveCpp/24.06.0/x86_64-el9-clang18-opt/bin/acpp
> export SYCLFLAGS="--acpp-targets='omp;cuda:sm_75;hip:gfx1031'"
> cmake -DCMAKE_BUILD_TYPE=RelWithDebInfo -DVECMEM_TEST_UBSAN=FALSE -DVECMEM_DEBUG_MSG_LVL=1 ../vecmem/
-- The CXX compiler identification is Clang 18.1.8
-- Detecting CXX compiler ABI info
-- Detecting CXX compiler ABI info - done
-- Check for working CXX compiler: /software/llvm/18.1.8/x86_64-el9-gcc11-opt/bin/clang++ - skipped
-- Detecting CXX compile features
-- Detecting CXX compile features - done
...
-- The SYCL compiler identification is AdaptiveCpp 24.06.0
-- Check for working SYCL compiler: /software/AdaptiveCpp/24.06.0/x86_64-el9-clang18-opt/bin/acpp
-- Check for working SYCL compiler: /software/AdaptiveCpp/24.06.0/x86_64-el9-clang18-opt/bin/acpp - works
-- Performing Test VECMEM_HAVE_SYCL_EXT_ONEAPI_PRINTF
-- Performing Test VECMEM_HAVE_SYCL_EXT_ONEAPI_PRINTF - Failed
...

> ACPP_VISIBILITY_MASK=cuda ./bin/vecmem_test_sycl
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: NVIDIA RTX A5000
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: NVIDIA RTX A5000
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: NVIDIA RTX A5000
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: NVIDIA RTX A5000
Running main() from /mnt/hdd1/krasznaa/projects/vecmem/build/_deps/googletest-src/googletest/src/gtest_main.cc
[==========] Running 186 tests from 9 test suites.
[----------] Global test environment set-up.
[----------] 8 tests from sycl_containers_test
[ RUN      ] sycl_containers_test.shared_memory
[       OK ] sycl_containers_test.shared_memory (310 ms)
[ RUN      ] sycl_containers_test.device_memory
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 8 bytes from 0x7f5275600400 to 0x7f5275800200
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 40 bytes from 0x7f5275600000 to 0x7f5275800400
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 40 bytes from 0x7f5275800000 to 0x7f5275600200
[       OK ] sycl_containers_test.device_memory (2 ms)
[ RUN      ] sycl_containers_test.atomic_shared_memory
[       OK ] sycl_containers_test.atomic_shared_memory (0 ms)
...
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 56 bytes from 0x7f5275601374 to 0x7f525a006c00
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 40 bytes from 0x7f52756013c4 to 0x7f525a006e00
[       OK ] sycl_soa_copy_tests_jagged/soa_copy_tests_jagged.host_to_fixed_device_to_resizable_device_to_host/7 (3 ms)
[----------] 32 tests from sycl_soa_copy_tests_jagged/soa_copy_tests_jagged (107 ms total)

[----------] Global test environment tear-down
[==========] 186 tests from 9 test suites ran. (1048 ms total)
[  PASSED  ] 186 tests.

> ACPP_VISIBILITY_MASK=hip ./bin/vecmem_test_sycl
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: AMD Radeon RX 6700 XT
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: AMD Radeon RX 6700 XT
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: AMD Radeon RX 6700 XT
[vecmem] sycl/src/utils/sycl/queue_wrapper.sycl:30 Created an "owning wrapper" around a queue on device: AMD Radeon RX 6700 XT
Running main() from /mnt/hdd1/krasznaa/projects/vecmem/build/_deps/googletest-src/googletest/src/gtest_main.cc
[==========] Running 186 tests from 9 test suites.
[----------] Global test environment set-up.
[----------] 8 tests from sycl_containers_test
[ RUN      ] sycl_containers_test.shared_memory
[       OK ] sycl_containers_test.shared_memory (183 ms)
[ RUN      ] sycl_containers_test.device_memory
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 8 bytes from 0x7f297f7d0000 to 0x7f2869a01000
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 40 bytes from 0x7f297f7d4000 to 0x7f2869a02000
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 40 bytes from 0x7f2869a00000 to 0x7f297f7d2000
[       OK ] sycl_containers_test.device_memory (1 ms)
[ RUN      ] sycl_containers_test.atomic_shared_memory
[       OK ] sycl_containers_test.atomic_shared_memory (4 ms)
...
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 56 bytes from 0x7f2869a01b74 to 0x7f297f7d7000
[vecmem] sycl/src/utils/sycl/async_copy.sycl:92 Performed memory copy of 40 bytes from 0x7f2869a01bc4 to 0x7f297f7d5000
[       OK ] sycl_soa_copy_tests_jagged/soa_copy_tests_jagged.host_to_fixed_device_to_resizable_device_to_host/7 (5 ms)
[----------] 32 tests from sycl_soa_copy_tests_jagged/soa_copy_tests_jagged (166 ms total)

[----------] Global test environment tear-down
[==========] 186 tests from 9 test suites ran. (561 ms total)
[  PASSED  ] 186 tests.

The OpenMP backed tests don't all succeed, but that's something to worry about later... 🤔

[paulgessinger]

Looks good, as far as I can tell.