Skip to content

Commit 3a85f47

Browse files
nez0bTysonRayJones
nez0b
and
TysonRayJones
authored
Optimised bitwise ops with BMI2 PEXT/PDEP (#796)
When the new QUEST_ENABLE_BMI2 option is ON, the (new) bitwise functions getValueOfPossiblySortedBits() and (a new overload of) insertBitsWithMaskedValues() will use the BMI2 PEXT and PDEP instructions. This accelerates the CPU amplitude indexing logic, likely visible when simulation small Quregs, where memory costs do not dominate. The GPU backend does not make use of these instructions. This PR was the winning submission to UnitaryHACK 2026, challenge #717. This PR also made changes to QUEST_OUTPUT_LIB_NAME when QUEST_APPEND_CONFIG_TO_LIB_NAME is ON: - "+mt" was renamed to "+omp" - "+bmi2", "+adios2", and "+subcomm" were added (missed in previous commits) PoJen added the intrinsics, performed all benchmarking, and prototyped the extensive cpu_subroutines.cpp changes. Tyson refactored, and updated the build, macros and CI. --------- Co-authored-by: Tyson Jones <tyson.jones.input@gmail.com>
1 parent 9307f5e commit 3a85f47

9 files changed

Lines changed: 299 additions & 43 deletions

File tree

‎.github/workflows/compile.yml‎

Lines changed: 53 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -39,7 +39,11 @@ jobs:
3939
# test only compilation succeeds (no execution)
4040
build-test:
4141
name: >
42-
${{ matrix.os == 'ubuntu-latest' && 'Linux' || matrix.os == 'macos-latest' && 'MacOS' || 'Windows' }}
42+
${{ startsWith(matrix.os, 'ubuntu' ) && 'Linux' ||
43+
startsWith(matrix.os, 'macos' ) && 'MacOS' ||
44+
startsWith(matrix.os, 'windows') && 'Windows' ||
45+
'Unknown' }}
46+
${{ endsWith(matrix.os, 'intel') && '(Intel)' || '' }}
4347
[${{ matrix.precision }}]
4448
${{ matrix.omp == 'ON' && 'OMP' || '' }}
4549
${{ matrix.mpi == 'ON' && 'MPI' || '' }}
@@ -48,6 +52,7 @@ jobs:
4852
${{ matrix.hip == 'ON' && 'HIP' || '' }}
4953
${{ matrix.cuquantum == 'ON' && 'CUQ' || '' }}
5054
${{ matrix.adios2 == 'ON' && 'CKPT' || '' }}
55+
${{ matrix.bmi2 == 'ON' && 'BMI' || '' }}
5156
5257
runs-on: ${{ matrix.os }}
5358

@@ -59,16 +64,18 @@ jobs:
5964
# (causes CUDA and MPI installation to fail on Windows)
6065
max-parallel: 8
6166

62-
# compile QuEST with all combinations of below flags
67+
# compile QuEST with all combinations of below flags (Intel runners for BMI2 instrinsics);
68+
# incredibly, this (with exclusions below) achieves 256 combos, which is the Github limit!
6369
matrix:
64-
os: [windows-2022, ubuntu-latest, macos-latest]
70+
os: [windows-2022, ubuntu-latest, macos-latest, macos-15-intel, macos-26-intel]
6571
precision: [1, 2, 4]
6672
omp: [ON, OFF]
6773
mpi: [ON, OFF]
6874
cuda: [ON, OFF]
6975
hip: [ON, OFF]
7076
cuquantum: [ON, OFF]
7177
adios2: [ON, OFF]
78+
bmi2: [ON, OFF]
7279
mpilib: ['', 'mpich', 'ompi', 'impi', 'msmpi']
7380

7481
# disable deprecated API on MSVC, and assign unique compilers,
@@ -82,6 +89,12 @@ jobs:
8289
- os: macos-latest
8390
compiler: clang++
8491
deprecated: ON
92+
- os: macos-15-intel
93+
compiler: clang++
94+
deprecated: ON
95+
- os: macos-26-intel
96+
compiler: clang++
97+
deprecated: ON
8598
- os: windows-2022
8699
compiler: cl
87100
deprecated: OFF
@@ -104,14 +117,42 @@ jobs:
104117
# cannot use GPU on MacOS
105118
- cuda: ON
106119
os: macos-latest
120+
- cuda: ON
121+
os: macos-15-intel
122+
- cuda: ON
123+
os: macos-26-intel
107124
- hip: ON
108125
os: macos-latest
126+
- hip: ON
127+
os: macos-15-intel
128+
- hip: ON
129+
os: macos-26-intel
109130

110131
# cannot use cuquantum on Windows or MacOS
111132
- cuquantum: ON
112133
os: windows-2022
113134
- cuquantum: ON
114135
os: macos-latest
136+
- cuquantum: ON
137+
os: macos-15-intel
138+
- cuquantum: ON
139+
os: macos-26-intel
140+
141+
# cannot use BMI2 on non-Intel MacOS
142+
- bmi2: ON
143+
os: macos-latest
144+
145+
# use ONLY BMI2 on Intel MacOS, just to shrink matrix (Github imposes 256 max)
146+
- bmi2: OFF
147+
os: macos-15-intel
148+
- bmi2: OFF
149+
os: macos-26-intel
150+
151+
# do not combine BMI2 with MPI or ADIOS2 (they don't interact), just to shrink matrix
152+
- bmi2: ON
153+
mpi: ON
154+
- bmi2: ON
155+
adios2: ON
115156

116157
# don't enumerate MPI libraries when not using MPI
117158
- mpi: OFF
@@ -132,6 +173,14 @@ jobs:
132173
mpilib: 'msmpi' # MacOS: [MPICH, OpenMPI]
133174
- os: macos-latest
134175
mpilib: 'impi'
176+
- os: macos-15-intel
177+
mpilib: 'msmpi'
178+
- os: macos-15-intel
179+
mpilib: 'impi'
180+
- os: macos-26-intel
181+
mpilib: 'msmpi'
182+
- os: macos-26-intel
183+
mpilib: 'impi'
135184
- os: windows-2022
136185
mpilib: 'mpich' # Windows: [Intel MPI, MS MPI]
137186
- os: windows-2022
@@ -252,6 +301,7 @@ jobs:
252301
-DQUEST_ENABLE_HIP=${{ matrix.hip }}
253302
-DQUEST_ENABLE_CUQUANTUM=${{ matrix.cuquantum }}
254303
-DQUEST_ENABLE_ADIOS2=${{ matrix.adios2 }}
304+
-DQUEST_ENABLE_BMI2=${{ matrix.bmi2 }}
255305
-DCMAKE_CUDA_ARCHITECTURES=${{ env.cuda_arch }}
256306
-DCMAKE_HIP_ARCHITECTURES=${{ env.hip_arch }}
257307
-DCMAKE_CXX_COMPILER=${{ matrix.compiler }}

‎.github/workflows/test_free.yml‎

Lines changed: 17 additions & 6 deletions
Original file line numberDiff line numberDiff line change
@@ -27,9 +27,14 @@ jobs:
2727
# excluding the v4 integration tests, for free
2828
serial-unit-test:
2929
name: >
30-
${{ matrix.os == 'ubuntu-latest' && 'Linux' || matrix.os == 'macos-latest' && 'MacOS' || 'Windows' }}
30+
${{ startsWith(matrix.os, 'ubuntu' ) && 'Linux' ||
31+
startsWith(matrix.os, 'macos' ) && 'MacOS' ||
32+
startsWith(matrix.os, 'windows') && 'Windows' ||
33+
'Unknown' }}
34+
${{ endsWith(matrix.os, 'intel') && '(Intel)' || '' }}
3135
[${{ matrix.precision }}]
3236
serial
37+
${{ matrix.bmi2 == 'ON' && '(BMI)' || '' }}
3338
unit v${{ matrix.version }}
3439
3540
runs-on: ${{ matrix.os }}
@@ -38,17 +43,22 @@ jobs:
3843
# continue other jobs if any fail
3944
fail-fast: false
4045

41-
# we will compile QuEST with all precisions but no parallelisation
46+
# we will compile QuEST with all precisions but no parallelisation (though with Intel BMI2)
4247
matrix:
43-
os: [ubuntu-latest, macos-latest, windows-latest]
44-
version: [3, 4]
48+
os: [ubuntu-latest, macos-latest, windows-latest, macos-15-intel, macos-26-intel]
49+
version: [3, 4]
4550
precision: [1, 2, 4]
51+
bmi2: [ON, OFF]
4652

47-
# MSVC cannot compile deprecated v3 tests
4853
exclude:
54+
# MSVC cannot compile deprecated v3 tests
4955
- os: windows-latest
5056
version: 3
51-
57+
58+
# cannot use BMI2 on non-Intel MacOS
59+
- bmi2: ON
60+
os: macos-latest
61+
5262
# constants
5363
env:
5464
build_dir: "build"
@@ -69,6 +79,7 @@ jobs:
6979
-DQUEST_DISABLE_DEPRECATION_WARNINGS=${{ matrix.version == 3 && 'ON' || 'OFF' }}
7080
-DQUEST_FLOAT_PRECISION=${{ matrix.precision }}
7181
-DQUEST_ENABLE_ADIOS2=ON
82+
-DQUEST_ENABLE_BMI2=${{ matrix.bmi2 }}
7283
7384
# force 'Release' build (needed by MSVC to enable optimisations), and force serial (to avoid ADIOS2 OOM)
7485
- name: Compile

‎CMakeLists.txt‎

Lines changed: 70 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -146,6 +146,15 @@ option(
146146
message(STATUS "NUMA awareness is turned ${QUEST_ENABLE_NUMA}. Set QUEST_ENABLE_NUMA to modify.")
147147

148148

149+
# BMI2
150+
option(
151+
QUEST_ENABLE_BMI2
152+
"Whether QuEST will accelerate CPU bit gather/scatter with x86 BMI2 (PEXT/PDEP) intrinsics."
153+
OFF
154+
)
155+
message(STATUS "BMI2 bitwise acceleration is turned ${QUEST_ENABLE_BMI2}. Set QUEST_ENABLE_BMI2 to modify.")
156+
157+
149158
# Distribution
150159
option(
151160
QUEST_ENABLE_MPI
@@ -297,6 +306,46 @@ if ((NOT (quest_tpb_remainder EQUAL 0)) OR NOT (QUEST_DEFAULT_NUM_GPU_THREADS_PE
297306
endif()
298307

299308

309+
# probe whether BMI2 intrinsics are recognised by compiler
310+
if (QUEST_ENABLE_BMI2)
311+
312+
# save current CMAKE_REQUIRED_FLAGS for later restoration
313+
set(_quest_saved_req_flags "${CMAKE_REQUIRED_FLAGS}")
314+
315+
# give probe compilation the bmi flag if exists (does not exist on MSVC)
316+
include(CheckCXXCompilerFlag)
317+
check_cxx_compiler_flag("-mbmi2" _quest_cxx_recognises_bmi2)
318+
if (_quest_cxx_recognises_bmi2)
319+
string(APPEND CMAKE_REQUIRED_FLAGS " -mbmi2")
320+
endif()
321+
322+
# probe whether intrinsics compile
323+
include(CheckCXXSourceCompiles)
324+
check_cxx_source_compiles(
325+
"
326+
#include <immintrin.h>
327+
int main() {
328+
_pext_u64(0ULL, 0ULL);
329+
_pdep_u64(0ULL, 0ULL);
330+
return 0;
331+
}
332+
"
333+
_quest_cxx_compiles_bmi2)
334+
335+
# restore CMAKE_REQUIRED_FLAS
336+
set(CMAKE_REQUIRED_FLAGS "${_quest_saved_req_flags}")
337+
unset(_quest_saved_req_flags)
338+
339+
# error if probe failed
340+
if (NOT _quest_cxx_compiles_bmi2)
341+
message(FATAL_ERROR "QUEST_ENABLE_BMI2 was ${QUEST_ENABLE_BMI2} but BMI2 intrinsics were not recognised by the compiler. ")
342+
endif()
343+
344+
# var _quest_cxx_recognises_bmi2 used later during BMI2 management
345+
346+
endif()
347+
348+
300349
# warn when numTPB will be later overridden by the current environment variable
301350
if(
302351
DEFINED ENV{QUEST_DEFAULT_NUM_GPU_THREADS_PER_BLOCK}
@@ -342,7 +391,7 @@ if (QUEST_APPEND_CONFIG_TO_LIB_NAME)
342391
string(CONCAT QUEST_OUTPUT_LIB_NAME ${QUEST_OUTPUT_LIB_NAME} "-fp${QUEST_FLOAT_PRECISION}")
343392

344393
if (QUEST_ENABLE_OMP)
345-
string(CONCAT QUEST_OUTPUT_LIB_NAME ${QUEST_OUTPUT_LIB_NAME} "+mt")
394+
string(CONCAT QUEST_OUTPUT_LIB_NAME ${QUEST_OUTPUT_LIB_NAME} "+omp")
346395
endif()
347396

348397
if (QUEST_ENABLE_MPI)
@@ -365,6 +414,18 @@ if (QUEST_APPEND_CONFIG_TO_LIB_NAME)
365414
string(CONCAT QUEST_OUTPUT_LIB_NAME ${QUEST_OUTPUT_LIB_NAME} "+depr")
366415
endif()
367416

417+
if (QUEST_ENABLE_BMI2)
418+
string(CONCAT QUEST_OUTPUT_LIB_NAME ${QUEST_OUTPUT_LIB_NAME} "+bmi2")
419+
endif()
420+
421+
if (QUEST_ENABLE_ADIOS2)
422+
string(CONCAT QUEST_OUTPUT_LIB_NAME ${QUEST_OUTPUT_LIB_NAME} "+adios2")
423+
endif()
424+
425+
if (QUEST_ENABLE_SUBCOMM)
426+
string(CONCAT QUEST_OUTPUT_LIB_NAME ${QUEST_OUTPUT_LIB_NAME} "+subcomm")
427+
endif()
428+
368429
endif()
369430

370431

@@ -556,7 +617,6 @@ if (QUEST_ENABLE_CUQUANTUM)
556617
endif()
557618

558619

559-
560620
# Checkpointing (ADIOS2)
561621
if (QUEST_ENABLE_ADIOS2)
562622

@@ -631,6 +691,13 @@ if (QUEST_ENABLE_ADIOS2)
631691
endif()
632692

633693

694+
# BMI2 (flag not necessary when unrecognised)
695+
if (QUEST_ENABLE_BMI2 AND _quest_cxx_recognises_bmi2)
696+
target_compile_options(QuEST PRIVATE
697+
$<$<COMPILE_LANGUAGE:CXX>:-mbmi2>)
698+
endif()
699+
700+
634701

635702
# ===============================
636703
# Set options to save in config.h
@@ -643,6 +710,7 @@ set(QUEST_COMPILE_MPI ${QUEST_ENABLE_MPI})
643710
set(QUEST_COMPILE_SUBCOMM ${QUEST_ENABLE_SUBCOMM})
644711
set(QUEST_COMPILE_CUQUANTUM ${QUEST_ENABLE_CUQUANTUM})
645712
set(QUEST_COMPILE_ADIOS2 ${QUEST_ENABLE_ADIOS2})
713+
set(QUEST_COMPILE_BMI2 ${QUEST_ENABLE_BMI2})
646714
set(QUEST_INCLUDE_DEPRECATED_FUNCTIONS ${QUEST_ENABLE_DEPRECATED_API})
647715

648716

‎docs/cmake.md‎

Lines changed: 1 addition & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -44,6 +44,7 @@ make
4444
| `QUEST_ENABLE_CUDA` | (`OFF`), `ON` | Determines whether QuEST will be built with support for NVIDIA GPU acceleration. If turned on, `CMAKE_CUDA_ARCHITECTURES` should probably also be set. |
4545
| `QUEST_ENABLE_CUQUANTUM` | (`OFF`), `ON` | Determines whether QuEST will make use of the NVIDIA CuQuantum library. Cannot be turned on if `QUEST_ENABLE_CUDA` is off. |
4646
| `QUEST_ENABLE_HIP` | (`OFF`), `ON` | Determines whether QuEST will be built with support for AMD GPU acceleration. If turned on, `CMAKE_HIP_ARCHITECTURES` should probably also be set. |
47+
| `QUEST_ENABLE_BMI2` | (`OFF`), `ON` | Determines whether QuEST will be built with BMI2 intrinsics to accelerate CPU simulation of few-qubit Quregs. This is not compatible with all compilers and CPUs. **Beware** that if enabled, and the compiled QuEST executable is later run upon a different machine which lacks the BMI2 instructions, execution will crash. |
4748
| `QUEST_ENABLE_ADIOS2` | (`OFF`), `ON` | Determines whether QuEST will be built with ADIOS2 to enable checkpointing, via functions `saveQuregToFile()` and `createQuregFromFile()`. |
4849
| `QUEST_DOWNLOAD_ADIOS2` | (`ON`), `OFF` | Determines whether to download ADIOS2 from Github, when ADIOS2 is enabled but not found. |
4950
| `QUEST_ENABLE_DEPRECATED_API` | (`OFF`), `ON` | Determines whether QuEST will be built with support for the deprecated (v3) API. ***Note**: this will generate compiler warnings and is not supported by MSVC.* |

‎quest/include/config.h.in‎

Lines changed: 4 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -42,6 +42,7 @@
4242
defined(QUEST_COMPILE_HIP) || \
4343
defined(QUEST_COMPILE_CUQUANTUM) || \
4444
defined(QUEST_COMPILE_ADIOS2) || \
45+
defined(QUEST_COMPILE_BMI2) || \
4546
defined(QUEST_ENABLE_NUMA) || \
4647
defined(QUEST_INCLUDE_DEPRECATED_FUNCTIONS) || \
4748
defined(QUEST_DISABLE_DEPRECATION_WARNINGS)
@@ -86,6 +87,7 @@
8687
#cmakedefine01 QUEST_COMPILE_CUQUANTUM
8788
#cmakedefine01 QUEST_COMPILE_HIP
8889
#cmakedefine01 QUEST_COMPILE_ADIOS2
90+
#cmakedefine01 QUEST_COMPILE_BMI2
8991

9092

9193
// crucial to QuEST source (informs optional NUMA usage)
@@ -128,6 +130,7 @@
128130
! defined(QUEST_COMPILE_HIP) || \
129131
! defined(QUEST_COMPILE_CUQUANTUM) || \
130132
! defined(QUEST_COMPILE_ADIOS2) || \
133+
! defined(QUEST_COMPILE_BMI2) || \
131134
! defined(QUEST_ENABLE_NUMA) || \
132135
! defined(QUEST_INCLUDE_DEPRECATED_FUNCTIONS) || \
133136
! defined(QUEST_DISABLE_DEPRECATION_WARNINGS)
@@ -156,6 +159,7 @@
156159
! (QUEST_COMPILE_HIP == 0 || QUEST_COMPILE_HIP == 1) || \
157160
! (QUEST_COMPILE_CUQUANTUM == 0 || QUEST_COMPILE_CUQUANTUM == 1) || \
158161
! (QUEST_COMPILE_ADIOS2 == 0 || QUEST_COMPILE_ADIOS2 == 1) || \
162+
! (QUEST_COMPILE_BMI2 == 0 || QUEST_COMPILE_BMI2 == 1) || \
159163
! (QUEST_ENABLE_NUMA == 0 || QUEST_ENABLE_NUMA == 1) || \
160164
! (QUEST_INCLUDE_DEPRECATED_FUNCTIONS == 0 || QUEST_INCLUDE_DEPRECATED_FUNCTIONS == 1) || \
161165
! (QUEST_DISABLE_DEPRECATION_WARNINGS == 0 || QUEST_DISABLE_DEPRECATION_WARNINGS == 1)

0 commit comments

Comments
 (0)