Speedup CSV file reference variant tuning is broken

[MrBurmark]

The reference variant tuning used in Executor::writeCSVReport is the same for all kernels, but that variant and tuning may not exist for all kernels. Attempting to run with CUDA or HIP Variants and excluding Lambda variants causes a segfault when attempting to write the speedup csv file.

[rhornung67]

I'm confused. I ran on rzadams yesterday with Seq and Lambda variants excluded for a subset of kernels and it worked fine. Does this issue apply when attempting to run ALL kernels?

[MrBurmark]

In terms of deciding on a reference variant for a kernel, we may want to be more explicit about which is picked. We have a few different kinds of references, references for correctness/checksums, and references for speedup both overall and per backend. We could do something like add an optional argument to addVariantTuning to indicate which variant tuning should be preferred as the checksum and which as the performance reference.

[MrBurmark]

I believe the segfault came from INDEXLIST which does not have tunings for all backends. I must have run just RAJA_CUDA or something like that so the kernel wasn't excluded but there were no tunings that ran.

[smithsg84]

I can confirm that there is a segfault related to INDEXLIST in the case of #639. In the run reported in #639 no variants/tunings existed for INDEXLIST causing a seg fault.

Related to Jason's comments, I don't really understand what the current reference is; the reference_tune_idx value in Executor. Each Kernel object has potentially unique tuning indices based on variants and tunings run for that kernel (tuning name vetor is stored locally and index is based on order of addition):

RAJAPerf/src/common/KernelBase.cpp

Line 165 in bf89bf4

variant_tuning_names[vid].emplace_back(std::move(name));

The reference_tune_idx appears to currently simply be the first variant/tuning added for each Kernel. reference_tune_idx is set to 0 or unknown via:

RAJAPerf/src/common/Executor.cpp

Line 389 in bf89bf4

// This logic seems a bit strange. Not sure why we do it this way.