macOS build (via Xcode) generates a hang warning when run with the Thread Performance Checker

[jpetrie]

Building an app with Jolt v5.5.0 (from the release tag) and running with the latest Xcode (26.4 - 17E192) with the Thread Performance Checker diagnostic enabled produces a hang warning diagnostic at runtime:

Thread running at User-interactive quality-of-service class waiting on a lower QoS thread running at Default quality-of-service class. Investigate ways to avoid priority inversions

The warning points to the dispatch_semaphore_wait() call inside Semaphore::Acquire() - the call stack is below. I'm updating the physics system on the main thread. Apple's documentation suggests ensuring the Quality-of-Service value for the job threads is the same, although I don't think this is straightforward since Jolt's using std::thread for the worker threads and there is no portable way for it to change that setting. You'd need to get the native_handle and use the pthread API to change the QoS, or something - I'm not that familiar with pthread, honestly. Jolt probably shouldn't assume the thread running the system is running at User-Interactive, either, I'm guessing.

I'm not sure exactly why I just started seeing this, I updated Xcode recently and then updated Jolt to get a version with the is_trivial fix, and I think the new Xcode might default to enabling the thread diagnostic now? It's easy enough to simply toggle the diagnostic off but it would be nice to be able to use it without seeing warnings I have to ignore.

The call stack from the PhysicsSystem site looks like:

JPH::Semaphore::Acquire(unsigned int) at .../Vendor/Source/Jolt/Jolt/Core/Semaphore.cpp:121
JPH::JobSystemWithBarrier::BarrierImpl::Wait() at .../Vendor/Source/Jolt/Jolt/Core/JobSystemWithBarrier.cpp:154
JPH::JobSystemWithBarrier::WaitForJobs(JPH::JobSystem::Barrier*) at .../Vendor/Source/Jolt/Jolt/Core/JobSystemWithBarrier.cpp:227
JPH::PhysicsSystem::Update(float, int, JPH::TempAllocator*, JPH::JobSystem*) at .../Vendor/Source/Jolt/Jolt/Physics/PhysicsSystem.cpp:625

[jrouwe]

Hello,

I'm aware of this warning and have looked at it several times to see if I could find a solution. JobSystemThreadPool spawns N - 1 threads where N is the number of logical threads that your CPU supports. It spawns those threads using std::thread which means that on macOS they get QoS default while the main thread has QoS user interactive. It's not difficult to change the thread priority by adding the following code to JobSystemThreadPool::ThreadMain:

pthread_set_qos_class_self_np(QOS_CLASS_USER_INTERACTIVE, 0);

We can then validate in e.g. JobSystem::Job::Execute that all jobs are executed on threads with the same priority:

qos_class_t qos_class = QOS_CLASS_UNSPECIFIED;
int relative_priority = -999;
pthread_get_qos_class_np(pthread_self(), &qos_class, &relative_priority);
JPH_ASSERT(qos_class == QOS_CLASS_USER_INTERACTIVE);
JPH_ASSERT(relative_priority == 0);

But that doesn't make the warning go away. The only way to get rid of the warning (that I have found) is to stop using dispatch_semaphore_t in the Semaphore class. Comment out the following line:

JoltPhysics/Jolt/Core/Semaphore.h

Line 18 in 3548a30

#define JPH_USE_GRAND_CENTRAL_DISPATCH

and the warning will be gone (regardless of thread priority). The problem with this is that you then fall back to using a mutex and condition variable, which is less efficient than a semaphore (creates more contention between threads as every thread has to lock the mutex before it can start waiting).

I believe that this warning is mostly warning for a theoretical thing that could happen and that it is not a real problem in this case. Jolt doesn't spawn more threads than CPU cores and any thread can work on any job. The callstack that it lists in the warning will indeed wait for a lower priority thread, but as long as your own application doesn't spawn N threads with priority > default and e.g. spin loops those threads forever, the job system threads should complete their work and allow the main thread to continue.

If anyone has any ideas for fixes for this then I'm interested.

[jpetrie]

Thanks for that info. It's strange that setting the QoS doesn't address the warning (I did try it locally, and it changes it slightly - noting that a User Interactive thread is waiting on one without a QoS class specified).

While continuing to poke around, I found a note that "the GCD global queue runs at [the default] level" (table 4-2 here). I would think that only applies to async tasks scheduled on the default queue using dispatch_async_f and such, not to dispatch_semaphore_t usage, but it does look like the semaphore is implemented with a reference to the default queue, if I'm following correctly.

I'll keep exploring.

[jrouwe]

This is an interesting idea and led me to try setting the main thread to QoS default:

pthread_set_qos_class_self_np(QOS_CLASS_DEFAULT, 0);

(e.g. put it in JobSystemThreadPool::Init). This makes the warning go away too! So it appears indeed that there is something associated with the semaphore that has the default QoS.