gemm execution time is unstable in multi process system

Hello,
We are using NVIDIA Jetson Orin platform and in a multi-process system, each process is assigned to a specific cpu and parallelizes cblas_sgem() sequentially through the remaining cores.
When the calculation is performed with only one process created, the execution time of cblas_sgem() is almost constant, but when the process increases, the calculation time becomes unstable.
Although cblas_sgem() using only one thread took about 25 ms, parallel processing with 6 threads each in 6 processes causes instability in running time from 10 ms to 60 ms.

A detailed description of the situation
We have 11 cpu cores from no. 1 to no. 11.
Create six processes, allocate them from cpu 1 to cpu 6, and synchronize the operation start time and execute them simultaneously.
Locked through Semaphore in the middle and sequentially parallelized cblas_sgem() with a total of 6 threads using 5 remaining cores from cpu 7 to cpu 11.
In this case, cblas_sgem() executed in the process allocated to cores 1 to 5 is about 11 ms, and the execution time is constant.
However, in the last process assigned to core 6, cblas_sgem() lags unsteadily from 30 ms to 60 ms, slower than when running with one thread.
The same code is executed for cores 1 to 6.

The code currently in use is as follows.

openblas_thread = 6;
openblas_set_num_threads (openblas_thread); // Set the number of openblas threads to 6

CPU_ZERO(&cpuset);
CPU_SET(sched_getcpu(), &cpuset);
pthread_setaffinity_np (pthread_self(), sizeof(cpuset), &cpuset); // reallocate itself to the original assigned CPU number (1 to 6, respectively)

for (int k = 0; k < openblas_thread-1; k++) { // allocate openblas_thread to CPU cores (7 to 11), excluding the main process(thread)
CPU_ZERO(&cpuset);
CPU_SET(11 - k, &cpuset);
openblas_setaffinity(k, sizeof(cpuset), &cpuset);
}

for(int i = 0; i < 30; i++) {
gemm_wrapper_func();
}

When I measured the execution time inside the openblas code, it was confirmed that the calculation time became unstable when processing cblas_sgem() called in the wrapper function.
Is the code above correct to use openblas normally?

Start with the reported cblas_sgem() entry point and the openblas_set_num_threads() and openblas_setaffinity() calls in the example. Reproduce the six-process, six-thread workload on the NVIDIA Jetson Orin setup, compare the timing variation with the affinity configuration, and document whether the usage is supported and what explains the instability.