lundi 7 janvier 2019

Why do they prefer dynamic scheduling in this code snippet

I solved an exercise, where I had to implement SGD (stochastic gradient descent) with momentum. The exercise was to parallelize it afterwards.

My suggestion was as follow:

#pragma omp for schedule(static) nowait
for (int i = 0; i < size; i++)
    {
         const double nablaE_w = (1.0/double(B)) * (grad[i] + lambda * param[i]);
         mom_w[i] = beta * mom_w[i] - eta * nablaE_w;
         param[i] = param[i] + mom_w[i];
    }

I use nowait because after the for-loop the calculation is finished.

But on the solution they uses:

#pragma omp for schedule (dynamic, 64/sizeof(double)) nowait

Also after reading several stackoverflow-answers, I still cannot see the advantage of scheduling(dynamic) and why they uses the chunksize=8

I am careful when using schedule(dynamic) - actually I never use it, because I have no idea how to do it correctly.

Aucun commentaire:

Enregistrer un commentaire