Openmp and Python

Cython

Cython has OpenMP support: With Cython, OpenMP can be added by using the prange (parallel range) operator and adding the -fopenmp compiler directive to setup.py.

When working in a prange stanza, execution is performed in parallel because we disable the global interpreter lock (GIL) by using the with nogil: to specify the block where the GIL is disabled.

To compile cython_np.pyx we have to modify the setup.py script as shown below. We tell it to inform the C compiler to use -fopenmp as an argument during compilation - to enable OpenMP and to link with the OpenMP libraries.

from distutils.core import setup
from distutils.extension import Extension
from Cython.Distutils import build_ext
setup(
    cmdclass = {"build_ext": build_ext},
    ext_modules = [
        Extension(
            "calculate",
            ["cython_np.pyx"],
            extra_compile_args = ["-fopenmp"],
            extra_link_args = ["-fopenmp"]
        )
    ]
)

With Cython’s prange, we can choose different scheduling approaches. With static, the workload is distributed evenly across the available CPUs. However, as some of your calculation regions are expensive in time, while others are cheap - if we ask Cython to schedule the work chunks equally using static across the CPUs, then the results for some regions will complete faster than others and those threads will then sit idle. Both the dynamic and guided schedule options attempt to mitigate this problem by allocating work in smaller chunks dynamically at runtime so that the CPUs are more evenly distributed when the workload’s calculation time is variable. Thus, for your code, the correct choice will vary depending on the nature of your workload.