Cannot Figure out Where to Find the Number of Threads That Python Is Seeing

I am running into an issue that I do not fully understand.

My crontab runs python jobs on a schedule as it's supposed to. Recently they've started failing when loading the pandas library with the following error :

OpenBLAS blas_thread_init: pthread_create failed for thread 4 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max
OpenBLAS blas_thread_init: pthread_create failed for thread 5 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max
OpenBLAS blas_thread_init: pthread_create failed for thread 6 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max
OpenBLAS blas_thread_init: pthread_create failed for thread 7 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max

Even though I do not know why so many processes would be created by my jobs, this is still a fairly straightforward error.

So I tried to keep track of the number of open processes to see if there was a specific job triggering the increase. I did that using:

ps -eLo pid,cmd,nlwp | wc -l

which gives me a fairly stable number of around 5k even when the job is failing with the above message (so I was expecting to see ~126k processes).

I am at a loss here : which command do I need to run to be able to keep track of the number of processes that Python sees (i.e ~126940).

2
Alexander Ross

Alexander Ross

Gaming, Esports & Interactive Media Writer

Alexander Ross has covered the video game industry for a decade, writing deep dives on game design, esports tournaments, VR developments, and gaming culture.

Share this article