Cannot Figure out Where to Find the Number of Threads That Python Is Seeing
I Am Running into an Issue That I Do Not Fully Understand. My Crontab Runs Python Jobs on a Schedule as It's Supposed To. Recently They've Started Failing When...
I am running into an issue that I do not fully understand.
My crontab runs python jobs on a schedule as it's supposed to. Recently they've started failing when loading the pandas library with the following error :
OpenBLAS blas_thread_init: pthread_create failed for thread 4 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max
OpenBLAS blas_thread_init: pthread_create failed for thread 5 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max
OpenBLAS blas_thread_init: pthread_create failed for thread 6 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max
OpenBLAS blas_thread_init: pthread_create failed for thread 7 of 8: Resource temporarily unavailable
OpenBLAS blas_thread_init: RLIMIT_NPROC 126940 current, 126940 max
Even though I do not know why so many processes would be created by my jobs, this is still a fairly straightforward error.
So I tried to keep track of the number of open processes to see if there was a specific job triggering the increase. I did that using:
ps -eLo pid,cmd,nlwp | wc -l
which gives me a fairly stable number of around 5k even when the job is failing with the above message (so I was expecting to see ~126k processes).
I am at a loss here : which command do I need to run to be able to keep track of the number of processes that Python sees (i.e ~126940).