Any suggestion to start more threads at the same time?

[smith jack]

I start many threads in order to make the work done, when the
concurrent number is set to 300, all thing just works fine, but when
the number is set to 350 or higher, error just comes out? what's wrong
? the error info is just as follows: failed to start .

I am confused, does this have something to do with the operating
system, i am now using Linux, any suggestion to make the system to
support more python threads?

thank you

 

[Irmen de Jong]

I start many threads in order to make the work done, when the
concurrent number is set to 300, all thing just works fine, but when
the number is set to 350 or higher, error just comes out? what's wrong
? the error info is just as follows: failed to start .

I am confused, does this have something to do with the operating
system, i am now using Linux, any suggestion to make the system to
support more python threads?

thank you

I don't think that many threads are going to help you in any meaningful way. Especially
with Python's GIL. Can't you redesign your program to use a fixed number of threads such
as 1 per cpu core?

Or check out the multiprocessing module.

But yeah, I think your OS is preventing you from creating more threads (either due to
some artificial limit or due to lack of system memory).

Irmen