queue runtime processes how?

[Wernt Me]

I need to run an arbitrary number of external executables, with as many as
possible running concurrently.

I have an exaggerated example. Let's say I have a billion processes to create
from arbitrary executables. I want to run the first n that will be able to
run without exhausting storage, then the next n, etc. Each process requires 2
threads one for the process and it's output and the other for it's error
stream.

How can I determine how many processes I should try to create at a time?

 

[Thomas Weidenfeller]

I want to run the first n that will be able to
run without exhausting storage, then the next n, etc.

This is in general impossible on a garden-variety multi-tasking
operating system. You can't predict how much system memory is available
at a particular time. The millisecond after you get some number, some
other process outside of your control might start and take away memory.

If you want to do things like this - you unfortunately didn't tell us
why you want to do this -, you need an operating system which provides
features for resource reservation in advance. I only know proprietary
OS' for telecom systems who do this.

How can I determine how many processes I should try to create at a time?

You can't in a reliable way.

/Thomas

 

[Wernt Me]

This is in general impossible on a garden-variety multi-tasking
operating system.

I'd be satisfied with determining how many to run at a time, without being
able to predict it.

My first thought is that I can catch out of memory exceptions, but I'm not
sure if it's a good idea to use all of the memory. Maybe I should choose some
fixed number that people can configure.

Essentially, I've been asked to invoke an arbitrary number of external
programs concurrently, for a general tool. My guess is that the number of
programs can easily be more than 100 programs.

Is there a way to be a polite program?