Flash Movie Conversion with Java (FFMPEG)

[Vince]

Hi all

I've been developing a customer application for quite a while now and
we're reaching the point where the application will be tested. Anyway,
one part of the application allows the user to upload movies in
different formats (mpg, avi etc) which will be converted automatically
into the FLV format (plus a preview picture of it will be created).

I was researching for a while to find an adequate Java API/Tool for this
purpose and bumped into different available frameworks like at example
JMF. After further investigations regarding JMF and others, I realized
quickly that I would break a fly on the wheel and I decided to switch
over to FFMPEG (http://en.wikipedia.org/wiki/FFmpeg) even though it's
not a native but Java external tool.

I basically wrote a wrapper class to parameterize and execute FFMPEG using:

....
int exitValue = -1;
Process process = Runtime.getRuntime().exec(convertCommand);
exitValue = doWaitFor(process);
....

I had to use a sort of hack (doWaitFor(process)) to check when the
process ends since FFMPEG doesn't return and exit values and since there
is a documented problem on Win32 platforms with the native
process.waitFor() function. For those who are interested I'll append the
doWaitFor method at the bottom of this post.

Anyway, coming closer to the application integration testing, I start
worrying a bit regarding the performance of that whole conversion
method. Since the hardware is running on multi processor and has enough
RAM I'm not really concerned about that, I'm more concerned regarding
the application performance when let's say 100 user do upload and
convert movies at the same time. So my questions at this point are:

1. Did everyone ever had to deal with Flash Movie Conversion, and if yes
which approach did you take

2. How will my way of conversion affect the overall application performance

Obviously there are some more questions but I guess I'll post them bit
by bit in accordance with your replies...

Thanks for any contribution!

Vince

PS: Following my doWaitFor Method:

private static int doWaitFor(Process p) {

int exitValue = -1; // returned to caller when p is finished

try {
InputStream in = p.getInputStream();
InputStream err = p.getErrorStream();
boolean finished = false; // Set to true when p is finished

while (!finished) {

try {
while (in.available() > 0) {
// Print the output of our system call
Character c = new Character((char) in.read());
System.out.print(c);
}

while (err.available() > 0) {
// Print the output of our system call
Character c = new Character((char) err.read());
System.out.print(c);
}


// Ask the process for its exitValue. If the process
// is not finished, an IllegalThreadStateException
// is thrown. If it is finished, we fall through and
// the variable finished is set to true.

exitValue = p.exitValue();
finished = true;

} catch (IllegalThreadStateException e) {
// Process is not finished yet;
// Sleep a little to save on CPU cycles
Thread.currentThread().sleep(500);

}
}

} catch (Exception e) {
// unexpected exception! print it out for debugging...
System.err.println("doWaitFor(): unexpected exception - "
+ e.getMessage());
}
// return completion status to caller
return exitValue;
}

 

[Vince]

Seems like no one's got experience with things like that huh!? :-(

 

[Mark Rafn]

Anyway, one part of the application allows the user to upload movies in
different formats (mpg, avi etc) which will be converted automatically
into the FLV format (plus a preview picture of it will be created).

I was researching for a while to find an adequate Java API/Tool ....
over to FFMPEG (http://en.wikipedia.org/wiki/FFmpeg) even though it's
not a native but Java external tool.

This is probably wise. There are FAR more formats supported by open-source
C apps than by any Java libraries.

I basically wrote a wrapper class to parameterize and execute FFMPEG using:
...
int exitValue = -1;
Process process = Runtime.getRuntime().exec(convertCommand);
exitValue = doWaitFor(process);

For more control of environment and commandline, you might need
ProcessBuilder rather than Runtime.exec(). In any case, you will likely want
to make sure you're reading the input and error streams of the Process.

I had to use a sort of hack (doWaitFor(process)) to check when the
process ends since FFMPEG doesn't return and exit values and since there
is a documented problem on Win32 platforms with the native
process.waitFor() function.

Really? Would you provide a link to the bug report? As far as I know, this
works correctly on all platforms, as long as you're reading stdout and stderr.
Since your main thread is waiting, you should read the outputs on other
threads.

Anyway, coming closer to the application integration testing, I start
worrying a bit regarding the performance of that whole conversion
method. Since the hardware is running on multi processor and has enough
RAM I'm not really concerned about that, I'm more concerned regarding
the application performance when let's say 100 user do upload and
convert movies at the same time.

Upload and convert can happen at different times, right? Allow upload to
some spool directory, and have a separate process (possibly in the same VM,
probably not) do the conversion using a pool of converters so that only N
are converting at a time.

 

[Roedy Green]

2. How will my way of conversion affect the overall application performance

This is a highly cpu-intensive process. What you might do is lower
the priority of the process so you will only soak up spare cycles and
without hurting response.

If you have 2 cpus, there is no point in running more than 2 processes
at once. Every extra one will hurt by consuming RAM.

So I suggest you invent a queue system with two server processes.

See http://mindprod.com/jgloss/queue.html

 

[Roedy Green]

So I suggest you invent a queue system with two server processes.

The other thing you might do is use a priority queue. Short jobs get
higher priority. Perhaps priority gets bumped the longer they wait.

 

[Vince]

For more control of environment and commandline, you might need
ProcessBuilder rather than Runtime.exec(). In any case, you will likely want
to make sure you're reading the input and error streams of the Process.

What would the advantage be to use ProcessBuilder instead of Runtime?
Anyway the doWaitFor method (see the appended code in my original post)
does constantly read - with a sleeper of 500ms - the input and error
streams of the process...

Really? Would you provide a link to the bug report?
http://www.google.ch/search?hl=de&q=java+waitfor+hangs&meta=

http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=1

Upload and convert can happen at different times, right?

If I do understand your question right then no or basically almost no.
Let me explain, the file gets uploaded and written to the HD, then
straight away FFMPEG gets launched. As soon as the process stops, the
original movie gets deleted from the HD and only the converted one remains.

Allow upload to
some spool directory, and have a separate process (possibly in the same VM,
probably not) do the conversion using a pool of converters so that only N
are converting at a time.

May you elaborate a bit more on this one?

Thank a lot

Vince

 

[Vince]

If you have 2 cpus, there is no point in running more than 2 processes
at once. Every extra one will hurt by consuming RAM.

I don't get your point about running more than 2 processes at once.
Basically one movie conversion is equal to one process. But if now 10
user decide to upload and thus convert their movie almost at the same
time, I'll get as much processes as requests... ???

Thanks a lot

Vince

 

[Mark Rafn]

Process process = Runtime.getRuntime().exec(convertCommand);

What would the advantage be to use ProcessBuilder instead of Runtime?

You get more control over what's actually run - you can give a List<String> or
String[] of params rather than a single String, so you can avoid extra
quoting if the parameters have spaces or special characters in them, you
can set the working directory, and you can control the environment. You
can also set up one ProcessBuilder, and start() it multiple times, if that
fits your use case.

In your simple case, you can also redirectErrorStream(true) before starting
the process, and then you won't have to read the error stream from the
Process.

Anyway the doWaitFor method (see the appended code in my original post)
does constantly read - with a sleeper of 500ms - the input and error
streams of the process...

Right, but polling for output is pretty gross. It's simpler to spin off a
thread to read the input. Hacky sample that should be expanded:

// WARNING: partial and not tested, so errors are likely
builder.setRedirectErrorStream(true);
Process p = builder.start();

final InputStream procStdOut = p.getInputStream();
// start a thread to read the stdout
new Thread("process InputStream handler for " + processName) {
public void run() {
byte[512] buf; // deal with larger blocks than 1 char at a time!
int amtRead;
try {
// read until EOF, blocking when there's nothing to read
while ((amtRead = procStdOut.read(buf) != -1) {
// do something with amtRead bytes of buf
System.out.write(buf, 0, amtRead);
}
} catch (IOException e) {
// shouldn't ever happen for this type of stream
// note failure, queue to retry or whatever you want.
LOGGER.error("impossible IOException!", e);
}
}
}.run();
int exitCode = p.waitFor();

http://www.google.ch/search?hl=de&q=java+waitfor+hangs&meta=
http://www.javaworld.com/javaworld/jw-12-2000/jw-1229-traps.html?page=1

These aren't bugs or issues with Process.waitFor(). And they're not win32
specific - it applies to almost all systems.

JavaDoc for java.lang.Process even says "Because some native platforms only
provide limited buffer size for standard input and output streams, failure to
promptly write the input stream or read the output stream of the subprocess
may cause the subprocess to block, and even deadlock."

If I do understand your question right then no or basically almost no.

Oh. Why not? Video conversion is a slow, expensive process, and you
shouldn't try to do it all at once.

Let me explain, the file gets uploaded and written to the HD, then
straight away FFMPEG gets launched. As soon as the process stops, the
original movie gets deleted from the HD and only the converted one remains.

I'd call that a broken design. It's just plain not going to scale well.

May you elaborate a bit more on this one?

Sure. Don't process the file right away. Save it to some directory (or DB
blob or whatnot) for incoming files, and have a separate system that
converts the format and deletes the original. That separate system can work
on N files at once, so you can optimize the memory, cpu, and disk usage for
large numbers of requests.

It does mean that you have to show your users a slightly different UI, as they
can now have files queued for processing and files ready for use.

 

[Vince]

Mark first I wanna thank you for your non negligible efforts and time
you did put with your replies...

...rather than a single String, so you can avoid extra
quoting if the parameters have spaces or special characters in them ...

I recall now having used the ProcessBuilder when I was struggling with
the waitFor at one stage but must have come back to the Runtime version
because it didn't solve my problem. With your approach below I gotta
reconsider my code...

In your simple case, you can also redirectErrorStream(true) before starting
the process, and then you won't have to read the error stream from the
Process.

I do not understand this point yet but I'll read about it and get clued up..

Right, but polling for output is pretty gross.

I thought it wasn't very nice to solve it this way, but good you hammer
the nail further in ;-) As said above I'll reconsider this whole part..

// WARNING: partial and not tested, so errors are likely
builder.setRedirectErrorStream(true);
Process p = builder.start();

I guess at this stage that builder is from type ProcessBuilder...

final InputStream procStdOut = p.getInputStream();
// start a thread to read the stdout
new Thread("process InputStream handler for " + processName) {
public void run() {
byte[512] buf; // deal with larger blocks than 1 char at a time!
int amtRead;
try {
// read until EOF, blocking when there's nothing to read
while ((amtRead = procStdOut.read(buf) != -1) {
// do something with amtRead bytes of buf
System.out.write(buf, 0, amtRead);
}
} catch (IOException e) {
// shouldn't ever happen for this type of stream
// note failure, queue to retry or whatever you want.
LOGGER.error("impossible IOException!", e);
}
}
}.run();
int exitCode = p.waitFor();

I'll test it and give you a feedback. By just reading the code though,
it all makes sense to me and looks pretty complete..

Oh. Why not? Video conversion is a slow, expensive process, and you
shouldn't try to do it all at once.
....

I'd call that a broken design. It's just plain not going to scale well.

I'm glad to hear that from you. We had long discussions with the
customer and he refuses to queue and thus delay the conversion. The
application is done for regional / International journalists which need
to redact their articles online and thus also need instant access to
their uploaded medias. There is no human process in place between the
time where the journalist releases his article and it gets visible on
the portal. Basically if the journalist prepares his article in word,
quickly copies the content into the portal and pushes the release
button, the article is visible for the public. If the media content
appears now only 10 minutes later because it has been queued for
conversion... no comment...

... That separate system can work
on N files at once, so you can optimize the memory, cpu, and disk usage for
large numbers of requests.

That's exactly why I'm concerned about the testing and performance... So
basically what can I do?

Thanks Vince

 

[Mark Rafn]

I do not understand this point yet but I'll read about it and get clued up..

Every process has output and error output. Normally you have to read both.
If you setRedirectErrorStream(true) before starting it, all the error output
will go to standard output, and you can ignore the error output stream.

I'm glad to hear that from you. We had long discussions with the
customer and he refuses to queue and thus delay the conversion.

So if 100 people all upload at the same time, the client would rather delay
them ALL becase they're sharing resources badly rather than processing some
quickly and some more slowly (but not more slowly than all at once).

A bit of reading on queueing theory should convince you that it's pretty much
ALWAYS better to queue than to overextend your resources. Once you're
convinced, you can decide how to convince the client. Or just let him have a
crappy broken app that tries to run hundreds of video transcodes at once if
that's what he demands.

 

[Vince]

A bit of reading on queueing theory should convince you that it's pretty much
ALWAYS better to queue than to overextend your resources. Once you're
convinced, you can decide how to convince the client.

Believe you me I'm convinced since a long time and also tried hardly to
convince the customer but he's as hard as rock. As you said we decided
to let him run into the wall with a conversion process that will become
real slow when multiple user upload their medias. We hope that it might
change his mind at one later stage...

I'm busy writing the test cases but I'll try to implement your code
ASAP... Keep you posted about the result ;-)

Vince

 

[Lew]

Believe you me I'm convinced since a long time and also tried hardly to

"Hardly" means "barely at all", so this sentence parses as you "tried barely
at all", which I suspect is the opposite of the intended meaning.

convince the customer but he's as hard as rock. As you said we decided
to let him run into the wall with a conversion process that will become
real slow when multiple user upload their medias. We hope that it might
change his mind at one later stage...

Nah, they'll just blame you.

I'm being cynical because this has happened to me - I warned a customer that
their decision would make the system unacceptably slow, they harshly overrode
my recommendation, then tried to withhold my fee because the system was
unacceptably slow.

 

[Vince]

"Hardly" means "barely at all", so this sentence parses as you "tried
barely at all", which I suspect is the opposite of the intended meaning.

Thanks pointing on this one, you're right, I meant I tried hard ;-)

Nah, they'll just blame you.

Obviously, I'm not excepting anything else.. Also a reason why I posted
here..

then tried to withhold my fee
because the system was unacceptably slow.

He can try, the platform is running in our data center, I'll power off
his server before he pulls the gun ;-)