[Ann] Checkpointing Ruby applications (DragonFlyBSD only)

[Michael Neumann]

Hi,

With DragonFlyBSD (www.dragonflybsd.org) it's now possible to checkpoint
processes. That means, that you can store a dump of the running
application to disk, from which you can resume later on (possibly on
another machine). I mention this, because it's especially interesting
for Web frameworks like Wee or Borges, which use Continuations and
Threads, that are not marshallable (but checkpointable ;-).

I've made a Ruby module around the sys_checkpoint syscall. You can find
the module here:

http://www.ntecs.de/viewcvs/viewcvs/DragonFly/checkpoint/

Of course, you have to restore sockets, pipes etc. on resume yourself.

Regards,

Michael

 

[Michael Neumann]

Hi,

With DragonFlyBSD (www.dragonflybsd.org) it's now possible to checkpoint
processes. That means, that you can store a dump of the running
application to disk, from which you can resume later on (possibly on
another machine). I mention this, because it's especially interesting
for Web frameworks like Wee or Borges, which use Continuations and
Threads, that are not marshallable (but checkpointable ;-).

I've made a Ruby module around the sys_checkpoint syscall. You can find
the module here:

http://www.ntecs.de/viewcvs/viewcvs/DragonFly/checkpoint/

Of course, you have to restore sockets, pipes etc. on resume yourself.

Maybe I'll give a simple example:

require 'checkpoint'
Checkpoint.checkpoint('dump')
p '.'
Checkpoint.resume('dump')

This will loop infinite and print '.' on the screen.
Or this one:

$conn = TCPServer....

Checkpoint.checkpoint('dump') do
# setup sockets on resume
$conn = TCPServer.open('localhost', 9999)
end

....

If you resume from "dump" (type 'checkpt -r dump' on command line, or
'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block will be
executed.

Regards,

Michael

 

[David Ross]

Hi,

With DragonFlyBSD (www.dragonflybsd.org) it's now possible to
checkpoint processes. That means, that you can store a dump of the
running application to disk, from which you can resume later on
(possibly on another machine). I mention this, because it's especially
interesting for Web frameworks like Wee or Borges, which use
Continuations and Threads, that are not marshallable (but
checkpointable ;-).

I've made a Ruby module around the sys_checkpoint syscall. You can
find the module here:

http://www.ntecs.de/viewcvs/viewcvs/DragonFly/checkpoint/

Of course, you have to restore sockets, pipes etc. on resume yourself.

Regards,

Michael

I'm glad there are others who appreciate BSDs. What made you interested?

David Ross

 

[Lothar Scholz]

Hello Michael,

MN> Checkpoint.checkpoint('dump') do
MN> # setup sockets on resume
MN> $conn = TCPServer.open('localhost', 9999)
MN> end

MN> If you resume from "dump" (type 'checkpt -r dump' on command line, or
MN> 'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block will be
MN> executed.

Did you any measurements how much time is saved for startup of a
complexer ruby program with lots of requires ? Something like
FreeRide would be nice. I don't have BSD installed so i can't do it myself.

Also why isn't it possible to directly run the checkpoint file from
the shell ?

 

[Michael Neumann]

Hello Michael,

MN> Checkpoint.checkpoint('dump') do
MN> # setup sockets on resume
MN> $conn = TCPServer.open('localhost', 9999)
MN> end

MN> If you resume from "dump" (type 'checkpt -r dump' on command line, or
MN> 'ruby -r checkpoint -e 'Checkpoint.resume("dump")'), the block will be
MN> executed.

Did you any measurements how much time is saved for startup of a
complexer ruby program with lots of requires ? Something like
FreeRide would be nice. I don't have BSD installed so i can't do it myself.

Not yet. I never had timing problems with lots of requires...
If I can compile freeride (FX...) easily I'll report my results.

Also why isn't it possible to directly run the checkpoint file from
the shell ?

It is possible:

checkpt -r checkpoint-file

Well, it's in reality a core-dump (or so I'm told), and it has to be
sent the CKPTEXIT signal, so that it can perform some setup-tasks... and
it will return from the sys_checkpoint syscall with a given value. Not
sure how this all could be done easily with a directly executable file,
without modifying the core-dump to much.

Regards,

Michael

 

[Michael Neumann]

I'm glad there are others who appreciate BSDs. What made you interested?

In BSD in general or in DragonFly? Well, with DragonFly it's a bit like
with Ruby. You can get easily responses from it's creator (indeed, it's
easier to get a response from Matt than from Matz ;-), and problems are
solved mostly very quickly. You know where to asks, community is small
and friendly.

But most importantly, I dropped FreeBSD 5.3, as it paniced on Monday 3
times in a row (maybe it's not FreeBSD's fault, but *I* encountered a
lot of crashes in the 5.x line). Now, I'll count every crashes of
DragonFly. Zero up to now ;-)

And of course the ideas behind DragonFly are IMHO superior. I am happy
each time when Matt has removed a lot of code (rewrote it) or made the
API much simpler.

Regards,

Michael

 

[Michael Neumann]

Not yet. I never had timing problems with lots of requires...
If I can compile freeride (FX...) easily I'll report my results.

It's not possible to restore freeride (or any GUI programm, I guess), as
X11 is involved... Sockets!

Regards,

Michael

 

[Lothar Scholz]

Hello Michael,


MN> It's not possible to restore freeride (or any GUI programm, I guess), as
MN> X11 is involved... Sockets!

Right, but you can set a checkpoint before connecting to X11.

Could be nice for large, commercial software. Startup time when you
require hundred thousand lines ruby code is not really funny.

Lisp/Smalltalk users know what i'm talking about.