Saving the state of a Thread?

P

Patrick Li

Hi,
Does anyone have any clever ideas on saving the state of a Thread?
I have a thread that runs for a couple of hours, and I need to save the
progress to disk, so that I can restart from where I left off.

Any ideas would be welcome.
-Patrick
 
I

Iñaki Baz Castillo

El Viernes, 22 de Agosto de 2008, Patrick Li escribi=C3=B3:
Hi,
Does anyone have any clever ideas on saving the state of a Thread?
I have a thread that runs for a couple of hours, and I need to save the
progress to disk, so that I can restart from where I left off.

Not sure about what you mean. Maybe it would be useful for you the usage of=
=20
variables in threads, for example:

this_thread =3D Thread.current
this_thread[:time_on] =3D Time.now

This stores the actual time into a variable for that thread, so you later c=
an=20
read it by doing:

this_thread[:time_on]

But not sure if you want this.



=2D-=20
I=C3=B1aki Baz Castillo
 
P

Patrick Li

Mmm, that's not quite it.

Here's an example:

eg.
def start
initialize()
doSomething()
doSomethingElse() <-- Three hours into processing
doMore()
cleanup()
end

so imagine this method is called inside a Thread. And three hours into
processing, the user wants to take a break and save his progress to
disk.

Next time the program starts up, I want to be able to jump directly to
the
doSomethingElse() line. With the appropriate local variables all set and
ready.

Is this possible?
-Patrick
 
A

ara.t.howard

Mmm, that's not quite it.

Here's an example:

eg.
def start
initialize()
doSomething()
doSomethingElse() <-- Three hours into processing
doMore()
cleanup()
end

so imagine this method is called inside a Thread. And three hours into
processing, the user wants to take a break and save his progress to
disk.

Next time the program starts up, I want to be able to jump directly to
the
doSomethingElse() line. With the appropriate local variables all set
and
ready.

Is this possible?
-Patrick
--


sure. do it by hand using amalgalite.


a @ http://codeforpeople.com/
 
S

shadowfirebird

Patrick said:
Hi,
Does anyone have any clever ideas on saving the state of a Thread?
I have a thread that runs for a couple of hours, and I need to save the
progress to disk, so that I can restart from where I left off.

Any ideas would be welcome.
-Patrick

Maybe you should think in terms of saving the state of the objects in a
thread, instead. Thought of that way, it becomes pretty easy -- you could use
yaml, for example.

Or have I got the wrong end of the stick?
 
I

Iñaki Baz Castillo

El Viernes, 22 de Agosto de 2008, Patrick Li escribi=C3=B3:
Mmm, that's not quite it.

Here's an example:

eg.
def start
initialize()
doSomething()
doSomethingElse() <-- Three hours into processing
doMore()
cleanup()
end

so imagine this method is called inside a Thread. And three hours into
processing, the user wants to take a break and save his progress to
disk.

Next time the program starts up, I want to be able to jump directly to
the
doSomethingElse() line. With the appropriate local variables all set and
ready.

Is this possible?

Using thread variables you could do:

def initialize
@thread =3D Thread.current
end

def doSomething()
@thread[:state] =3D 'doSomething'
end

def doSomethingElse()
@thread[:state] =3D 'doSomethingElse'
end

def start
initialize()
doSomething()
doSomethingElse() <-- Three hours into processing
doMore()
cleanup()
end



=2D-=20
I=C3=B1aki Baz Castillo
 
P

Patrick Li

Thanks for the replies.

Alright so I have some thoughts now:

def start
status = 0
doSomething()
status = 1
doSomethingElse()
status = 2
doMore()
status = 3
cleanup()
end

So if I know the status of a thread, (ie. status = 2), and I know the
values of the local variables.
Is there way I can "jump" to that line in the method?

Amalgalite: I'm not sure if I got your point. It seems to be a SQLite
library for Ruby?

@shadowfirebird: Yes, that's the most straightforward way of doing it.
But I'm trying to write a framework that will relieve programmers of
having to save state information manually.

ie. I'm trying to convert this:

stateVar = 124
def update
if(stateVar == 0) then doStuff(); stateVar = 1
if(stateVar == 1) then doMoreStuff(); stateVar = 2
if(stateVar == 2) then cleanup(); stateVar = 3
end

Into This:

start
doStuff()
doMoreStuff()
doEvenMore()
cleanup()
endProgram()
 
R

Robert Klemme

So if I know the status of a thread, (ie. status = 2), and I know the
values of the local variables.
Is there way I can "jump" to that line in the method?

I would not speak of the status of a thread. Instead I would model this
as status of calculation using command pattern with multiple operation
classes. Rough idea: break down the computation into multiple stages.
Each stage is handled by an instance of a single class. Every stage
knows its successor stages. When a stage is completed a central
coordinator saves the next state processor (or just the class) along
with its input via Marshalling into a file. Something along the lines of

#!/bin/env ruby

require 'fileutils'

class Coordinator
def initialize state_file, processor = nil
@state_file = state_file

if File.readable? state_file
@current_processor = load
else
@current_processor = processor
save processor
end
end

def run
loop do
next_stage = @current_processor.call

if next_stage
save next_stage
@current_processor = next_stage
else
# done, return last processor with result
clear
return @current_processor
end
end
end

private
def save st
printf "saving %p\n", st
File.open(@state_file + ".tmp", "wb") {|io| Marshal.dump(st,io)}
FileUtils.mv @state_file + ".tmp", @state_file
end

def load
File.open(@state_file, "rb") {|io| Marshal.load(io)}
end

def clear
FileUtils.rm_f @state_file
end
end

# simple exsample
Processor = Struct.new :start, :last, :sum do
def call
limit = [start + 100, last].min
self.sum ||= 0

for i in start .. limit
self.sum += i
end

return nil if limit == last

res = self.class.new
res.sum = self.sum
res.start = limit
res.last = self.last
res
end
end

# run
pr = Processor.new 1, 1_000_000, 0
c = Coordinator.new "x.bin", pr
puts c.run.sum


Of course I could have created two processor types, one for the initial
phase (i.e. where start + 10 < end) and one for the last chunk - but I
was lazy. At least the program works - you can break it at any step and
it will resume processing.

Kind regards

robert
 
P

Patrick Li

Thanks Robert for the input.

Yes I agree. That's a very clean and flexible approach. My code is
currently structured using state machines, which is similar in concept
to your operation stages.

This sort of programming is actually one of my primary motivators for
thinking about a thread-based paradigm instead.

Here's my original idea, in it's original un-condensed and unabbreviated
mental form. Maybe you have some ingenious ideas:

Currently, a real-time simulation loop looks something like this:

while true
scenario.update (secondsSinceLastIteration)
scenario.draw
end

And your scenario code would like something like this (ie. for a
traveling ball):

def update (timePassed)
@position += timePassed*velocity
end

You would agree that this is all very straightforward. But what if you
want the ball to travel back and forth 3 times? You'd have to do
something like this:

def update (timePassed)
if @numberOfBackAndForths == 3
return

if @travelingForward
@position += timePassed*velocity
@travelingForward = false if (reached end)
else
@position -= timePassed*velocity
if reached-end
@travelingForward = false
@numberOfBackAndForths += 1
end
end
end

So can see that @travelingForward essentially describes the state of the
system. This sort of manual state bookkeeping can get really out-of-hand
pretty quickly.

BASIC IDEA: Wouldn't it be nice to be able to write something like this?

def start
3.times do
#TRAVEL FORWARD
update(5 seconds) do |timePassed|
@position += timePassed*velocity
end

#TRAVEL BACKWARD
update(5 seconds) do |timePassed|
@position -= timePassed*velocity
end
end
end

Ahh... so much cleaner isn't it? This is a shift to a thread-based
paradigm.

So far, I have everything working, EXCEPT when the simulation needs to
be saved to disk.

I can save @position, and I can save how many iterations are left. But
after I resume the simulation, how do I get the thread to start from the
appropriate line of code? I never thought I'd ask for it, but a goto
statement sounds like a blessing right now.

Thanks for all your replies.
-Patrick
 
R

Ron Fox

Don't forget the state of a thread may also include the set of open
file handles. What you are trying to do is called 'checkpointing'.
See e.g. http://en.wikipedia.org/wiki/Application_checkpointing

in the general case, this is _hard_, though there are some packages
that support this at the application level.

A specific, application aware scheme can be simpler, because you
presumably know what state must be saved while the general case has to
assume that all things that make up state must be saved.

Ron.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,773
Messages
2,569,594
Members
45,123
Latest member
Layne6498
Top