How to modify a program while it's running?

[Joe Strout]

Here's my situation: I'm making an AIM bot, but the AIM server will
get annoyed if you log in too frequently (and then lock you out for a
while). So my usual build-a-little, test-a-little methodology doesn't
work too well.

So I'd like to restructure my app so that it can stay running and stay
logged in, yet I can still update and reload at least most of the
code. But I'm not sure what's the best way to do this. Should I move
the reloadable code into its own module, and then when I give my bot a
"reload" command, have it call reload on that module? Will that work,
and is there a better way?

Thanks,
- Joe

 

[Craig Allen]

Here's my situation: I'm making an AIM bot, but the AIM server will
get annoyed if you log in too frequently (and then lock you out for a
while). So my usual build-a-little, test-a-little methodology doesn't
work too well.

So I'd like to restructure my app so that it can stay running and stay
logged in, yet I can still update and reload at least most of the
code. But I'm not sure what's the best way to do this. Should I move
the reloadable code into its own module, and then when I give my bot a
"reload" command, have it call reload on that module? Will that work,
and is there a better way?

Thanks,
- Joe

yes you want reload. Design the high level part that knows how to log
in to be able to reload the stuff that changes. My guess is that is
the best way, though I wouldn't be surprised if there are other
solutions.

 

[Arnaud Delobelle]

Here's my situation: I'm making an AIM bot, but the AIM server will
get annoyed if you log in too frequently (and then lock you out for a
while). So my usual build-a-little, test-a-little methodology doesn't
work too well.

So I'd like to restructure my app so that it can stay running and stay
logged in, yet I can still update and reload at least most of the
code. But I'm not sure what's the best way to do this. Should I move
the reloadable code into its own module, and then when I give my bot a
"reload" command, have it call reload on that module? Will that work,
and is there a better way?

Thanks,
- Joe

You could have two processes: the first one that logs in and does the message
passing with the server, the other that implements the logic and talks
to the server via the first one. This way you can restart the second
one as often as you want without being logged out.

I never user reload() but I hear that it has quite a few caveats.

 

[Ivan Illarionov]

Here's my situation: I'm making an AIM bot, but the AIM server will  
get annoyed if you log in too frequently (and then lock you out for a  
while).  So my usual build-a-little, test-a-little methodology doesn't  
work too well.

So I'd like to restructure my app so that it can stay running and stay  
logged in, yet I can still update and reload at least most of the  
code.  But I'm not sure what's the best way to do this.  Should I move  
the reloadable code into its own module, and then when I give my bot a  
"reload" command, have it call reload on that module?  Will that work,  
and is there a better way?

Thanks,
- Joe

Take a look at:
http://code.djangoproject.com/browser/django/trunk/django/utils/autoreload.py

 

[Steven D'Aprano]

Here's my situation: I'm making an AIM bot, but the AIM server will get
annoyed if you log in too frequently (and then lock you out for a
while). So my usual build-a-little, test-a-little methodology doesn't
work too well.

Ouch! What AIM server are you running? Perhaps you need a less
curmudgeonly one?

So I'd like to restructure my app so that it can stay running and stay
logged in, yet I can still update and reload at least most of the code.
But I'm not sure what's the best way to do this. Should I move the
reloadable code into its own module, and then when I give my bot a
"reload" command, have it call reload on that module? Will that work,
and is there a better way?

That should work for functions, but less successfully with classes. The
problem is that existing objects will still have the old behaviour even
after reloading the class.

In the interactive interpreter, the easiest way around that is to just
throw the instance away and recreate it. That's not very convenient. You
may be able to work around that by keeping a list of instances, then
going through each one and saying:

instance.__class__ = mymodule.MyClass

although I haven't tried this and don't know if it even works.

 

[James Mills]

That should work for functions, but less successfully with classes. The
problem is that existing objects will still have the old behaviour even
after reloading the class.

You need to build a subscribe/unsubscribe facility.
Or a plugin system

My IRC/Jabber bot kdb uses such features
and is capable of loading/unloading/reloading plugins on the fly.

YOu're welcome to borrow some of it's code

http://hg.softcircuit.com.au/projects/kdb/

cheers
James

 

[Aaron Brady]

That should work for functions, but less successfully with classes. The
problem is that existing objects will still have the old behaviour even
after reloading the class.

Good catch, Mr. Steven. You could re-query on every call to a method
with __getattr__.

def __getattr__( I, name ):
cls= module.class_to_query
return cls.__getattr__( I, name ) #need to call __get__ on this

That way, when 'class_to_query' changes, the behavior changes.

Here's the implementation:
.... def methA( I ):
.... print 'methA one'
........ def __getattr__( I, key ):
.... return getattr( Behavior, key ).__get__( I, Behavior )
........ def methA( I ):
.... print 'methA two'
....methA two

You would have to soft-code 'Behavior' into the initializer of
'Dynamic' as a string, not the class object.

 

[James Mills]

@Aaron

Your code and suggestion is way too complicated.
Just register your objects. When you need to
reload your module, destroy the existing
objects and re-creat them.

This works well assuming you have a stable
running core that maintains the connection
and that code doesn't change much.

--JamesMills

 

[Aaron Brady]

@Aaron

Your code and suggestion is way too complicated.
Just register your objects. When you need to
reload your module, destroy the existing
objects and re-creat them.

This works well assuming you have a stable
running core that maintains the connection
and that code doesn't change much.

--JamesMills

The practical and general solutions aren't very similar. Practically,
I agree. You could do something generic like reassign __dict__ (like
the Borg pattern), or __class__.

To be completely general, you'd need an object which passes the type
test: "TypeError: unbound method f() must be called with A instance as
first argument (
got B instance instead)". Then you could call it in anything, Cls.meth
( obj ), regardless of if you created it before or after the class was
created.

Or you could have a class that looked up its (posessive) methods'
names on a deeper class which could be loaded later: the Delegate,
Proxy, or Adapter patterns.

Maybe Cls.meth( obj ) should be permitted to work on arbitrary objects
by a compiler switch or __future__ import.

 

[Paul Rubin]

So I'd like to restructure my app so that it can stay running and stay
logged in, yet I can still update and reload at least most of the
code. But I'm not sure what's the best way to do this. Should I move
the reloadable code into its own module, and then when I give my bot a
"reload" command, have it call reload on that module? Will that work,
and is there a better way?

If you are on Linux, an alternative might be to start a new version of
your program in a separate process, then transfer the open connections
from the old process to the new ones through Unix domain sockets. The
SCM_RIGHTS message lets you pass file descriptors around between
processes. I've never tried this myself but have always wanted to. I
think someone submitted a patch for Python's socket module a year or
so ago to support that operation, but I don't know if it was accepted.
You could always apply it yourself.

Generally, trying to hot-patch code is messy and dangerous even in
systems that were designed for it.

 

[Aaron Brady]

If you are on Linux, an alternative might be to start a new version of
your program in a separate process, then transfer the open connections
from the old process to the new ones through Unix domain sockets.  The
SCM_RIGHTS message lets you pass file descriptors around between
processes.  I've never tried this myself but have always wanted to.  I
think someone submitted a patch for Python's socket module a year or
so ago to support that operation, but I don't know if it was accepted.
You could always apply it yourself.

Generally, trying to hot-patch code is messy and dangerous even in
systems that were designed for it.

Sorry for the repeated posts, but sockets made me think of pickling.
(You can't pickle sockets.)
<__main__.A instance at 0x00B53328>

So, 's' was unpickled to be an instance of a new class. Probably not
the most elegant. Just make sure the sockets are preserved in a
separate object.

For yet another option, exec the code, and assign it to a method in a
class.
.... def f( self ): pass
....new f