__del__ pattern?

Discussion in 'Python' started by Chris Curvey, Aug 15, 2005.

  1. Chris Curvey

    Chris Curvey Guest

    I need to ensure that there is only one instance of my python class on
    my machine at a given time. (Not within an interpreter -- that would
    just be a singleton -- but on the machine.) These instances are
    created and destroyed, but there can be only one at a time.

    So when my class is instantiated, I create a little lock file, and I
    have a __del__ method that deletes the lock file. Unfortunately, there
    seem to be some circumstances where my lock file is not getting
    deleted. Then all the jobs that need that "special" class start
    queueing up requests, and I get phone calls in the middle of the night.

    Is there a better pattern to follow than using a __del__ method? I
    just need to be absolutely, positively sure of two things:

    1) There is only one instance of my special class on the machine at a
    time.
    2) If my special class is destroyed for any reason, I need to be able
    to create another instance of the class.
     
    Chris Curvey, Aug 15, 2005
    #1
    1. Advertising

  2. Chris Curvey

    BranoZ Guest

    > So when my class is instantiated, I create a little lock file, and I
    > have a __del__ method that deletes the lock file. Unfortunately, there
    > seem to be some circumstances where my lock file is not getting
    > deleted.


    Maybe the interpreter died by the signal.. in that case the __del__
    is not called.

    You can try 'flock', instead of lock files.

    import fcntl

    class Test1(object):

    def __init__(self):
    self.lock=open('/var/tmp/test1', 'w')
    fcntl.flock(self.lock.fileno(), fcntl.LOCK_EX)
    print 'Lock aquired!'

    def __del__(self):
    fcntl.flock(self.lock.fileno(), fcntl.LOCK_UN)
    self.lock.close()

    In this case, if interpreter dies, the lock is released by OS.

    If you try to create another instance in the same interpreter
    or another, the call will block in __init__. You can change it to
    raise an exception instead.

    BranoZ
     
    BranoZ, Aug 15, 2005
    #2
    1. Advertising

  3. > So when my class is instantiated, I create a little lock file, and I
    > have a __del__ method that deletes the lock file.


    > Is there a better pattern to follow than using a __del__ method? I
    > just need to be absolutely, positively sure of two things:
    >
    > 1) There is only one instance of my special class on the machine at a
    > time.
    > 2) If my special class is destroyed for any reason, I need to be able
    > to create another instance of the class.
    >


    Just some ideas

    1) You could open a socket listening to a port
    * Not sure what happens if the interpreter dies
    * There cold be conflicts with other programs

    2) Update the lockfile every xxx. If the lockfile
    is older than e.g. 2*xxx disregard it.

    3) Write the process id into the lockfile and check
    if a process with this id is alive.
    * I don't know if / how this can be done in python

    Leonhard
     
    Leonhard Vogt, Aug 15, 2005
    #3
  4. "Chris Curvey" <> writes:

    > I need to ensure that there is only one instance of my python class on
    > my machine at a given time.


    I recommend modifying your requirements such that you ensure that
    there is only one "active" instance of your class at any one time (or
    something like that), and then use try:finally: blocks to ensure your
    locks get removed.

    > Is there a better pattern to follow than using a __del__ method? I
    > just need to be absolutely, positively sure of two things:
    >
    > 1) There is only one instance of my special class on the machine at a
    > time.
    > 2) If my special class is destroyed for any reason, I need to be able
    > to create another instance of the class.


    As another poster mentioned, you also need to work out what you're
    going to do if your process gets killed in a way that doesn't allow
    finally blocks to run (this doesn't have much to do with Python).

    Cheers,
    mwh

    --
    The above comment may be extremely inflamatory. For your
    protection, it has been rot13'd twice.
    -- the signature of "JWhitlock" on slashdot
     
    Michael Hudson, Aug 15, 2005
    #4
  5. Chris Curvey

    Tom Anderson Guest

    On Mon, 15 Aug 2005, Chris Curvey wrote:

    > Is there a better pattern to follow than using a __del__ method? I just
    > need to be absolutely, positively sure of two things:


    An old hack i've seen before is to create a server socket - ie, make a
    socket and bind it to a port:

    import socket

    class SpecialClass:
    def __init__(self):
    self.sock = socket.socket()
    self.sock.bind(("", 4242))
    def __del__(self):
    self.sock.close()

    Something like that, anyway.

    Only one socket can be bound to a given port at any time, so the second
    instance of SpecialClass will get an exception from the bind call, and
    will be stillborn. This is a bit of a crufty hack, though - you end up
    with an open port on your machine for no good reason. If you're running on
    unix, you could try using a unix-domain socket instead; i'm not sure what
    the binding semantics of those are, though.

    I think Brano's suggestion of using flock is a better solution.

    tom

    --
    Gin makes a man mean; let's booze up and riot!
     
    Tom Anderson, Aug 15, 2005
    #5
  6. Chris Curvey

    Peter Hansen Guest

    Tom Anderson wrote:
    > Only one socket can be bound to a given port at any time, so the second
    > instance of SpecialClass will get an exception from the bind call, and
    > will be stillborn. This is a bit of a crufty hack, though - you end up
    > with an open port on your machine for no good reason. If


    If you bind with self.sock.bind(('localhost', 4242)) instead, at least
    you don't have much of a security risk since the port won't be available
    for connections from outside the same machine. Using '' instead of
    'localhost' means bind to *all* interfaces, not just the loopback one.

    -Peter
     
    Peter Hansen, Aug 16, 2005
    #6
  7. Chris Curvey

    Tom Anderson Guest

    On Mon, 15 Aug 2005, Peter Hansen wrote:

    > Tom Anderson wrote:
    >
    >> Only one socket can be bound to a given port at any time, so the second
    >> instance of SpecialClass will get an exception from the bind call, and
    >> will be stillborn. This is a bit of a crufty hack, though - you end up
    >> with an open port on your machine for no good reason. If

    >
    > If you bind with self.sock.bind(('localhost', 4242)) instead, at least
    > you don't have much of a security risk since the port won't be available
    > for connections from outside the same machine.


    Excellent suggestion, thanks!

    > Using '' instead of 'localhost' means bind to *all* interfaces, not just
    > the loopback one.


    Doesn't '' mean 'bind to the *default* interface'?

    tom

    --
    All we need now is a little energon and a lotta luck
     
    Tom Anderson, Aug 16, 2005
    #7
  8. Chris Curvey

    Peter Hansen Guest

    Tom Anderson wrote:
    > On Mon, 15 Aug 2005, Peter Hansen wrote:
    >> Using '' instead of 'localhost' means bind to *all* interfaces, not
    >> just the loopback one.

    >
    > Doesn't '' mean 'bind to the *default* interface'?


    What does "default" mean, and is that definition in conflict with what I
    said?

    The docs say it means INADDR_ANY. They don't say what that means, so
    you'd have to read up on the C socket calls to learn more.

    Or some helpful soul will clarify for the class... :)

    -Peter
     
    Peter Hansen, Aug 16, 2005
    #8
  9. Chris Curvey

    Guest

    Tom Anderson wrote:
    > On Mon, 15 Aug 2005, Chris Curvey wrote:
    >
    > > Is there a better pattern to follow than using a __del__ method? I just
    > > need to be absolutely, positively sure of two things:

    >
    > An old hack i've seen before is to create a server socket - ie, make a
    > socket and bind it to a port:
    >
    > import socket
    >
    > class SpecialClass:
    > def __init__(self):
    > self.sock = socket.socket()
    > self.sock.bind(("", 4242))
    > def __del__(self):
    > self.sock.close()
    >
    > Something like that, anyway.
    >
    > Only one socket can be bound to a given port at any time, so the second
    > instance of SpecialClass will get an exception from the bind call, and
    > will be stillborn. This is a bit of a crufty hack, though - you end up
    > with an open port on your machine for no good reason.


    Much worse, it's a bug. That pattern is for programs that need to
    respond at a well-known port. In this case it doesn't work; the
    fact that *someone* has a certain socket open does not mean that
    this particular program is running.


    --
    --Bryan
     
    , Aug 16, 2005
    #9
  10. Chris Curvey

    Guest

    Chris Curvey wrote:
    > I need to ensure that there is only one instance of my python class on
    > my machine at a given time. (Not within an interpreter -- that would
    > just be a singleton -- but on the machine.) These instances are
    > created and destroyed, but there can be only one at a time.
    >
    > So when my class is instantiated, I create a little lock file, and I
    > have a __del__ method that deletes the lock file. Unfortunately, there
    > seem to be some circumstances where my lock file is not getting
    > deleted. Then all the jobs that need that "special" class start
    > queueing up requests, and I get phone calls in the middle of the night.


    For a reasonably portable solution, leave the lock file open.
    On most systems, you cannot delete an open file, and if the
    program terminates, normally or abnormally, the file will be
    closed.

    When the program starts, it looks for the lock file, and if
    it's there, tries to delete it; if the delete fails, another
    instance is probably running. It then tries to create the
    lock file, leaving it open; if the create fails, you probably
    lost a race with another instance. When exiting cleanly, the
    program closes the file and deletes it.

    If the program crashes without cleaning up, the file will still
    be there, but a new instance can delete it, assuming
    permissions are right.


    There are neater solutions that are Unix-only or Windows-only.
    See BranzoZ's post for a Unix method.

    --
    --Bryan
     
    , Aug 16, 2005
    #10
  11. Chris Curvey

    BranoZ Guest

    wrote:
    > For a reasonably portable solution, leave the lock file open.
    > On most systems, you cannot delete an open file,..

    On most UNIXes, you can delete an open file.
    Even flock-ed. This is BTW also an hack around flock.

    1. Process A opens file /var/tmp/test1, and flocks descriptor.
    2. Process H unlinks /var/tmp/test1
    3. Process B opens file /var/tmp/test1, and flocks _another_
    descriptor
    4. Processes A and B are running simultaneously

    Do you need protection agains H ?

    Use file that is writeable by A and B in a directory that is
    writeable only by root.

    BranoZ
     
    BranoZ, Aug 17, 2005
    #11
  12. writes:

    > Chris Curvey wrote:
    >> I need to ensure that there is only one instance of my python class on
    >> my machine at a given time. (Not within an interpreter -- that would
    >> just be a singleton -- but on the machine.) These instances are
    >> created and destroyed, but there can be only one at a time.
    >>
    >> So when my class is instantiated, I create a little lock file, and I
    >> have a __del__ method that deletes the lock file. Unfortunately, there
    >> seem to be some circumstances where my lock file is not getting
    >> deleted. Then all the jobs that need that "special" class start
    >> queueing up requests, and I get phone calls in the middle of the night.

    >
    > For a reasonably portable solution, leave the lock file open.
    > On most systems, you cannot delete an open file,


    Uh, you can on unix -- what else did you have in mind for "most
    systems"?

    Cheers,
    mwh

    --
    Well, yes. I don't think I'd put something like "penchant for anal
    play" and "able to wield a buttplug" in a CV unless it was relevant
    to the gig being applied for... -- Matt McLeod, asr
     
    Michael Hudson, Aug 17, 2005
    #12
  13. Chris Curvey

    Bryan Olson Guest

    BranoZ wrote:
    > wrote:
    >
    >>For a reasonably portable solution, leave the lock file open.
    >>On most systems, you cannot delete an open file,..

    >
    > On most UNIXes, you can delete an open file.
    > Even flock-ed. This is BTW also an hack around flock.


    Yes, sorry; my bad.

    > Use file that is writeable by A and B in a directory that is
    > writeable only by root.


    Is that portable? What's the sequence the program should try?


    --
    --Bryan
     
    Bryan Olson, Aug 19, 2005
    #13
  14. Chris Curvey

    BranoZ Guest

    Bryan Olson wrote:
    > > Use file that is writeable by A and B in a directory that is
    > > writeable only by root.

    >
    > Is that portable?


    I have the feeling that you are asking if it works on Windows.
    No idea! I have only user experience with Windows.

    On UNIX it is as portable as 'flock', which means all modern
    Unices (be careful about NFS).

    > What's the sequence the program should try?


    1.
    open a file, which name was previously agreed on
    (like /var/tmp/<prog-name>-<user-name>)

    If it fails, report error and exit. System error or
    somebody has created unaccessible file by the same name.

    2.
    Try to aquire a flock on the descriptor from step 1.

    If it fails, some running process already has the lock, exit

    3.
    lock will be released and lockfile closed automaticaly by OS
    on process exit.

    import sys, fcntl

    try:
    lockfile=open('/var/tmp/test1', 'w')
    fcntl.flock(lockfile.fileno(),
    fcntl.LOCK_EX | fcntl.LOCK_NB)
    except IOError:
    print sys.exc_info()[1]
    sys.exit(-1)

    You can flock any open file, no matter if it is read/write/append.

    BranoZ
     
    BranoZ, Aug 19, 2005
    #14
  15. Chris Curvey

    Jorgen Grahn Guest

    On Tue, 16 Aug 2005 08:03:58 -0400, Peter Hansen <> wrote:
    > Tom Anderson wrote:
    >> On Mon, 15 Aug 2005, Peter Hansen wrote:
    >>> Using '' instead of 'localhost' means bind to *all* interfaces, not
    >>> just the loopback one.

    >>
    >> Doesn't '' mean 'bind to the *default* interface'?

    >
    > What does "default" mean, and is that definition in conflict with what I
    > said?
    >
    > The docs say it means INADDR_ANY. They don't say what that means, so
    > you'd have to read up on the C socket calls to learn more.
    >
    > Or some helpful soul will clarify for the class... :)


    INADDR_ANY means "every network interface you can find". This includes the
    local loopback and all physical and logical network interfaces.

    /Jorgen

    --
    // Jorgen Grahn <jgrahn@ Ph'nglui mglw'nafh Cthulhu
    \X/ algonet.se> R'lyeh wgah'nagl fhtagn!
     
    Jorgen Grahn, Aug 21, 2005
    #15
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jane Austine
    Replies:
    2
    Views:
    372
    Steven Taschuk
    Jul 2, 2003
  2. Jane Austine
    Replies:
    1
    Views:
    483
    Erik Max Francis
    Jul 2, 2003
  3. seth
    Replies:
    0
    Views:
    367
  4. Kepes Krisztian

    Java final vs Py __del__

    Kepes Krisztian, Nov 27, 2003, in forum: Python
    Replies:
    2
    Views:
    376
    Jay O'Connor
    Nov 27, 2003
  5. Jey Kottalam

    __del__ warnings apply to tp_dealloc?

    Jey Kottalam, Feb 3, 2004, in forum: Python
    Replies:
    1
    Views:
    374
    Greg Chapman
    Feb 3, 2004
Loading...

Share This Page