__del__ pattern?

[Chris Curvey]

I need to ensure that there is only one instance of my python class on
my machine at a given time. (Not within an interpreter -- that would
just be a singleton -- but on the machine.) These instances are
created and destroyed, but there can be only one at a time.

So when my class is instantiated, I create a little lock file, and I
have a __del__ method that deletes the lock file. Unfortunately, there
seem to be some circumstances where my lock file is not getting
deleted. Then all the jobs that need that "special" class start
queueing up requests, and I get phone calls in the middle of the night.

Is there a better pattern to follow than using a __del__ method? I
just need to be absolutely, positively sure of two things:

1) There is only one instance of my special class on the machine at a
time.
2) If my special class is destroyed for any reason, I need to be able
to create another instance of the class.

 

[BranoZ]

So when my class is instantiated, I create a little lock file, and I

have a __del__ method that deletes the lock file. Unfortunately, there
seem to be some circumstances where my lock file is not getting
deleted.

Maybe the interpreter died by the signal.. in that case the __del__
is not called.

You can try 'flock', instead of lock files.

import fcntl

class Test1(object):

def __init__(self):
self.lock=open('/var/tmp/test1', 'w')
fcntl.flock(self.lock.fileno(), fcntl.LOCK_EX)
print 'Lock aquired!'

def __del__(self):
fcntl.flock(self.lock.fileno(), fcntl.LOCK_UN)
self.lock.close()

In this case, if interpreter dies, the lock is released by OS.

If you try to create another instance in the same interpreter
or another, the call will block in __init__. You can change it to
raise an exception instead.

BranoZ

 

[Leonhard Vogt]

So when my class is instantiated, I create a little lock file, and I

have a __del__ method that deletes the lock file.

Is there a better pattern to follow than using a __del__ method? I
just need to be absolutely, positively sure of two things:

1) There is only one instance of my special class on the machine at a
time.
2) If my special class is destroyed for any reason, I need to be able
to create another instance of the class.

Just some ideas

1) You could open a socket listening to a port
* Not sure what happens if the interpreter dies
* There cold be conflicts with other programs

2) Update the lockfile every xxx. If the lockfile
is older than e.g. 2*xxx disregard it.

3) Write the process id into the lockfile and check
if a process with this id is alive.
* I don't know if / how this can be done in python

Leonhard

 

[Michael Hudson]

I need to ensure that there is only one instance of my python class on
my machine at a given time.

I recommend modifying your requirements such that you ensure that
there is only one "active" instance of your class at any one time (or
something like that), and then use try:finally: blocks to ensure your
locks get removed.

Is there a better pattern to follow than using a __del__ method? I
just need to be absolutely, positively sure of two things:

1) There is only one instance of my special class on the machine at a
time.
2) If my special class is destroyed for any reason, I need to be able
to create another instance of the class.

As another poster mentioned, you also need to work out what you're
going to do if your process gets killed in a way that doesn't allow
finally blocks to run (this doesn't have much to do with Python).

Cheers,
mwh

 

[Tom Anderson]

Is there a better pattern to follow than using a __del__ method? I just
need to be absolutely, positively sure of two things:

An old hack i've seen before is to create a server socket - ie, make a
socket and bind it to a port:

import socket

class SpecialClass:
def __init__(self):
self.sock = socket.socket()
self.sock.bind(("", 4242))
def __del__(self):
self.sock.close()

Something like that, anyway.

Only one socket can be bound to a given port at any time, so the second
instance of SpecialClass will get an exception from the bind call, and
will be stillborn. This is a bit of a crufty hack, though - you end up
with an open port on your machine for no good reason. If you're running on
unix, you could try using a unix-domain socket instead; i'm not sure what
the binding semantics of those are, though.

I think Brano's suggestion of using flock is a better solution.

tom

 

[Peter Hansen]

Only one socket can be bound to a given port at any time, so the second
instance of SpecialClass will get an exception from the bind call, and
will be stillborn. This is a bit of a crufty hack, though - you end up
with an open port on your machine for no good reason. If

If you bind with self.sock.bind(('localhost', 4242)) instead, at least
you don't have much of a security risk since the port won't be available
for connections from outside the same machine. Using '' instead of
'localhost' means bind to *all* interfaces, not just the loopback one.

-Peter

 

[Tom Anderson]

If you bind with self.sock.bind(('localhost', 4242)) instead, at least
you don't have much of a security risk since the port won't be available
for connections from outside the same machine.

Excellent suggestion, thanks!

Using '' instead of 'localhost' means bind to *all* interfaces, not just
the loopback one.

Doesn't '' mean 'bind to the *default* interface'?

tom

 

[Peter Hansen]

Doesn't '' mean 'bind to the *default* interface'?

What does "default" mean, and is that definition in conflict with what I
said?

The docs say it means INADDR_ANY. They don't say what that means, so
you'd have to read up on the C socket calls to learn more.

Or some helpful soul will clarify for the class...

-Peter

 

[bryanjugglercryptographer]

An old hack i've seen before is to create a server socket - ie, make a
socket and bind it to a port:

import socket

class SpecialClass:
def __init__(self):
self.sock = socket.socket()
self.sock.bind(("", 4242))
def __del__(self):
self.sock.close()

Something like that, anyway.

Only one socket can be bound to a given port at any time, so the second
instance of SpecialClass will get an exception from the bind call, and
will be stillborn. This is a bit of a crufty hack, though - you end up
with an open port on your machine for no good reason.

Much worse, it's a bug. That pattern is for programs that need to
respond at a well-known port. In this case it doesn't work; the
fact that *someone* has a certain socket open does not mean that
this particular program is running.

 

[bryanjugglercryptographer]

I need to ensure that there is only one instance of my python class on
my machine at a given time. (Not within an interpreter -- that would
just be a singleton -- but on the machine.) These instances are
created and destroyed, but there can be only one at a time.

So when my class is instantiated, I create a little lock file, and I
have a __del__ method that deletes the lock file. Unfortunately, there
seem to be some circumstances where my lock file is not getting
deleted. Then all the jobs that need that "special" class start
queueing up requests, and I get phone calls in the middle of the night.

For a reasonably portable solution, leave the lock file open.
On most systems, you cannot delete an open file, and if the
program terminates, normally or abnormally, the file will be
closed.

When the program starts, it looks for the lock file, and if
it's there, tries to delete it; if the delete fails, another
instance is probably running. It then tries to create the
lock file, leaving it open; if the create fails, you probably
lost a race with another instance. When exiting cleanly, the
program closes the file and deletes it.

If the program crashes without cleaning up, the file will still
be there, but a new instance can delete it, assuming
permissions are right.


There are neater solutions that are Unix-only or Windows-only.
See BranzoZ's post for a Unix method.

 

[BranoZ]

For a reasonably portable solution, leave the lock file open.
On most systems, you cannot delete an open file,..

On most UNIXes, you can delete an open file.
Even flock-ed. This is BTW also an hack around flock.

1. Process A opens file /var/tmp/test1, and flocks descriptor.
2. Process H unlinks /var/tmp/test1
3. Process B opens file /var/tmp/test1, and flocks _another_
descriptor
4. Processes A and B are running simultaneously

Do you need protection agains H ?

Use file that is writeable by A and B in a directory that is
writeable only by root.

BranoZ

 

[Michael Hudson]

For a reasonably portable solution, leave the lock file open.
On most systems, you cannot delete an open file,

Uh, you can on unix -- what else did you have in mind for "most
systems"?

Cheers,
mwh

 

[Bryan Olson]

> (e-mail address removed) wrote:
>
>
> On most UNIXes, you can delete an open file.
> Even flock-ed. This is BTW also an hack around flock.

Yes, sorry; my bad.

> Use file that is writeable by A and B in a directory that is
> writeable only by root.

Is that portable? What's the sequence the program should try?

 

[BranoZ]

Is that portable?

I have the feeling that you are asking if it works on Windows.
No idea! I have only user experience with Windows.

On UNIX it is as portable as 'flock', which means all modern
Unices (be careful about NFS).

What's the sequence the program should try?

1.
open a file, which name was previously agreed on
(like /var/tmp/<prog-name>-<user-name>)

If it fails, report error and exit. System error or
somebody has created unaccessible file by the same name.

2.
Try to aquire a flock on the descriptor from step 1.

If it fails, some running process already has the lock, exit

3.
lock will be released and lockfile closed automaticaly by OS
on process exit.

import sys, fcntl

try:
lockfile=open('/var/tmp/test1', 'w')
fcntl.flock(lockfile.fileno(),
fcntl.LOCK_EX | fcntl.LOCK_NB)
except IOError:
print sys.exc_info()[1]
sys.exit(-1)

You can flock any open file, no matter if it is read/write/append.

BranoZ

 

[Jorgen Grahn]

What does "default" mean, and is that definition in conflict with what I
said?

The docs say it means INADDR_ANY. They don't say what that means, so
you'd have to read up on the C socket calls to learn more.

Or some helpful soul will clarify for the class...

INADDR_ANY means "every network interface you can find". This includes the
local loopback and all physical and logical network interfaces.

/Jorgen