What's the best way to iniatilize a function

[Jack]

I have a set of functions to wrap a library. For example,

mylib_init()
mylib_func()
mylib_exit()

or

handle = mylib_init()
mylib_func(handle)
mylib_exit(handle)

In order to call mylib_func(), mylib_init() has to be called once.
When it's done, or when program exits, mylib_exit() should
be called once to free resources.

I can list all three functions in a module and let the
application manage the init call and exit call. Or, better,
to have the wrapper function manage these calls. I'm currently
using a singleton class (see below.) It seems to work fine.

My questions here are:

1. every time I call the function:

MyLib().func()

part of the object creation code is called, at least to check if
there is an existing instance of the class, then return it. So it
may not be very efficient. Is there a better way?

2. what's the right way to call mylib_exit()? I put it in __del__(self)
but it is not being called in my simple test.


STATUS_UNINITIALIZED = 0
STATUS_INITIALIZED = 1
STATUS_ERROR = 2

class MyLib (object):
instance = None
status = STATUS_UNINITIALIZED

def __new__(cls, *args, **kargs):
if cls.instance is None:
cls.instance = object.__new__(cls, *args, **kargs)
return cls.instance

def __init__(self):
if self.status == STATUS_UNINITIALIZED:
mylib_init()
self.status = STATUS_INITIALIZED

def func(self):
return mylib_func()

def __del__(self):
mylib_exit()

 

[Steven D'Aprano]

I have a set of functions to wrap a library. For example,

mylib_init()
mylib_func()
mylib_exit()

or

handle = mylib_init()
mylib_func(handle)
mylib_exit(handle)

In order to call mylib_func(), mylib_init() has to be called once.
When it's done, or when program exits, mylib_exit() should
be called once to free resources.

Sounds like a module to me.

Write it as a module. Python will call your initialization code, once, the
first time you import it. Python will handle the garbage collection of
your resources when your module goes away. Modules are automatically
singletons.


# mylib wrapper(?) module
# Initialize a bunch of data
DATA = range(1000000)

def func():
return DATA[0] # or something useful...


Note that mylib.func() doesn't need to check that it is initialized,
because if it isn't, it can't be called! That is to say, if some of your
initialization code fails, an exception will be raised and the module will
not be imported.

Note two gotchas:

(1) Python can automatically free most data structures and close open
files, but if your needs are more sophisticated, this approach may not be
suitable.

(2) Module imports are cached. Calling "import mylib" imports the module,
but calling "del mylib" doesn't free it because of the cache.

Calling "del mylib; del sys.modules['mylib']" should remove the cache,
allowing Python's normal garbage collection to free your resources, but
note the above caveats regarding the types of resources that Python can
automatically free.

I can list all three functions in a module and let the
application manage the init call and exit call. Or, better,
to have the wrapper function manage these calls. I'm currently
using a singleton class (see below.) It seems to work fine.

Why do you care that it is a singleton? Surely the important thing is that
all the instances share the same state, not that there is only one
instance?

The easiest way to ensure all instances share the same state is to put all
the data you care about into class attributes instead of instance
attributes:


class MyLib(object):
DATA = range(100000) # initialized when the class is defined
def func(self):
return self.DATA[0]

Note that there are no __init__ or __new__ methods needed.

If you don't want the (trivial) expense of creating new instances just to
call the func method, use a class method that doesn't need an instance:

class MyLib(object):
DATA = range(100000) # initialized when the class is defined
@classmethod
def func(cls):
return cls.DATA[0]


See also the source to the random module for another way to expose a
class-based interface as if it were a set of functions.

My questions here are:

1. every time I call the function:

MyLib().func()

part of the object creation code is called, at least to check if
there is an existing instance of the class, then return it. So it
may not be very efficient. Is there a better way?

Yes. The caller should save the instance and reuse it, the same as any
other expensive instance:

obj = MyLib()
obj.func()
obj.func()

2. what's the right way to call mylib_exit()? I put it in __del__(self)
but it is not being called in my simple test.

instance.__del__ is only called when there are no references to the
instance.

instance = MyLib()
another = instance
data = [None, 1, instance]
something = {"key": instance}
del another # __del__ will not be called, but name "another" is gone
del instance # __del__ still not called, but name "instance" is gone
data = [] # __del__ still not called
something.clear() # now __del__ will be called!


It isn't easy to keep track of all the places you might have a reference
to something, but Python can do it much better than you can.

 

[Jack]

Thanks Steven, for the reply. Very helpful. I've got a lot to learn in
Python

Some questions:

(1) Python can automatically free most data structures and close open
files, but if your needs are more sophisticated, this approach may not be
suitable.

Since it's a wrapper of a DLL or .so file, I actually need to call
mylib_exit()
to do whatever cleanup the library needs to do.

instance.__del__ is only called when there are no references to the
instance.

I didn't call del explicitly. I'm expecting Python to call it when
the program exits. I put a logging line in __del__() but I never
see that line printed. It seems that __del__() is not being called
even when the program exits. Any idea why?

 

[Gregor Horvath]

I didn't call del explicitly. I'm expecting Python to call it when
the program exits. I put a logging line in __del__() but I never
see that line printed. It seems that __del__() is not being called
even when the program exits. Any idea why?

http://effbot.org/pyfaq/my-class-defines-del-but-it-is-not-called-when-i-delete-the-object.htm

better solutions:

http://docs.python.org/ref/try.html
http://docs.python.org/whatsnew/pep-343.html

Gregor

 

[Steven D'Aprano]

http://effbot.org/pyfaq/my-class-defines-del-but-it-is-not-called-when-i-delete-the-object.htm

better solutions:

http://docs.python.org/ref/try.html
http://docs.python.org/whatsnew/pep-343.html

They might be better solutions, but not for the Original Poster's problem.

The O.P. has a library that has to hang around for long time, not just a
call or two, and then needs to be closed.

try...except and with define *blocks of code*, not long-lasting libraries
with data, functions, etc. Maybe you could shoe-horn the O.P.'s problem
into a with block, but the result would surely be ugly and fragile.

 

[Laurent Pointal]

I didn't call del explicitly. I'm expecting Python to call it when
the program exits. I put a logging line in __del__() but I never
see that line printed. It seems that __del__() is not being called
even when the program exits. Any idea why?

If you make your extension available as a module (Python or C), you can use
atexit: http://docs.python.org/lib/module-atexit.html

Note: your exit handler will only be called at normal termination.

 

[Steven D'Aprano]

Thanks Steven, for the reply. Very helpful. I've got a lot to learn in
Python

Some questions:


Since it's a wrapper of a DLL or .so file, I actually need to call
mylib_exit()
to do whatever cleanup the library needs to do.

Well, there may be better ways, but I'd do something like this:


# === mylib module ===

# Exception raised if library is not initiated
class Uninitiated(Exception):
pass


def _init():
"""Private set-up code."""
global DATA, _INITIALIZED
DATA = range(100000) # or something more useful
_INITIALIZED = True

def _close():
"""Private tear-down code."""
global DATA, _INITIALIZED
del DATA, _INITIALIZED



def isinited():
"""Return True if the library is initialized, otherwise False."""
try:
_INITIALIZED; return True
except NameError:
return False



def init():
"""Public set-up code."""
if not isinited(): _init()

def close():
"""Public tear-down code."""
if isinited(): _close()

exit = close # alias for exit/close.


def func():
if isinited():
return DATA[0]
else:
raise Uninitiated("Library is not initialized")


All of the above can be modified to be in a class instead of a module.
That's especially useful if you can have multiple instances, perhaps with
different state.

I didn't call del explicitly. I'm expecting Python to call it when
the program exits. I put a logging line in __del__() but I never
see that line printed. It seems that __del__() is not being called
even when the program exits. Any idea why?

Python tries really really hard to call instance.__del__() but there are
pathological cases where it just can't. Maybe you've found one of them.

But I'm guessing that you may have defined a __del__ method, but not
actually created an instance. If so, __del__ will never be called.

This should work:

class Foo(object):
def __del__(self):
print "All gone now"

instance = Foo()


When you exit, instance will be garbage collected and instance.__del__()
will be called. But without the instance, __del__ is not called on the
class directly.


Hope this was some help,