Loading just in time

[D'Arcy J.M. Cain]

I am trying to create a utility module that only loads functions when
they are first called rather than loading everything. I have a bunch
of files in my utility directory with individual methods and for each I
have lines like this in __init__.py:

def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
calc_tax = _func_
return _func_(*arg, **name)

This works the first time I call utility.calc_tax but if I call it
again I get a "TypeError: 'module' object is not callable" error. Is
there any way to do what I want or do I have to put everything back
into a single file. Of course, I can simply change all my calls to
utility.calc_tax.calc_tax(...) but I have a lot of code to change if I
do that.

Thanks.

 

[D'Arcy J.M. Cain]

You are stuck in a futile battle called "premature optimization". I would

I thought that I might be.

suggest that you stop worrying about any performance you would gain from doing
something like this. Python has been "highly" optimized to handle imports in a
very efficient way. Just put your functions in a file and import them.

Performance optimization wasn't really my goal here. What I was
looking for was the ability to spread the functions around different
files to manage them better from a proggrammer's POV.

Oh well. I guess I will just combine them all again. At least I found
some dead code, duplicate functions and just plain bad code doing this
little exercise.

 

[samwyse]

I am trying to create a utility module that only loads functions when
they are first called rather than loading everything.  I have a bunch
of files in my utility directory with individual methods and for each I
have lines like this in __init__.py:

def calc_tax(*arg, **name):
    from calc_tax import calc_tax as _func_
    calc_tax = _func_
    return _func_(*arg, **name)

This doesn't do what you think. The line "calc_tax = _func_" is
probably modifying a local variable that is then thrown away. I've
got a slightly different (simpler) version to illustrate:

=== main.py ===

def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
#global calc_tax
calc_tax = _func_
print '_func_ is', repr(_func_)
print 'calc_tax is', repr(calc_tax)
return _func_(*arg, **name)

print 'before: calc_tax is', repr(calc_tax)
result = calc_tax()
print 'after: calc_tax is', repr(calc_tax)

=== calc_tax.py ===

def calc_tax(*arg, **name):
return 42

=== end of files ===

Running main.py gives this:

before: calc_tax is <function calc_tax at 0x015049F0>
_func_ is <function calc_tax at 0x014950B0>
calc_tax is <function calc_tax at 0x014950B0>
after: calc_tax is <function calc_tax at 0x015049F0>

Note that the value of calc_test is the same in the first and last
lines.

If you uncomment the line "#global calc_tax" and run it again, you get
this:

before: calc_tax is <function calc_tax at 0x01504A30>
_func_ is <function calc_tax at 0x014950B0>
calc_tax is <function calc_tax at 0x014950B0>
after: calc_tax is <function calc_tax at 0x014950B0>

Interestingly, neither version gives me a TypeError, no matter how
many times I call calc_tax.

(BTW, you might want to look up Memoization; it's very similar to what
you want to do, and might give you a way to more efficiently code
things.)

 

[Ross Ridge]

def calc_tax(*arg, **name):
from calc_tax import calc_tax as _func_
calc_tax = _func_
return _func_(*arg, **name)

This should do what you want:

def calc_tax(*arg, **name):
global calc_tax
from calc_tax import calc_tax
return calc_tax(*arg, **name)

I suspect though that the cost of importing a lot of little modules
won't be as bad as you might think.

Ross Ridge