Overriding a global

[Roy Smith]

I've got a code pattern I use a lot. In each module, I create a logger
for the entire module and log to it all over:

logger = logging.getLogger('my.module.name')

class Foo:
def function(self):
logger.debug('stuff')
logger.debug('other stuff')

and so on. This works, but every once in a while I decide that a
particular function needs a more specific logger, so I can adjust the
logging level for that function independent of the rest of the module.
What I really want to do is:

def function(self):
logger = logger.getChild('function')
logger.debug('stuff')
logger.debug('other stuff')

which lets me not have to change any lines of code other than inserting
the one to redefine logger. Unfortunately, that's not legal Python (it
leads to "UnboundLocalError: local variable 'logger' referenced before
assignment").

Any ideas on the best way to implement this?

 

[MRAB]

I've got a code pattern I use a lot. In each module, I create a logger
for the entire module and log to it all over:

logger = logging.getLogger('my.module.name')

class Foo:
def function(self):
logger.debug('stuff')
logger.debug('other stuff')

and so on. This works, but every once in a while I decide that a
particular function needs a more specific logger, so I can adjust the
logging level for that function independent of the rest of the module.
What I really want to do is:

def function(self):
logger = logger.getChild('function')
logger.debug('stuff')
logger.debug('other stuff')

which lets me not have to change any lines of code other than inserting
the one to redefine logger. Unfortunately, that's not legal Python (it
leads to "UnboundLocalError: local variable 'logger' referenced before
assignment").

Any ideas on the best way to implement this?

You could use a different name:

def function(self):
logger2 = logger.getChild('function')
logger2.debug('stuff')
logger2.debug('other stuff')

or use 'globals':

def function(self):
logger = globals()['logger'].getChild('function')
logger.debug('stuff')
logger.debug('other stuff')

 

[Roy Smith]

or use 'globals':

def function(self):
logger = globals()['logger'].getChild('function')
logger.debug('stuff')
logger.debug('other stuff')

Ah-ha! That's precisely what I was looking for. Much appreciated.

 

[Terry Reedy]

What I really want to do is:

def function(self):

Add a global statement to rebind a global name:
global logger

 

[Terry Reedy]

Add a global statement to rebind a global name:
global logger

But I see that that is not what you want to do, which is to override the
global name just within the function while still accessing the global
name. MRAB's solution does that nicely.

 

[Peter Otten]

I've got a code pattern I use a lot. In each module, I create a logger
for the entire module and log to it all over:

logger = logging.getLogger('my.module.name')

class Foo:
def function(self):
logger.debug('stuff')
logger.debug('other stuff')

and so on. This works, but every once in a while I decide that a
particular function needs a more specific logger, so I can adjust the
logging level for that function independent of the rest of the module.
What I really want to do is:

def function(self):
logger = logger.getChild('function')
logger.debug('stuff')
logger.debug('other stuff')

which lets me not have to change any lines of code other than inserting
the one to redefine logger. Unfortunately, that's not legal Python (it
leads to "UnboundLocalError: local variable 'logger' referenced before
assignment").

Any ideas on the best way to implement this?

def function(self, logger=logger.getChild("function")):
logger.debug("stuff")

But the "best way" is to use another name.

 

[Jean-Michel Pichavant]

or use 'globals':

def function(self):
logger = globals()['logger'].getChild('function')
logger.debug('stuff')
logger.debug('other stuff')

Ah-ha! That's precisely what I was looking for. Much appreciated.

Using the same name for 2 different objects is a bad idea in general.
In debug mode, i.e. the logger is configured with the debug level, you
could simply write down the filename and the line number on your log
events.

formatter = logging.Formatter('%(name)s : %(filename)s %(lineno)s
%(message)s')

So you can easily see who logged what (if I'm not wrong, that's what
you're trying to do)

JM

 

[Steven D'Aprano]

Using the same name for 2 different objects is a bad idea in general.

We have namespaces precisely so you don't need to care about making names
globally unique.

 

[Dave Angel]

We have namespaces precisely so you don't need to care about making names
globally unique.

True, but in this code, the function is trying to both use the global
value, but also a local that deliberately has the same name, but a
different meaning and "value". The CPython compiler doesn't make this
easy, and I think the globals() technique is unnecessarily obscure, as
is the default-argument trick.

If a function knows of the presence of a global, it's not asking too
much for it to not re-use the same name in local scope.

Since it seems to be in vogue to propose language changes, how about a
new place for 'as' ?
def myfunc():
global logger as g_logger
logger = g_logger.debug('stuff').getChild('function')

 

[Dave Angel]

It's just a function wanting to act as-if it were in a different
environment than its default. By that same reasoning you could state that
"If a function knows of the presence of a built-in, it's not asking
too much for it to not re-use the same name in local scope."
<SNIP>

It's entirely different. It's only the same if the function actually
tries to call the built-in, then also wants a local variable with a
different purpose. Think a little about what I mean that the function
"knows of the presence." I did not say the programmer knows of the
presence.

 

[Steven D'Aprano]

I disagree. The language makes it difficult, and it *should* be
difficult to do what you describe.

The tricks to achieve it are obscure and ugly, which is a good thing
IMO: they're a code smell that the design of the code needs changing.

Devil's Advocate: perhaps not. Think of local and global names as
analogous to instance and class attributes. There are good use cases for
making something a class attribute, while allowing instances to override
that name with an instance attribute. I see a reasonable case for saying
"use this global, unless a local overrides it".

Similarly, globals override built-ins with the same name; while
monkeypatching needs to be used with care, it is a legitimate technique.

To a human reader, the following pseudocode might be ambiguous, but
either case makes sense:

x = 1
def spam():
print x # prints 1
x = 2 # does this create a new local x, or modify the old global x?
print x # unambiguously prints 2

print x # prints 1 or 2


Python doesn't allow this, but another language might; in Python, a
reasonable way to get similar behaviour might be:

x = 1
def spam():
print globals()['x']
x = 2 # unambiguously creates a new local x
print x # unambiguously prints 2

print x # unambiguously prints 1

 

[Ian Kelly]

Rebinding logger locally in a function is really no
different to a subclass rebinding a variable from its main class using that
class' value. The only difference is that, in that case, you have an
alternate binding to the original value.

No, there is another difference, the reason for rebinding the name.
In a subclass, you would rebind a class attribute because that
particular attribute, which you need to change, is used and expected
by external code, either in the base class or in code that uses its
API (or both). Local variables in functions, on the other hand, are
not externally visible, so there is no need to do this in order to
conform to the expectations of external code. All it does in that
case is to sow potential confusion.

 

[Joshua Landau]

No, there is another difference, the reason for rebinding the name.
In a subclass, you would rebind a class attribute because that
particular attribute, which you need to change, is used and expected
by external code, either in the base class or in code that uses its
API (or both). Local variables in functions, on the other hand, are
not externally visible, so there is no need to do this in order to
conform to the expectations of external code. All it does in that
case is to sow potential confusion.

So you're saying you should never extend methods or attributes that
aren't meant to be used as part of of the API? Because I can claim
guilty on this point.

I'd make this longer, but I've got class now

 

[Jean-Michel Pichavant]

We have namespaces precisely so you don't need to care about making names
globally unique.

I don't get your point, namespaced names are unique, by definition.

foo.aname <> bar.aname

The OP showed a code where there was a confusion between a global name
and a local one. There's no namespace involved. Having a local name
identical to a global one is a bad idea, def.

JM

 

[Steven D'Aprano]

I don't get your point, namespaced names are unique, by definition.

foo.aname <> bar.aname

Assuming foo and bar are not names for the same object, there are at
least three namespaces here: the local namespace, where foo and bar can
be found, the foo.__dict__ namespace, and the bar.__dict__ namespace.

The OP showed a code where there was a confusion between a global name
and a local one. There's no namespace involved. Having a local name
identical to a global one is a bad idea, def.

Of course there are namespaces involved. There is the global namespace,
and the local namespace. That's how you can have x inside a function
without it overwriting global x outside of it, because they are different
namespaces. Which is my point.

When I write this:

x = 1

def spam():
x = 2

def ham():
x = 3

The three x's don't clash because they are in three separate namespaces.

 

[Jean-Michel Pichavant]

Assuming foo and bar are not names for the same object, there are at
least three namespaces here: the local namespace, where foo and bar can
be found, the foo.__dict__ namespace, and the bar.__dict__ namespace.




Of course there are namespaces involved. There is the global namespace,
and the local namespace. That's how you can have x inside a function
without it overwriting global x outside of it, because they are different
namespaces. Which is my point.

When I write this:

x = 1

def spam():
x = 2

def ham():
x = 3

The three x's don't clash because they are in three separate namespaces.

I know that, why do I have the feel this is juste a semantic issue ?
Aren't you/I/we confusion namespaces & scopes ?

Anyway, semantic is not what we're interested in here.

writing

x = 1

def spam():
x = 2


is in general a bad idea. That was my point.

JM

 

[Ian Kelly]

So you're saying you should never extend methods or attributes that
aren't meant to be used as part of of the API? Because I can claim
guilty on this point.

No, I'm only saying that replacing attributes in subclasses is
accepted because it is necessary due to external dependencies, and
that local variables in functions don't have that excuse.

 

[Ian Kelly]

But they aren't needed due to external dependencies if they're
implementation-specific and not part of the API, no?

By "external dependencies" I mean anything that's not specifically
part of the subclass. This includes the base class. If they're not
part of the API, then the base class presumably uses them for
something, and by replacing them, you change the behavior of that base
functionality. That's an external dependency.

 

[Jean-Michel Pichavant]

On 13 December 2011 13:30, Jean-Michel Pichavant

writing

x = 1

def spam():
x = 2

is in general a bad idea. That was my point.


Why? I have a few (probably wrong) guesses.

Because you expect it to be the same every time you use it?
Well, then this should be "in general a bad idea":
x = 1; print(x); x = 2; print(x)

you're changing the value of x, that's fine. In the example above, the
assignement is not the problem. The problem is that you create 2
different 'x', one in globals(), and one in locals(). Using the same
name within 2 implicit namespaces is a bad idead in general because it
can be difficult to know which one is used.

If you want to have fun, try this code, prepared to be amazed. There's
something funny with the global statement, it's applied on the whole
block no matter where it is stated in the block.
x=1 # global

def spam():
x = 2 # local (or so you may think)
print x
global x # I need to use the global one now
print x
print locals()

For more fun you could create a 'x' name in __builtin__ and import it so
that people never know which x you are using.

Even though it makes total sense to me.

Is it because it's used to different purpose between similarly-looking
functions?
This looks fine, though:
def func1(): x=1; print(x)
def func2(): x=2; print(x)

Is it because it looks like a reassignment of the more global x?
I don't have an example here but, simply put, I don't believe this. We
can use "id" as our own local variable without thinking that we're
tampering with "__builtins__.id". I don't see it as much of a leap
from builtin to global (except that you /*can*/ do "dir = 1; del dir;
dir" without error).

That said, I'm sorta' just guessing the reason you might think it's a
bad idea.

The problem makes little sense when using names like x or func1. Besides
namespace issues, naming 2 *different objects* with the same meaningful
name is usually a bad idea and points the fact that your names are no
that meaningful. To go back to the original post, having a 'logger' that
may name 2 different logger object during the execution is a bad idea.
One quick way to fix it is to name the logger 'currentLogger', this way
you warn the reader that the logger named by curentLogger may change
over time.

As someone sugggested in this thread one other option is to use a
different name for the second logger.

JM

 

[Chris Angelico]

The problem makes little sense when using names like x or func1. Besides
namespace issues, naming 2 *different objects* with the same meaningful name
is usually a bad idea and points the fact that your names are no that
meaningful.

So... it's a bad idea for me to use 'i' many times in my code, with
the same name having different meanings in different places? In
languages with infinitely-nesting scopes (one of Python's great lacks,
imho), I've often had three different variables with the same names,
all perfectly valid, and all doing what they should. It's not just
loop indices - I used to have a piece of code in which 'res' was a
MySQL resource being processed in a loop, and I had three nested
loops. Each time I referenced 'res', it used the innermost available
resource, which was precisely what I wanted. If I'd arbitrarily had to
guarantee that all variable names were unique, I would have had
completely unnecessary fiddling around.

Python wouldn't let you do that with three nested 'res'es in one
function, but you can perfectly reasonably have a global and a local.
It makes perfect sense... which is a good reason for keeping it legal.

ChrisA