global, globals(), _global ?

[robert]

Using global variables in Python often raises chaos. Other languages use
a clear prefix for globals.

* you forget to declare a global
* or you declare a global too much or in conflict
* you have a local identical variable name and want to save/load it
to/from the global with same name
* while you add code, the definition of globals moves more and more
apart from their use cases -> weirdness; programmers thinking is fragmented
* using globals()['xy'] is 'stringy non-program-code'


Thus, since long time at the head of my bigger modules I often put...

_global = sys.modules[__name__]

....and use global variables only/mainly like

def f(y):
v=x=_global.x
_global.y=y
...

for me this method is much more clear overall. And it is in line with
the clear exposition of "self" as regular object in Python methods (in a
much more self-similiar way compared to frozen @ $ m_... hacks in other
languages)

I know, this home created _global creates a circular ref, but usually
this is not serious as modules are unload only at the end of a program.

( side question: does this maybe hinder global objects from being
__del__'ed correctly; is at least gc() run when all modules are pulled
off at exit() )

Somehow I miss a nice standard method for using globals in an
unfragmented way everywhere. What do you think?

Robert

 

[Xaver Hinterhuber]

Hi Robert,

I was using global variables some time ago, too.
But with the time the program simply got unmaintainable, because it is very
hard
to trace, why a global variable has some special value and not the one, you
thought it should have.
So I redesigned the program and now I can do it without global variables.

To me global variables are now in most cases a sign of bad design.
Is there no other way to do it?

Greets

Xaver

 

[robert]

Hi Robert,

I was using global variables some time ago, too.
But with the time the program simply got unmaintainable, because it is very
hard
to trace, why a global variable has some special value and not the one, you
thought it should have.
So I redesigned the program and now I can do it without global variables.

To me global variables are now in most cases a sign of bad design.
Is there no other way to do it?

Yes, one should put common _application_ variables of importance to
config instances or data-modules/classes etc.

Yet globals variables in Python are anyway "module commons". That is a
frequent practical need.

Grep e.g. the python standard modules for use of global variables and
use of "global "

There are many practical module globals, which control e.g. the mode of
a module, store precomputed/cached stuff etc., and where you do not open
an extra data structure (for now).

Most variable read-s in Python anyway go to module globals - as there
are no other kinds of namespaces except __builtins__

( And that later scheme is fairly wonderful - compare for example the
namespace fuzz in C/C++, Pascal, Ruby, ... where you never know which
module file addeds what to which namespace;

In Ruby they even scribble from anywhere to _any_ class _and_ any
namespace without the barrier of a dot or subclassing or anything -
using somehow by random the same name already joins!? A threat for good
modularization of code. Not far from free flat memory programming
Don't know how they keep bigger projects managable in this language.
)


Yet, python module globals are set with this strange fragmented "global
name ; name=..".

But the common storage object in question is an ordinary module anyway.
Why not stay self-similiar and use it as such in dot-syntax for write-s
(and often for read-s in places, where you want to express the use of a
special global explicitly in order to not confuse things with locals,
and want to maintain the code readable.

Robert

 

[Fredrik Lundh]

Most variable read-s in Python anyway go to module globals - as there
are no other kinds of namespaces except __builtins__

your post made some sense until I got to this paragraph, which appears to
completely ignore local variables, arguments, and variables in intermediate
scopes ? maybe you could clarify ?

</F>

 

[Roy Smith]

Using global variables in Python often raises chaos. Other languages use
a clear prefix for globals.

Unsing globals raises chaos in any language. They should be shunned and
avoided.

 

[Daniel Dittmar]

Unsing globals raises chaos in any language. They should be shunned and
avoided.

Solution: replace globals with Singletons ;-)

Daniel

 

[robert]

:




your post made some sense until I got to this paragraph, which appears to
completely ignore local variables, arguments, and variables in intermediate
scopes ? maybe you could clarify ?

Of course the local namespaces on the stack are the unmentioned default
(the intermediate frames are half way on the stack ..).

The discussion focused on off-stack, "common" name spaces. Python has
done the right magic in order to let modularize code well.

Just self-similiar self-inspection is not (yet) done to the best in some
places: this "_global" own module, self inspection in
functions/classes/methods (nameless recursion), stack handling/callcc

Robert

 

[Alex Martelli]

( And that later scheme is fairly wonderful - compare for example the
namespace fuzz in C/C++, Pascal, Ruby, ... where you never know which
module file addeds what to which namespace;

Pascal (per se) doesn't really have much by the way of namespaces
(alas). C++ (per se) does, with zap::zop=23 being rather unambiguous (C
compatibility and "using namespace zap" can muddy it up, but that's like
saying, e.g., that "from zap import *" muddies things up in Python:
true, but the obvious solution in both cases is "just don't do it";-).

Sure, any C++ or Ruby soure file can reopen a namespace or class,
respectifely -- but how's that different from Python's
"anothermodule.zop=23"? It's much of a muchness.

In Ruby they even scribble from anywhere to _any_ class _and_ any
namespace without the barrier of a dot or subclassing or anything -
using somehow by random the same name already joins!? A threat for good
modularization of code. Not far from free flat memory programming
Don't know how they keep bigger projects managable in this language.

Uh? I don't see what you mean -- in Ruby, an assignment can be clearly
situated regarding what namespace it affects. The only example of
"using somehow by random the same name" I can think of is within a
block, where (e.g.) 'a=2' sets a new block-local name _unless_ 'a' was
previously used within the lexically enclosing method, in which case it
resets said method's 'a', but while unpleasant that's a fairly localized
problem. Maybe you can give some examples?


Alex

 

[Alex Martelli]

Using global variables in Python often raises chaos. Other languages use
a clear prefix for globals.

Ruby does ($ means global), but, what other languages? Perl, C, C++,
Java (taking a class's statics as Java's equivalent of other languages'
globals), etc, etc, all use the same lexical form for identifiers
whether local or global, disambiguating in other ways, not by "a clear
prefix" ('my' in Perl to declare locals, sort of like 'global' in Python
to declare globals, just with the default the other way 'round, etc).
Can you please explain what you mean here?

Anyway, I'd LOVE expunging the hated 'global' in favour of an explicit
namespace (and another one for "free variables in containing functions"
for closures), but while this gets often proposed in python-dev, Guido
is apparently never going to approve such a change (even in Py3k where
backwards compatibility could be broken), so I've given up on it.


Alex

 

[Duncan Booth]

Good C++ code uses m_ , g_ for members, globals, ...

.... for some definition of 'Good'.

 

[Alex Martelli]

...
I though first of Ruby.

Good C++ code uses m_ , g_ for members, globals, ... e.g. the

_Some_ C++ code uses such guidelines, but I think you're assuming way
too much by implying that (all) good C++ code does that.

For example, the Google style guide (and I assure you we have a *LOT* of
C++ code, and it *IS* pretty good indeed) uses a different convention
(not a prefix) for instance variables, and no 'stropping' distinction
between locals and globals (even though I'm more into the Python side of
things, than the C++ side, I do work regularly at Google in both
languages, I passed my "readability certification" for C++ as well as
Python, proving I know and apply the Google style rules very well, and
I'm very aware of the long exhaustive debates on which each language's
style guide is based and by which it keeps getting tweaked when needed).

Remember, for example, that part of what defines "good" C++ style (in a
firm developing lots of software in both C++ and Python) is the ease of
using SWIG (or other tool of choice, but Google prefers SWIG) to wrap
the C++ code so Python can use it too.

Anyway, guess we should move this particular discussion to a C++ forum,
and similarly for the Java one, the Perl one, and so forth. Let's leave
it at this: Ruby's the only language that mandates stropping for globals
(in any language you may choose to adopt such _conventions_, just like
you could choose, e.g., Hungarian notation -- silly, but possible;-).

Microsoft MFC code ( you have to declare the vars anyway - and it is

I've worked FAR too much with Microsoft's MFC (back when I was a Windows
and C++ guru, in the '90s) to accept their being claimed as an example
of "good C++ code" as anything but not-thinly-veiled sarcasm!

MS itself doesn't like the MFC any more, indeed they first tried
replacing it with the WTL before (essentially) giving up on C++ (the
main designer of C# was the same guy who earlier designed WTL;-) and
opensourcing WTL (IMHO still the best way to do Windows GUI in C++, if
you're condemned to such a worse-than-death task;-).

compiler checked unlike in dict-languages ), :: for global namespace.

"Per-module-global", though, is the unnamed namespace (used to be
'static', but that's deprecated at global scope nowadays)

I think its good to leave the default global binding (that is probably
whats Guido doesn't want to give up for good reason)

Then, on the principle that there should be preferably only one obvious
way to do something, we'll never have an _global object as you propose
(even by any other and much better name;-), because it systematically
produces TWO equally obvious or unobvious way to access globals.

Yet frames could expose their top-dict anyway as a virtual inspection
object (__l/__l_<funcname> or __f_stack[-2]) . Or at least writing to
locals() could be defined as writing to the top-frame-locals.
Sub-Frames could write free there (its a rare & conscious task anyway)
and the Frame itself could read either way.

Not sure I entirely understand what you're proposing, but locals access
must remain compile-time-optimized for crucial practical reasons, so
"writing to locals()" just will not work right.

Another ugly possibilties maybe by enriching variable assignment.

Maybe '=' can be enriched like .= ..= := =: ::= =:: xxx-set ... etc.
in order to be an expression with result value and to allow to writing
to certain namspaces.

Ugly enough that Guido has never wanted to entertain such possibilities
each of the many, many times they've been proposed on python-dev over
the years. python-dev is archived and easily searchable -- maybe you
can research the last few years of discussion there, assuming you're at
all interested in seeing your proposals implemented in Python.


Alex

 

[robert]

...


Then, on the principle that there should be preferably only one obvious
way to do something, we'll never have an _global object as you propose
(even by any other and much better name;-), because it systematically
produces TWO equally obvious or unobvious way to access globals.

a path to the globals whould maybe not much different from an alternate
__dict__ or globals() access. and see below ..

Yet frames could expose their top-dict anyway as a virtual inspection
object (__l/__l_<funcname> or __f_stack[-2]) . Or at least writing to
locals() could be defined as writing to the top-frame-locals.
Sub-Frames could write free there (its a rare & conscious task anyway)
and the Frame itself could read either way.

Not sure I entirely understand what you're proposing, but locals access
must remain compile-time-optimized for crucial practical reasons, so
"writing to locals()" just will not work right.

Why will a definite write to _certain_ top local dict consume any extra
time?
The target can still be directed fix at compile time. See below.

Ugly enough that Guido has never wanted to entertain such possibilities

Maybe a syntax could be applied which is already known for float
litterals and which seems to be intuitive and compatible to me:

a=1
..a=1 # same
d=5
def f()
b=1
.b=1 # same
print a
print ..a # same
..a += 1 # compile-time known: its the global!
def g():
..b=2 # f-local b !
..c=3 # write a 'free local' into f-locals
..d=4
...a += 1 # the compiler knows: its global!
g()
print b # local
print 1 + .c # ok
print c # error!, but thats acceptable
print d # global
print .d # local!, but thats acceptable


This syntax shows some kind of self-similarity:
* a dot . always references a definite dict/namespace/obj (or a float)
* a free dot . simply references the top local dictionary (or a float)
* more dots step back through the frame stack
* If no dot is there the usual "automatic name space" is assumed

No old code would be broken.

As far as I see this would not slow down the interpreter at run time
(except maybe for the new "..b=" intermediate local write's)

Robert

 

[Alex Martelli]

Why will a definite write to _certain_ top local dict consume any extra
time?

Writing to a dict intrinsically takes longer than writing to a fixed
offset into a vector. locals(), as a dict, is in fact built as a shim
on top of the fixed vector where a function's locals are kept. We're
talking of a difference in timing of over a factor of two, over a
microsecond per local-variable access on my laptop -- the average
function does so much locals-accessing that I would expect it to slow
down proportionally, by a factor of two or so, if locals were actually
kept in a dictionary rather than in a vector.

Try it out by running the same code as a module's main body versus
running it as a function -- see how much the latter approach speeds
things up. Indeed, "put the code inside a function" is the first
optimization tip to give to any newbie, exactly because a module's
top-level code actually does read/write a dict for each variable access,
while a function's body code does not.

The target can still be directed fix at compile time. See below.

None of the examples you give below exemplifies "writing to
locals() could be defined as writing to the top-frame-locals" as you had
proposed in the snippet I was responding to.

Maybe a syntax could be applied which is already known for float
litterals and which seems to be intuitive and compatible to me:

a=1
.a=1 # same

Just about exactly this syntax was recently discussed on python-dev and
Guido (and many others) shot it down in no time. Please do take the tiny
trouble to research the python-dev archives, as I already suggested,
before proposing something that's so recently been roundly rejected.

As far as I see this would not slow down the interpreter at run time
(except maybe for the new "..b=" intermediate local write's)

Each local access that goes to a dict instead of a fixed vector will
intrinsically and inevitably slow down by 2-3 times. And the fact that
sometimes accessing a local c is the same as accessing .c, and sometimes
it isn't, is something I'm *EXTREMELY* happy to not have to teach,
explain, and justify.


Alex

 

[robert]

Writing to a dict intrinsically takes longer than writing to a fixed
offset into a vector. locals(), as a dict, is in fact built as a shim
on top of the fixed vector where a function's locals are kept. We're
talking of a difference in timing of over a factor of two, over a
microsecond per local-variable access on my laptop -- the average
function does so much locals-accessing that I would expect it to slow
down proportionally, by a factor of two or so, if locals were actually
kept in a dictionary rather than in a vector.

Try it out by running the same code as a module's main body versus
running it as a function -- see how much the latter approach speeds
things up. Indeed, "put the code inside a function" is the first
optimization tip to give to any newbie, exactly because a module's
top-level code actually does read/write a dict for each variable access,
while a function's body code does not.

looking at typical code, I'd guess 50% of the variable reads and 75% of
writes are local.

testing a number crunch computation with locals only, I got down from

1.42501854286 seconds to 1.04999126984 seconds (=> 75% )

Thus I guess, typical functions would loose 15%..20% of their speed,
Methods 10%, by using dicts only. (Dict-creation costs?) byte-code size
would increase maybe 30%.

None of the examples you give below exemplifies "writing to
locals() could be defined as writing to the top-frame-locals" as you had
proposed in the snippet I was responding to.



Just about exactly this syntax was recently discussed on python-dev and
Guido (and many others) shot it down in no time. Please do take the tiny
trouble to research the python-dev archives, as I already suggested,
before proposing something that's so recently been roundly rejected.



Each local access that goes to a dict instead of a fixed vector will
intrinsically and inevitably slow down by 2-3 times. And the fact that
sometimes accessing a local c is the same as accessing .c, and sometimes
it isn't, is something I'm *EXTREMELY* happy to not have to teach,
explain, and justify.

The compiler whould know which variables go where.
The mechanism and a restrictive discrimination worse than the suggestion
here is maybe already in current Python:
.... exec "a=2"
.... print a
.... .... exec "a=2"
.... print a
.... def g(): print a
Traceback (SyntaxError: unqualified exec is not allowed in function 'f'

-------

That is: self-similarity breaks more and more when it is not accepted.

The frame-syntax and free dict suggestion is maybe more in line with
original dict-centric Python.

( A potential dual a / .a access is nothing else than 0.1 == .1 )

The speedy vector (if necessary; the 30% memory are maybe more
important) could still be preserved for the majority of code. Free dict
optional/even in addition. The compiler knows perfectly beforehand, when
a free dict has to be added in rare cases.

Even a nested "..b" access from inner frame can go to and even trigger
the vector binding in the outer. thus a one-dot-is-'same' ".b" syntax is
not even necessary as I see now (only the compiler will become little
more complex). ".b" could alternatively immediately go down the
hierarchy. And with 'exec' and danger the vector could be replaced with
dict silently by the compiler also for access from inner frames.

Even the pseudo locals()-dict (better would be an object or .) could
direct write's to local vector/dict (as it knows the name-to-index
translation because it can read anyway).

The fact is:
* Python has that big problem with unnecessary barriers for nested frame
access - especially painfull with callback functions where you want to
put back data into the calling frame. That is a unnecessary disgrace
compared to Ruby "blocks". With frame-space syntax Python nesteds could
be even more clear&practical than blocks
* That need for a modern atomic ad-hoc route to globals.
* 'exec' breaks with SyntaxError
* self-inspection for the otherwise wonderful dict-language breaks
unnecessarily

That could be most probably all healed in one rush with grace so and
without any downwards compatibility issues and speed costs (except for
the compiler part). The dict graile could be kept and guarantee.

( I don't know if all that has already been discussed to the end 'in no
time', but I'd search longer for the right search terms to start
searching )

Robert

 

[Marc 'BlackJack' Rintsch]

The fact is:
* Python has that big problem with unnecessary barriers for nested frame
access - especially painfull with callback functions where you want to
put back data into the calling frame.

You mean accessing the locals in the function that invoked the callback!?
That sounds ugly. Makes it hard to decouple the caller and the callee
here.

Ciao,
Marc 'BlackJack' Rintsch

 

[robert]

You mean accessing the locals in the function that invoked the callback!?
That sounds ugly. Makes it hard to decouple the caller and the callee
here.

That is a frequent need. For read's its anyway wired up. E.g. callbacks:

def f():
a=1
def g(var):
print a # automatic
.lastvar=var # binds in outer frame
run_w_callback(1,2,g)
print lastvar


Ruby blocks for example do that regularly.

Robert

 

[Marc 'BlackJack' Rintsch]

That is a frequent need. For read's its anyway wired up. E.g. callbacks:

def f():
a=1
def g(var):
print a # automatic
.lastvar=var # binds in outer frame
run_w_callback(1,2,g)
print lastvar


Ruby blocks for example do that regularly.

I think you're trying to write Ruby programs in Python here. Just define
a callable object as callback::

class G:
def __init__(self, a):
self.a = a
self.lastvar = None

def __call__(self, var):
print self.a
self.lastvar = var

def f2():
g = G(1)
run_w_callback(1, 2, g)
print g.lastvar

Ciao,
Marc 'BlackJack' Rintsch