PyMyth: Global variables are evil... WRONG!

[Rick Johnson]

PyMyth: Global variables are evil... WRONG!

============================================================
Python's Global Hysteria:
============================================================
How many times have your heard or read the phrase: "Global
variables are evil"? Well if you've been a member of the
Python community, I'll bet you've heard or read it more than
most other community members.

In this thread, i want to get to the bottom of this whole
"global-phobia" thing once and for all, and hopefully help
you folks understand that globals are not all that bad --
when DESIGNED and USED correctly that is!

============================================================
The denial of the 99%:
============================================================
Python has globals, but we just can't admit it!

The creators thought they would prevent the little
"PyPeople" from hurting themselves by "handicapping" our
globals in the form of "module level" globals.

But even the "module level" globals can be accessed
"globally" if the module they are contained in is imported
everywhere.

import glomod
glomod.var # Access
global.var = newValue

Which leads me to the conclusion that the designers were
either naive or underestimated the chutzpah of this fine
community.

============================================================
The Power of the 1%:
============================================================
Most of this "fear of globals" can be traced back to a
minority of naive coders who used globals haphazardly and
were hurt by them, <sarcasm>and now we ALL have to suffer
because future idiots need to be protected from themselves!
</sarcasm>

Too many times I've seen idiotic uses for globals.

One more atrocious example was "dialog return values" stored
as globals so the "dumb coder" could eaisly restore the
previous state of the dialog values between successive
calls.

Obviously the best solution would be for the dialog object
ITSELF to store (and restore) the values when necessary, but
alas, here i am again expecting people to create interfaces
that are consistent and logical. *rolls-eyes*

Of course we can only blame the dialog designer and not the
caller for this design flaw, but the caller still has no
excuse for globally storaging these values.

If the caller cannot repair the mistake by extending the
dialog object himself, then he can store the values within
the current module or class level scope. There is no excuse
for elevating this information to global scope.

However, there ARE justified uses of global variables!

============================================================
Justifying Global Variables:
============================================================
Globals are justified when they are used to communicate
information between scopes that otherwise were meant to be
mutually exclusive. One good example would be package sub-
modules.

Now. Some might believe that the purity of namespaces
becomes tainted if they can be violently penetrated by
global variables. But again, they are falling back on gut
reactions fostered from years of brain washing. Yes, globals
can be bad (if misused), but for the most part they are
crucial to transferring information CLEANLY between isolated
namespaces

But even globals themselves are not good enough UNLESS we
engineer a "global hierarchy"!

============================================================
From Flat to Nested == From Tricked to Bested:
============================================================
One of the shortcomings of the current implementation of
global variables is their inherent "flat" access nature.
There is no hierarchy of globals, and once all the good
names are taken, your screwed!

"But Rick, even when we use globals, we don't need that many"

Only if you consider the single package that represents your
program, but what about the thousands of support libraries
with millions of lines of code that work in the background
to make your measly few thousand lines of code work? What
about all the globals that they have injected?

"Never thought of that really"

Throwing out global names UNADORNED is folly. Could you
imagine telephone numbers without area codes?

Hmm: Am i calling "Jim" in Beverly Hills California???

california.beverlyhills.jim

or "Jim" in "Spokane Washington"???

washington.spokane.jim

You see, you need to QUALIFY these names not only to make
then unique, but also so the caller will understand who he
is calling.

Same goes for globals.

 

[Tim Daneliuk]

Globals are justified when they are used to communicate
information between scopes that otherwise were meant to be
mutually exclusive.

I think this is certainly the use case most people would suggest.

But I think you may have missed the real reason most modern
designers object to inter-module globals: The presence of
such entities almost always means the code has been designed
badly. Whether we're writing pristine OO code (cough, cough)
or more traditional procedural stuff, information hiding
is a good thing. When and where there is a need for modules
(or programs, or machines, or "clouds", or interstellar space ...)
to share information, my view is this is better done via some sort
of interface/message passing mechanism. The more "things" that have
to be shared across modules/programs/machines/clouds/space ... the more
*brittle* the end system becomes.

This problem is broader than just variables, BTW. It exists anytime you
share *anything* across many computational components, especially in highly
parallel and/or distributed systems. For example, RPCs tend to be lousy
at building high scale distributed system because they usually have
a call-and-wait semantic which will fail if the caller never responds.
In this case the system is sharing the semantics of *time*. As a general
matter, exactly-once asynchronous messaging tends to be a better implementation
model here.

Just my 2 cents worth. Interesting read. Thanks for taking the time

 

[Rick Johnson]

I think this is certainly the use case most people would
suggest. But I think you may have missed the real reason
most modern designers object to inter-module globals: The
presence of such entities almost always means the code has
been designed badly. Whether we're writing pristine OO
code (cough, cough) or more traditional procedural stuff,
information hiding is a good thing.

Yes, and i agree. But you cannot "hide" everything. There
will always be a need to share information.

When and where there is a need for modules (or programs,
or machines, or "clouds", or interstellar space ...) to
share information, my view is this is better done via some
sort of interface/message passing mechanism.

But python modules can't be interfaces because interfaces
should protect internal data, prevent external forces from
meddling with internal state (EXCEPT via the rules of a
predefined "contract"), hide dirty details from the caller,
and have clearly defined access points.

BUT PYTHON MODULES DON'T FOLLOW THIS DESIGN PATTERN!

No, Python modules can be poked, prodded, and violated by
any pervert who can spell the word "import".

Attribute values can be reassigned and state can be
externally manipulated resulting in all types of undefined
behaviors -- that does not sound like an interface to me.

So if python modules are importable everywhere, and mutable
from everywhere, then python modules are merely addresses to
a collection of global variables? And they're only
interfaces "superficially"?

So that leaves us with Python's current implementation of
unofficial "global" variables implemented as "puesdo-
interfaces" by module objects that are victims waiting to be
violated.

Interesting.

IF IT WALKS LIKE A GLOBAL DUCK AND...

 

[Chris Angelico]

But python modules can't be interfaces because interfaces
should protect internal data, prevent external forces from
meddling with internal state (EXCEPT via the rules of a
predefined "contract"), hide dirty details from the caller,
and have clearly defined access points.

BUT PYTHON MODULES DON'T FOLLOW THIS DESIGN PATTERN!

No, Python modules can be poked, prodded, and violated by
any pervert who can spell the word "import".

And C++ objects can be poked by anyone who can do a pointer cast. And
Java objects by anyone who notices that 'const' checks don't apply to
byte-code. In fact, the only language I can think of that actually
"prevent external forces from meddling with internal state" is
HQ9++, which follows the very best principles of data hiding.

ChrisA

 

[Ricardo Aráoz]

El 12/11/13 01:46, Rick Johnson escribió:

No, Python modules can be poked, prodded, and violated by any pervert
who can spell the word "import". Attribute values can be reassigned
and state can be externally manipulated resulting in all types of
undefined behaviors --

Nice!
My code, my responsibility, my business.... not yours.

that does not sound like an interface to me. So if python modules are
importable everywhere, and mutable from everywhere, then python
modules are merely addresses to a collection of global variables? And
they're only interfaces "superficially"? So that leaves us with
Python's current implementation of unofficial "global" variables
implemented as "puesdo- interfaces" by module objects that are victims
waiting to be violated. Interesting. IF IT WALKS LIKE A GLOBAL DUCK
AND...

Nice choice of words, "violated", "victims".... Really conductive to a
balanced unprejudiced thought process.

 

[Tim Chase]

Yes, and i agree. But you cannot "hide" everything. There
will always be a need to share information.

You may not be able to (or want to) hide everything, but sharing
should at least happen over defined protocols (functions/methods).
Otherwise, you wander off into the weeds of hackery and
monkey-patching which makes maintaining the code a lot hairier.

-tkc

 

[jongiddy]

============================================================

Justifying Global Variables:

============================================================

Globals are justified when they are used to communicate

information between scopes that otherwise were meant to be

mutually exclusive. One good example would be package sub-

modules.

Can you please give an example where having a module provide a global variable would work better than any of:
1. providing a module function to change the operation of my module
2. providing a class with a method to change the operation of an instance
3. providing an additional parameter to module functions / instance methods to change operation
4. providing additional module functions / instance methods to perform different operations

 

[Rick Johnson]

Can you please give an example where having a module
provide a global variable would work better than any of:
[snip]

Well my point is that the attributes of any Python module
are "emulating" globals already. The fact that we have to
mutate them via a "psuedo" interface makes us "falsely"
believe we are using an interface -- but we aren't!

PYTHON MADE ACCESSING GLOBAL VARIABLES MORE DIFFICULT!

As example. I could import the math module and start
fiddling with attributes. Let's say for example i can change
the value of math.pi. Most people would probably say "what's
the problem with that?", here, let me show you.

# mod1.py
print "mod1.py"
import math
math.pi = "tasty"

# mod2.py
print "mod2.py"
import math
print ' value of math.pi is:', math.pi
radius = 10
math.pi * radius

#modmain.py
import mod1
import mod2
print dir(mod1)
print dir(mod2)

When you run "modmain.py" you will see that not only have we
changed the global variable "pi" to a string, but thanks to
a dumb overloading of the multiply operator, python will
happily give use the wrong answer -- but that's another
problem, let's stay focused on the "global" problem for now!

When i type "math.pi", i "feel" as though i'm accessing an
interface, BUT I'M NOT! If i was indeed accessing a TRUE
interface, the interface would have thrown a Type error for
attempting to assign a value of type string to what should
only be a float. The interface would have protected it's
internal data from corruption, but it didn't.

BECAUSE IT'S NOT AN INTERFACE!

It's just a dumb namespace with no exposed hooks to control
it's underlying behavior.

MY POINT IS:

Python designers act like globals are SO evil, but then they
give us modules which are containers for global variables,
and now the globals contained within those modules are
wolves in sheep's clothing. Are they just trying to fool
themselves, or fool us?

FOOD FOR THOUGHT:

What's worse? A wolf? Or a wolf in sheep's clothing?

 

[Chris Angelico]

As an analogy music has may general rules that musicians are wise to
follow.
Some pieces of music that break these rules are great because they have
broken the rules but most are not. those that are great are great because
the musician in question understands the reasons for the rules & how
breaking them will affect the end product, the ones that are bad are
because the musician does not know enough about music to even know the
rule exists.

Agreed. Pieces of music that violate the rules arbitrarily are just
annoying to try to play (and nearly impossible for the church to
follow), but when you find those really brilliant pieces that do
something special (Sir Arthur Sullivan, I'm looking at you -
especially having just been reading the booklet that came with the new
Beauty Stone recording), it's magnificent.

Global variables basically don't exist in Python. You have per-module
state. And if you start monkey-patching, you're fiddling with someone
else's module. It's all fun and games till someone loses a function...

sys.stdout.write=None

ChrisA

 

[jongiddy]

When i type "math.pi", i "feel" as though i'm accessing an

interface, BUT I'M NOT!

I'm not sure where you get the feeling you're accessing an interface. If I typed this, I would feel like I was trying to change a fundamental constant..

You have just demonstrated that going into other modules and changing theirattributes (whether they are variables, functions or classes) is generallya BAD idea, and I don't think I've ever seen anyone recommend doing this, except possibly as a workaround or for debugging purposes.

On the other hand, you initially suggested that modifying globals containedwithin namespaces (i.e. modules) is a good way to communicate between modules. That is, you suggested in your initial post that going into other modules and changing their attributes is a GOOD idea.

This inconsistency is why I was asking for a good example (i.e. a realisticexample where the use of global variables provides the best solution).

Just because a tool allows you to do something does not make it a good idea.. Try this paraphrase of your last post: Ladder designers act like standingon the top rung is SO evil, but then they give us ladders with a top rung,Are they just trying to fool themselves, or fool us?

 

[Rick Johnson]

I'm not sure where you get the feeling you're accessing an
interface.

Because the constant PI should never change. Sure we can
argue about granularity of PI, but that argument has no
weight on the fact that PI should be a constant.

By placing PI in the module "math", we are creating a pseudo
interface. We (the creators) are "assuming" that PI will be a
constant and never change, and the caller is assuming that
pi will remain static, but not only can it be mutated, it
can be mutated globally.

math.pi is neither a constant nor a local module attribute,
math.pi is really a global variable. This is true not only
for math.pi, but ALL attributes of ALL python modules.

If I typed this, I would feel like I was trying
to change a fundamental constant. You have just
demonstrated that going into other modules and changing
their attributes (whether they are variables, functions or
classes) is generally a BAD idea, and I don't think I've
ever seen anyone recommend doing this, except possibly as
a workaround or for debugging purposes. On the other hand,
you initially suggested that modifying globals contained
within namespaces (i.e. modules) is a good way to
communicate between modules.

You missed my point: only IF python modules were TRUE
interfaces. It's not a good idea under python's current
implementation of modules. My thread is intended to
underscore this very design flaw.

This inconsistency

There is no inconsistency in my logic, quite the contrary.

I think you're having difficulty following along because
you've been brainwashed by Python for too long. You've
falsely believed that Python does not have globals, but
it does! You just have to mutate then via a pseudo
interface.

is why I was asking for a good example (i.e.
a realistic example where the use of global variables
provides the best solution).

I gave a good example in my very first post:

RR: "Globals are justified when they are used to
[share] information between scopes that otherwise
were meant to be mutually exclusive. One good
example would be package sub-modules."

Just because a tool allows you to do something does not
make it a good idea. Try this paraphrase of your last
post: Ladder designers act like standing on the top rung
is SO evil, but then they give us ladders with a top rung,
Are they just trying to fool themselves, or fool us?

EXACTLY. And whilst your comparison is both intelligent and
funny, you're just re-iterating my point!

We have all been brainwashed by authorities. First they
give us rules, then they give us the power to break
those rules. If standing on the top rung is SO
dangerous, then don't manufacture your ladders with top
rungs. Else, shut up about it!

"THOU SHALT NOT KILL"

Well i can't live WITH the B!@$%!!!

"THOU SHALT NOT COMMIT ADULTERY"

Then why create me with a unquenchable desire for sex?

"THOU SHALT NOT TAKE THE LORDS NAME IN VAIN"

GOD DAMMIT! Stop making all these opposing rules.

So the designers detest globals huh? But python module
attributes are really global because they can be accessed
and mutated ANYWHERE!". This enigma leads to two logical
design philosophies: (anything else is folly)

1. Accept that globals are useful, and make them
available through a "real" global syntax, not
some attribute of a module that "appears" to be
local, but in reality is global. Then prevent
external mutation of module attributes directly,
and instead, require that mutation must follow a
contract defined by internal module "setter"
functions.

2. Remain convinced that global mutation is evil
and prevent mutation of all python module
attributes. You can import modules, but you can't
mutate their contents.

THIS IS HOW YOU DESIGN FOR CONSISTENCY, NOT HYPOCRISY!

@LanguageDesigners: But whatever you do, don't hide globals
behind pseudo interfaces, because then you have created
something worse than a global; you've created a global in
sheep's clothing. And you're a hypocrite.

 

[Rick Johnson]

On Tuesday, November 12, 2013 11:00:37 AM UTC-6, Rick Johnson wrote:
[snip]

We have all been brainwashed by authorities. First they
give us rules, then they give us the power to break
those rules.

The devil himself said it best:

Hmm. How do we humans cope with all these opposing rules? If
we created an AI with all the ridiculous, convoluted and
opposing rules that we humans live with on a daily basis, it
would blow chunks.

How do our minds possibly cope with such illogical rules
without blowing chunks?

We cope by re-inventing reality. We cope by convincing
ourselves that truths are not true and lies are true.

We cope by designing languages that obfuscate globals behind
pseudo interfaces so we can get farm fuzzy feelings in our
tummy, then we can "secretly" break the rule when non-one is
looking.

By doing this we convince ourselves that we are "pure".

HUMANS: What a pathetic bunch of slaves!

 

[Tim Chase]

Because the constant PI should never change. Sure we can
argue about granularity of PI, but that argument has no
weight on the fact that PI should be a constant.

By placing PI in the module "math", we are creating a pseudo
interface. We (the creators) are "assuming" that PI will be a
constant and never change, and the caller is assuming that
pi will remain static, but not only can it be mutated, it
can be mutated globally.

But the module-scoping of "globals" is perfectly valid:
"Tom Selleck"

As an example from the stdlib of setting globals via a protocol,
locale.setlocale() does exactly this. So I'm not sure why you have
your knickers in a knot. Module-level items can be accessed
globally (though often a bad idea, or at least you have to beware of
side-effects where other things might break), and if you don't like
the "modules are not objects", someone in this thread already showed
you that you can insert objects/classes into the sys.modules and get
full getter/setter functionality with minimal trouble.

Finally, we're all (mostly) consenting adults here. If I want to be
an idiot and

math.PI = 3.14

and suddenly stuff breaks, I get to keep all the pieces my breakage
caused.

-tkc

 

[jongiddy]

1. Accept that globals are useful, and make them

available through a "real" global syntax, not

some attribute of a module that "appears" to be

local, but in reality is global. Then prevent

external mutation of module attributes directly,

and instead, require that mutation must follow a

contract defined by internal module "setter"

functions.



2. Remain convinced that global mutation is evil

and prevent mutation of all python module

attributes. You can import modules, but you can't

mutate their contents.

From your first post, I take it you're not keen on option #2.

For #1, module globals are exactly the hierarchically namespaced globals that you desire in your first post, except they are variables, not get/set handlers (which is what I take you to mean by an "interface").

Why not create a PEP to provide handlers for module attributes? You could base it on PEP 213, which describes the same feature for classes.

As a bonus, this would trivially support option #2 (e.g. non-mutable math.pi) by raising an exception for the set operator.

 

[Terry Reedy]

No, Python modules can be poked, prodded, and violated by
any pervert who can spell the word "import".

Or by clever programmers.

Attribute values can be reassigned and state can be
externally manipulated

Perhaps for good reasons.

resulting in all types of undefined behaviors

Not necessarily. Manipulation can also eliminate undefined behaviors.

Suppose I want to test a Idle function that uses a tkinter.messagebox
class to communicate with users. Perhaps the module has

from tkinter.messagebox import askretrycancel

The behavior of askretrycancel(*args) is undefined insofar as it depends
on the arbitrary actions of a human user (it may even hang forever). In
an automated test, there is no user, and it is difficult to simulate the
eyes and fingers of one while leaving askretrycancel as it is.
Monkeypatching with a mock solves this. Simplifying a bit, and leaving
out details, I have done something like this.

from mock_tkinter import mbox # mocks all messageboxes
mock_arc = mbox()
import mod
mod.askretrycancel = mock_arc

This makes mod.askretrycancel deterministic in that I can preload a
response into mock_arc before calling the function or method in mod.
(This simulates a user's fingers.) The mock also saves the values sent
to it to see if they are what they should be. (This simulates a user's
eyes.)

 

[Rick Johnson]

From your first post, I take it you're not keen on option #2.

That is correct, i prefer to use REAL interfaces.

For #1, module globals are exactly the hierarchically
namespaced globals that you desire in your first post,
Agreed.

except they are variables, not get/set handlers (which is
what I take you to mean by an "interface").

Yes, python modules expose ALL members publicly and have no
support for contractual interfacing.

Why not create a PEP to provide handlers for module
attributes? You could base it on PEP 213, which describes
the same feature for classes.

I would love to see modules expand to something more than
mere "boxes to stuff variables". But then again, modules
are not going to provide the ease of access that globals
demand due to the import problem.

Modules will not be a good place to store globals because
modules must be imported and globals should never need
importing.

I've just released a module on Python-list called "G.py"
that does exactly what i propose. It's not perfect, but i
think you'll understand what i'm proposing after testing
it.

As a bonus, this would trivially support option #2 (e.g.
non-mutable math.pi) by raising an exception for the set
operator.

"math.pi" should be "math.PI". and PI should be a CONSTANT.
And not just a pseudo constant, but a REAL constant that
cannot be changed.

 

[Tim Chase]

"math.pi" should be "math.PI".

It's a real shame that this fails:

oh...wait...

and PI should be a CONSTANT. And not just a pseudo constant, but a
REAL constant that cannot be changed.

How much precision do you want? Perhaps you really do want do just
use because you know you only have 2-3 places of precision in your
other numbers:

math.pi = 3.14

Or maybe you want to monkey-patch in a fraction/decimal instead for
greater precision:

math.pi = decimal.Decimal('3.1415926535897932384626433832795028841971693993751058209749445923078164062862089986280348253421170679')

It comes back to Python being a language for consenting adults.

-tkc

 

[Andrew Cooper]

"math.pi" should be "math.PI". and PI should be a CONSTANT.
And not just a pseudo constant, but a REAL constant that
cannot be changed.

And what do you do when the wizards bend space-time to make PI exactly
3, for the ease of other calculations when building a sorting machine?

Does usenet start delivering these posts in the past/future?

I suspect real python programmers will be able to feel quite smug when
they can change the value of math.pi to suit the situation.

~Andrew

 

[Tim Daneliuk]

Yes, and i agree. But you cannot "hide" everything. There
will always be a need to share information.


But python modules can't be interfaces because interfaces
should protect internal data, prevent external forces from
meddling with internal state (EXCEPT via the rules of a
predefined "contract"), hide dirty details from the caller,
and have clearly defined access points.

BUT PYTHON MODULES DON'T FOLLOW THIS DESIGN PATTERN!

I think this is an unfair criticism. You can do this in
ANY language if you know how. For example, if I understand
stack frames, I can write code that fiddles with local
variables in compiled code. For that matter, if I understand
pointers at the assembler level, I can probably do the same
with globals.

Global access/visibility is a matter of convenience - it's harder
to do in a static compiled language and easier to do in a
late bound, dynamic language. The fact that this is
*possible* in Python doesn't speak to Python's fitness for
any particular purpose nor does it speak to whether globals
are a good- or bad idea. Anyone that knows enough to fiddle
with the internal implementation of a module and or subvert
global constants like math.pi, also know what they're doing
is "perverse" and have to be willing to live with the side
effects and consequences of such acts.

I cut my teeth in this business writing realtime machine control
software in assembly language (and PL/M!) where there was NO
memory protection and everything - code, data, stack - was
"global" in some sense, or at least could be made to be.
We managed to routinely write highly reliable, blazingly
fast code that didn't break, didn't bork, and did what it
was supposed to. There's no reason to believe that Python
programs cannot do the exact same thing (well, not the realtime
part) with just as much robustness and correctness as those
old asm programs.

Programming well has far less to do with the language and has
far more to do with the designer and coder...

 

[Rick Johnson]

And what do you do when the wizards bend space-time to
make PI exactly 3, for the ease of other calculations when
building a sorting machine?

Are you telling me that these wizards can't be bothered to
write the integer "3"? They have to reassign math.pi instead?

Look if you need some other granularity of PI besides
what's offered in math.PI, then do your own calculation.

The last thing you want to do is create a variable when a
constant is what you need.