Statement evals as False in my IDE and True elsewhere

[CM]

This is puzzling. (Using Python 2.5, WinXP, Boa Constructor 0.6.1 definitely running the code through Python 2.5)

If I run these lines in my program, through my IDE (Boa Constructor),

fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']
fake_result = not all(i == '[omitted]' for i in fake_data)
print 'This is fake result: ', fake_result

I get this result:
This is fake result: False

BUT, if I run those *exact same lines* (copied and pasted) in the Python 2.5 shell within Boa Constructor, or with IDLE with Python 2.5, I get:
This is fake result: True

....which is what it seems like it should evaluate to, right? What the heck is going on? How is this even possible? There is nothing that I know of in my code to cause this change, but perhaps there is. Otherwise I am at a total loss.

Thanks,
Che M

 

[Peter Otten]

This is puzzling. (Using Python 2.5, WinXP, Boa Constructor 0.6.1
definitely running the code through Python 2.5)

If I run these lines in my program, through my IDE (Boa Constructor),

fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']
fake_result = not all(i == '[omitted]' for i in fake_data)
print 'This is fake result: ', fake_result

I get this result:
This is fake result: False

BUT, if I run those *exact same lines* (copied and pasted) in the Python
2.5 shell within Boa Constructor, or with IDLE with Python 2.5, I get:
This is fake result: True

...which is what it seems like it should evaluate to, right? What the
heck is going on? How is this even possible? There is nothing that I
know of in my code to cause this change, but perhaps there is. Otherwise
I am at a total loss.
Hint:

.... fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']
.... fake_result = not all(i == '[omitted]' for i in fake_data)
.... print 'This is fake result: ', fake_result
....This is fake result: False

 

[Chris Angelico]

fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']
fake_result = not all(i == '[omitted]' for i in fake_data)
print 'This is fake result: ', fake_result

Trying to get my head around this. You want to see if all the values
in fake_data are '[omitted]' or not? That is to say, if there's
anything that isn't '[omitted]'? Not sure that that's a normal thing
to be asking, but that's what your code appears to do.

What happens if you try this?

fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']
fake_result = set(fake_data)>{'[omitted]'}

In theory, that should do the exact same thing as your code (returning
True if there's anything in fake_data that is not '[omitted]').

The other thing to try is peppering your code with print statements.
Divide the work up into pieces - show the entire loop and what happens
- print out everything you can imagine. See where the difference
begins between inside and outside the IDE. Once you find that, you'll
have a clue as to what's wrong.

ChrisA

 

[CM]

Hint:
... fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']
... fake_result = not all(i == '[omitted]' for i in fake_data)
... print 'This is fake result: ', fake_result
This is fake result: True
This is fake result: False

That's brilliant, thanks! Now I'm just a bit unsure of what to do about it.. First, I don't actually have the line "from numpy import all" in that module's code, although I have imports of numpy; I think it is getting brought in through Matplotlib's pylab module, which I do import in that module.

In any case, what's the most Pythonic way to deal with this? My first thought was to create the old all function and rename it so there would be no conflict:

(both of what follows taken from answers here: http://stackoverflow.com/questions/18774388/re-import-aliased-shadowed-python-built-in-methods)

builtin_all = __builtins__.all

but I got the error:

AttributeError: 'dict' object has no attribute 'all'

So I wound up doing this:

from __builtin__ import *

which fixed the problem...but seems less than optimal, because what if I wanted to have numpy's all still in play?

Again, thanks for restoring my faith in causality,
Che M

 

[Chris Angelico]

builtin_all = __builtins__.all

but I got the error:

AttributeError: 'dict' object has no attribute 'all'

Try using square brackets notation instead. Apparently your
__builtins__ is a dictionary, not a module, though I don't know why
(probably something to do with numpy, which I've never actually used).
But try this:

builtin_all = __builtins__["all"]

It might work.

ChrisA

 

[CM]

fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']

fake_result = not all(i == '[omitted]' for i in fake_data)

print 'This is fake result: ', fake_result

Trying to get my head around this. You want to see if all the values
in fake_data are '[omitted]' or not? That is to say, if there's
anything that isn't '[omitted]'? Not sure that that's a normal thing
to be asking, but that's what your code appears to do.

That's what I want, yes. It probably sure isn't a normal thing to be asking, and I wouldn't be surprised if I am approaching it the wrong way. Essentially, if ALL the items in that list are '[omitted]', I must not process the list, but if even one of them is something other than '[omitted]', I need to process it.

If there is a more Pythonic / better way to approach that, I'd like to knowit.

In theory, that should do the exact same thing as your code (returning
True if there's anything in fake_data that is not '[omitted]').

Yes, as you saw and as Peter showed, that the builtin all was shadowed by numpy's all. I wouldn't have thought of that, but it makes sense now. Thesesorts of shadowing problems are so rare for me that I never think about that possibility.

 

[CM]

Try using square brackets notation instead. Apparently your

__builtins__ is a dictionary, not a module, though I don't know why
(probably something to do with numpy, which I've never actually used).

But try this:
builtin_all = __builtins__["all"]

It might work.

Yes, it does. Thanks!

Che M

 

[Peter Otten]

Hint:

def demo():

... fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']
... fake_result = not all(i == '[omitted]' for i in fake_data)
... print 'This is fake result: ', fake_result

demo()

This is fake result: True

from numpy import all
demo()

This is fake result: False

That's brilliant, thanks! Now I'm just a bit unsure of what to do about
it. First, I don't actually have the line "from numpy import all" in that
module's code, although I have imports of numpy; I think it is getting
brought in through Matplotlib's pylab module, which I do import in that
module.

In any case, what's the most Pythonic way to deal with this? My first
thought was to create the old all function and rename it so there would be
no conflict:

(both of what follows taken from answers here:
http://stackoverflow.com/questions/18774388/re-import-aliased-shadowed- python-built-in-methods)

builtin_all = __builtins__.all

but I got the error:

AttributeError: 'dict' object has no attribute 'all'

So I wound up doing this:

from __builtin__ import *

which fixed the problem...but seems less than optimal, because what if I
wanted to have numpy's all still in play?

import numpy # you can now use numpy's all as numpy.all(...)
del all # remove numpy's all from your module's global namespace and
# thus make the builtin visible again

 

[Chris Angelico]

Essentially, if ALL the items in that list are '[omitted]', I must not process the list, but if even one of them is something other than '[omitted]', I need to process it.

Okay. The set example that I gave will work, then - as long as all
items are hashable (if they're strings, they are). Up to you which
one's more readable.

In any case, that could do with a supporting comment. My first
suspicion was that it ought to be written as:

fake_result = '[omitted]' not in fake_data

and that it was calculating the wrong thing. But it is doing what you
think it is.

ChrisA

 

[Terry Reedy]

Try using square brackets notation instead. Apparently your
__builtins__ is a dictionary, not a module, though I don't know why

For technical reasons Guido once explained and I have fogotten, it
depends on whether you are in main module or an imported module -- and
maybe the Python version.

 

[Chris Angelico]

For technical reasons Guido once explained and I have fogotten, it depends
on whether you are in main module or an imported module -- and maybe the
Python version.

Ah, okay. Anyway, the error message makes it clear. I love clear error messages.

ChrisA

 

[Terry Reedy]

fake_data = ['n/a', 'n/a', 'n/a', 'n/a', '[omitted]', '12']

fake_result = not all(i == '[omitted]' for i in fake_data)

Trying to get my head around this. You want to see if all the values
in fake_data are '[omitted]' or not? That is to say, if there's
anything that isn't '[omitted]'? Not sure that that's a normal thing
to be asking, but that's what your code appears to do.

That's what I want, yes. It probably sure isn't a normal thing to be asking, and I wouldn't be surprised if I am approaching it the wrong way. Essentially, if ALL the items in that list are '[omitted]', I must not process the list, but if even one of them is something other than '[omitted]', I need to process it.

If there is a more Pythonic / better way to approach that, I'd like to know it.

not all(x) == any(not x), so...

any(i != '[omitted]' for i in fake_data)

While nothing you import should *ever* mask a builtin, this would also
solve the all problem