Can global variable be passed into Python function?

D

Dave Angel

Grant Edwards said:
Not only is it logical, the C standard explicitly requires it. Here's
a second-hand citation since I don't happend to have an actual copy of
the standard on hand:

C1x �6.7.2.1.13:

A pointer to a structure object, suitably converted, points to
its initial member ... and vice versa. There may be unnamed
padding within a structure object, but not at its beginning.

The quote you make from the C standard doesn't mention malloc, so
you're arguing different things. It's not the compiler that casts
the malloc return value to the struct type.

C++ does implicitly convert the result, and the return value of
new already has the struct type. But the runtime stores at least
two kinds of overhead on occasion, the array size, and the
vtable. So the malloc address can not be assumed to match the
struct beginning. (Not even considering that one can override
operator new at two levels)

Glad python doesn't have any of this mess.
 
C

Chris Angelico

The quote you make from the C standard doesn't mention malloc, so
you're arguing different things. It's not the compiler that casts
the malloc return value to the struct type.

C++ does implicitly convert the result, and the return value of
new already has the struct type. But the runtime stores at least
two kinds of overhead on occasion, the array size, and the
vtable. So the malloc address can not be assumed to match the
struct beginning. (Not even considering that one can override

Whatever pointer malloc returns is the beginning of the *usable*
space. Any overhead for array size etc has to be before that; a
virtual function table pointer would be inside that space, but that's
very much compiler-dependent.

ChrisA
 
M

Marko Rauhamaa

Roy Smith said:
Python already has a switch statement. It's just spelled funny...

[...]

try:
raise value
except Case1:
print "did case 1"
except (Case2, Case3):
print "did either case 2 or 3"
else:
print "did default"

Not bad! Definitely worth considering.
No fall-through, however.

Fall-trough is an unfortunate C syntax accident, not a feature worth
emulating.


Marko
 
M

Marko Rauhamaa

Grant Edwards said:
any decent compiler should be able to generate the same code for a
switch statement and for an equivalent chained if/else.

It's not so easy to be decent, especially when it comes to a language as
dynamic as Python.


Marko
 
M

Marko Rauhamaa

Ben Finney said:
The unreliability is “will objects defined elsewhere have a different
identity?â€

That question is not interesting in my context, and has no bearing on
the correctness of the program.


Marko
 
S

Steven D'Aprano

Well, it doesn't since

False

No, that is working correctly, so the comment that equals works fine is
correct: returning False is the correct thing to do if one or both of the
objects are a NAN. NANs are supposed to compare unequal to everything,
including themselves.

The is operator and the == operator do not have the same purpose and they
do not do the same thing. "is" should not be considered an improved ==
without the quirks, this is not PHP and we're not comparing == and ===.
The argument here is not about which operator performs the check we want,
but over what check we want: do we want an identity test or an equality
test?
 
M

Marko Rauhamaa

Steven D'Aprano said:
No, that is working correctly, so the comment that equals works fine
is correct: returning False is the correct thing to do if one or both
of the objects are a NAN. NANs are supposed to compare unequal to
everything, including themselves.

Nobody is saying there's a bug in the implementation of "==". I'm just
saying "==" cannot be taken as a universal superset of "is". Therefore
a program cannot blindly use "==" to test for identity.

That's why "==" is a bit fishy. It immediately raises the question: what
does it mean for a == b, especially since the exact implementation of a
and b are intended to be opaque.

Example:

The os module defines the constants os.SEEK_SET, os.SEEK_CUR and
os.SEEK_END that can be used as arguments for os.lseek(). Must those
constants be used, or can a regular integer be used instead? The
documentation clearly states that integers can be used:

SEEK_SET or 0 to set the position relative to the beginning of the
file; SEEK_CUR or 1 to set it relative to the current position;
SEEK_END or 2 to set it relative to the end of the file.

However, on the same reference page, os.posix_fadvise() is defined. We
read:

advice is one of POSIX_FADV_NORMAL, POSIX_FADV_SEQUENTIAL,
POSIX_FADV_RANDOM, POSIX_FADV_NOREUSE, POSIX_FADV_WILLNEED or
POSIX_FADV_DONTNEED

and:

os.POSIX_FADV_NORMAL
os.POSIX_FADV_SEQUENTIAL
os.POSIX_FADV_RANDOM
os.POSIX_FADV_NOREUSE
os.POSIX_FADV_WILLNEED
os.POSIX_FADV_DONTNEED

Flags that can be used in advice in posix_fadvise()

Now, what kinds of object are those constants? We are not supposed to
know or care. We could peek into the implementation, but it would be a
grave mistake to trust the implementation choices in the application.

So in my application code I might set:

favd_flag = os.POSIX_FADV_RANDOM

in some other part of my code I might want to see how "flag" was set.
Should I use "==" or "is" to test it?

If I take the API documentation on its face value, I *must* use "==" for
os.SEEK*:

if seek_flag == os.SEEK_END:
...

and I *must* use "is" for os.POSIX_FAVD_*:

if fsavd_flag is os.POSIX_FADV_RANDOM:
...

Since, for all I know, os.POSIX_FAVD_RANDOM might return a random value
for __eq__().


Marko
 
M

Marko Rauhamaa

Marko Rauhamaa said:
If I take the API documentation on its face value, [...]
I *must* use "is" for os.POSIX_FAVD_*:

if fsavd_flag is os.POSIX_FADV_RANDOM:
...

However, since a documentation flaw is more than likely, it is even more
prudent to avoid both "==" and "is" and create a set of shadow constants
in the application code:

if self.fsavd_flag is self.FAVD_RANDOM:
os.posix_fadvice(fd, offset, len, os.POSIX_FAVD_RANDOM):


Marko
 
C

Chris Angelico

However, on the same reference page, os.posix_fadvise() is defined. We
read:

advice is one of POSIX_FADV_NORMAL, POSIX_FADV_SEQUENTIAL,
POSIX_FADV_RANDOM, POSIX_FADV_NOREUSE, POSIX_FADV_WILLNEED or
POSIX_FADV_DONTNEED

Now, what kinds of object are those constants? We are not supposed to
know or care. We could peek into the implementation, but it would be a
grave mistake to trust the implementation choices in the application.

So in my application code I might set:

favd_flag = os.POSIX_FADV_RANDOM

in some other part of my code I might want to see how "flag" was set.
Should I use "==" or "is" to test it?

In the absence of any advice to the contrary, I would use == to test.
The flags are most likely to be, in order:

* An enumeration, in a sufficiently new Python
* Integers
* Strings
* Arbitrary object()s

All of the above will compare correctly with ==, and if someone stuffs
in an object that compares equal to more than one of them, they're
likely to have problems at the far end. If identity is really crucial,
I would expect there to be a comment in the docs.

And there's another thing you can do to test.
<class 'int'>

So use ==. If it's later changed and you have to instead use 'is', you
can change your code.

ChrisA
 
M

Marko Rauhamaa

Chris Angelico said:
And there's another thing you can do to test.

<class 'int'>

Is that what you do in your programs?
So use ==. If it's later changed and you have to instead use 'is', you
can change your code.

Quite a robust API, huh?

Anyway, you recognize that the API definer is within their rights to
require 'is' semantics for the constants. IOW, the application must use
the given constant objects and nothing that happens to be == to them.

Consequently, the implementer is within their rights to define:

def __eq__():
return True


Marko
 
S

Steven D'Aprano

Nobody is saying there's a bug in the implementation of "==". I'm just
saying "==" cannot be taken as a universal superset of "is". Therefore a
program cannot blindly use "==" to test for identity.

Um, yes? Nor can you use ">" to test for identity, or "+", or "%", or any
other operator other than "is". Why do you think it is a problem that ==
doesn't test for identity? It's not supposed to test for identity.

Why do you want to test for identity? I think I've asked five times now,
why you care whether a state value has one instance or a thousand
instances, and you haven't even attempted an answer.


That's why "==" is a bit fishy. It immediately raises the question: what
does it mean for a == b, especially since the exact implementation of a
and b are intended to be opaque.

It means that a equals b. For ints, it means that they have the same
numeric value. The same applies for floats. For strings, it means that
they contain the same code points in the same order. And so on. For all
built-in types, equality is well-defined. For custom types you create
yourself, the onus is on you to ensure that equality is meaningful and
well-defined.

Example:

The os module defines the constants os.SEEK_SET, os.SEEK_CUR and
os.SEEK_END that can be used as arguments for os.lseek(). Must those
constants be used, or can a regular integer be used instead? The
documentation clearly states that integers can be used:

SEEK_SET or 0 to set the position relative to the beginning of the
file; SEEK_CUR or 1 to set it relative to the current position;
SEEK_END or 2 to set it relative to the end of the file.

However, on the same reference page, os.posix_fadvise() is defined. We
read:

advice is one of POSIX_FADV_NORMAL, POSIX_FADV_SEQUENTIAL,
POSIX_FADV_RANDOM, POSIX_FADV_NOREUSE, POSIX_FADV_WILLNEED or
POSIX_FADV_DONTNEED

and:

os.POSIX_FADV_NORMAL
os.POSIX_FADV_SEQUENTIAL
os.POSIX_FADV_RANDOM
os.POSIX_FADV_NOREUSE
os.POSIX_FADV_WILLNEED
os.POSIX_FADV_DONTNEED

Flags that can be used in advice in posix_fadvise()

Now, what kinds of object are those constants? We are not supposed to
know or care.

Incorrect. We are supposed to know and care.

os.posix is exactly the sort of library I mentioned earlier when I said
sometimes you're constrained by compatibility with some other system. In
this case, the os module is explicitly designed to be compatible with the
POSIX interface, which is defined to use certain integer values as flags.
This is not an implementation choice which implementers can change at
will, it is part of the interface.

The specific *values* possibly may be allowed to vary from platform to
platform, and since this is C even the definition of "int" may be
platform specific, but not that fact that they are ints. Hence the value
of POSIX_FADV_RANDOM could, theoretically, be different under Linux and
FreeBSD (say). It probably isn't, but it could be. If you hard-code the
magic number 1 in your code, you're risking the (tiny) chance of it
failing on some obscure POSIX system. But that doesn't imply that we must
test for object identity. There could be a million different instances,
all with the value POSIX_FADV_RANDOM.

Python does not guarantee that there is only a single 1 instance. If you
want to test whether a value is os.POSIX_FADV_RANDOM, the right way is to
compare that value for equality with os.POSIX_FADV_RANDOM, not identity.

We could peek into the implementation, but it would be a
grave mistake to trust the implementation choices in the application.
So in my application code I might set:

favd_flag = os.POSIX_FADV_RANDOM

A much better choice than hard-coding the magic value 1. But that choice
has absolutely nothing to do with whether 1 is a singleton or not.

in some other part of my code I might want to see how "flag" was set.
Should I use "==" or "is" to test it?

Equals, of course. There is absolutely no question about that. To even
*think* that you should test it with "is" means that you have completely
misunderstood what you are doing here. Why are you relying on an
implementation detail that CPython happens to cache and reuse small
integers like 1? What happens if you run your code under an
implementation of Python that doesn't cache small ints? Or if your
platform happens to set POSIX_FADV_RANDOM to a non-cached value like
8531201?

Python does not promise that POSIX_FADV_RANDOM will be a singleton value.
Using "is" is unsafe.

If I take the API documentation on its face value, I *must* use "==" for
os.SEEK*:
Correct.


if seek_flag == os.SEEK_END:
...

and I *must* use "is" for os.POSIX_FAVD_*:
Incorrect.


if fsavd_flag is os.POSIX_FADV_RANDOM:
...

Since, for all I know, os.POSIX_FAVD_RANDOM might return a random value
for __eq__().

For all *you* know, perhaps, but since os.posix_fadvise is a thin wrapper
around the POSIX C function fadvise, and that is documented as expecting
ints for the advice parameter, that cannot be the case.

Unfortunately Python has not had the money put into it to make it an ISO
standard like Java, and so there are certain areas where the behaviour is
known by common practice but not officially documented. (A bit like
British common law.) That the os module is a thin wrapper around os-
specific services may not be explicitly stated, but it is nevertheless
true.
 
M

Marko Rauhamaa

Steven D'Aprano said:
Incorrect. We are supposed to know and care.

Then, the documentation is seriously flawed. It gives no hint whatsoever
on the nature of those objects.
os.posix is exactly the sort of library I mentioned earlier when I
said sometimes you're constrained by compatibility with some other
system. In this case, the os module is explicitly designed to be
compatible with the POSIX interface, which is defined to use certain
integer values as flags. This is not an implementation choice which
implementers can change at will, it is part of the interface.

The values of those Python constants don't need to have any relationship
with those of the underlying operating system.
Python does not guarantee that there is only a single 1 instance.

Nobody has ever argued such a thing. I certainly haven't.

However, nothing in the API spec gives you the right to call the
function with an integer.
If you want to test whether a value is os.POSIX_FADV_RANDOM, the right
way is to compare that value for equality with os.POSIX_FADV_RANDOM,
not identity.

That might well be true but is not explicitly or implicitly specified in
the documentation. (The os.SEEK_* constants are explicitly defined.)
For all *you* know, perhaps, but since os.posix_fadvise is a thin
wrapper around the POSIX C function fadvise, and that is documented as
expecting ints for the advice parameter, that cannot be the case.

A pretty serious documentation flaw, then. Misleading even.

I take it from the documentation that you *must* use the given constant
objects and not some improvised integers.
That the os module is a thin wrapper around os- specific services may
not be explicitly stated, but it is nevertheless true.

The example was given for illustration purposes; any criticism against
the accuracy of the documentation is a sideshow.


Marko
 
M

Mark Lawrence

You keep vacillating between two positions: pick one for the context.

Either you care about object identity in this context, and the above
observation is relevant.

Or, you don't care about object identity in this context, and you should
avoid ‘is’ and use ‘==’.

But you cannot have both.

Unless you're a troll.
 
M

Mark Lawrence

Nobody is saying there's a bug in the implementation of "==". I'm just
saying "==" cannot be taken as a universal superset of "is". Therefore
a program cannot blindly use "==" to test for identity.

That's why "==" is a bit fishy. It immediately raises the question: what
does it mean for a == b, especially since the exact implementation of a
and b are intended to be opaque.

Example:

The os module defines the constants os.SEEK_SET, os.SEEK_CUR and
os.SEEK_END that can be used as arguments for os.lseek(). Must those
constants be used, or can a regular integer be used instead? The
documentation clearly states that integers can be used:

SEEK_SET or 0 to set the position relative to the beginning of the
file; SEEK_CUR or 1 to set it relative to the current position;
SEEK_END or 2 to set it relative to the end of the file.

However, on the same reference page, os.posix_fadvise() is defined. We
read:

advice is one of POSIX_FADV_NORMAL, POSIX_FADV_SEQUENTIAL,
POSIX_FADV_RANDOM, POSIX_FADV_NOREUSE, POSIX_FADV_WILLNEED or
POSIX_FADV_DONTNEED

and:

os.POSIX_FADV_NORMAL
os.POSIX_FADV_SEQUENTIAL
os.POSIX_FADV_RANDOM
os.POSIX_FADV_NOREUSE
os.POSIX_FADV_WILLNEED
os.POSIX_FADV_DONTNEED

Flags that can be used in advice in posix_fadvise()

Now, what kinds of object are those constants? We are not supposed to
know or care. We could peek into the implementation, but it would be a
grave mistake to trust the implementation choices in the application.

So in my application code I might set:

favd_flag = os.POSIX_FADV_RANDOM

in some other part of my code I might want to see how "flag" was set.
Should I use "==" or "is" to test it?

If I take the API documentation on its face value, I *must* use "==" for
os.SEEK*:

if seek_flag == os.SEEK_END:
...

and I *must* use "is" for os.POSIX_FAVD_*:

if fsavd_flag is os.POSIX_FADV_RANDOM:
...

Since, for all I know, os.POSIX_FAVD_RANDOM might return a random value
for __eq__().


Marko

Will you please be kind enough to stop writing this drivel. You've been
told repeatedly that you don't know what you're talking about, so have
the decency to just belt up. Now try testing these words for identity.
 
D

Dave Angel

Chris Angelico said:
Whatever pointer malloc returns is the beginning of the *usable*
space. Any overhead for array size etc has to be before that; a
virtual function table pointer would be inside that space, but that's
very much compiler-dependent.

Sure, for some definition of "usable". Overhead such as block
size, freelist pointer etc., are obviously outside of the
returned block. But the array size that's specified in a call to
new [], and the vptr, are definitely inside the malloc'ed block,
and may be before the struct data.
 
C

Chris Angelico

Sure, for some definition of "usable". Overhead such as block
size, freelist pointer etc., are obviously outside of the
returned block. But the array size that's specified in a call to
new [], and the vptr, are definitely inside the malloc'ed block,
and may be before the struct data.

Hmm. Last I was working with it, the array size to new[] was outside
the block, just as the block size to malloc(). The vptr is part of any
struct/class with virtual functions, and effectively acts as a hidden
class member, so you get one of those inside the block, and it's
included in sizeof.

//Allocated Space: The Final Frontier!
#include <stdio.h> //cout sucks :)

class Foo
{
int x;
int y;
int z;
};

class Bar
{
int x;
int y;
int z;
virtual int get_x() {return x;}
};

int main()
{
printf("sizeof(int) = %u\n",sizeof(int));
printf("sizeof(int*) = %u\n",sizeof(int*));
printf("sizeof(Foo) = %u\n",sizeof(Foo));
printf("sizeof(Bar) = %u\n",sizeof(Bar));
Foo *foo = new Foo[10];
printf("foo = %p/%p = %u\n",foo,foo+10,(char *)(foo+10)-(char *)foo);
Bar *bar = new Bar[10];
printf("bar = %p/%p = %u\n",bar,bar+10,(char *)(bar+10)-(char *)bar);
return 0;
}


rosuav@sikorsky:~$ g++ frontier.cpp && ./a.out
sizeof(int) = 4
sizeof(int*) = 8
sizeof(Foo) = 12
sizeof(Bar) = 24
foo = 0xf38010/0xf38088 = 120
bar = 0xf38090/0xf38180 = 240



The rules of structs are that they be contiguous, that they be laid
out sequentially, and that any padding needed between structures is at
the end of the previous one (which is why three of 4 bytes makes 12
bytes, but three of 4 bytes plus 8 bytes makes 24 - the eight-byte
pointer has to be aligned on a multiple of eight bytes, so having a
20-byte structure that starts with an 8-byte pointer is a no-no). The
allocated block of memory is, by definition, the same as the pointer
to its first element. As it happens, the pointer bar is not synonymous
with &bar->x, &bar->y, or &bar->z, which means the vptr is at the
beginning of bar, which makes sense; but the compiler's not obliged to
do that, and in some cases may choose not to - for instance, if bar
(with a virtual function) inherited from foo (with none), it might be
convenient to allow a pointer-cast to not change the value of the
pointer. (g++ 4.7.2 still puts the vptr at the beginning of bar in
that case, but other compilers or other versions may differ.)

Array size is outside the block, presumably before it, as &foo[0] is
by definition identical to foo, and there's no room inside the
structure for any spare data. Virtual function table is inside the
block because it's a hidden member of the object (like __class__ in
Python, only better hidden).

ChrisA
 
M

Marko Rauhamaa

Marko Rauhamaa said:
Roy Smith said:
Python already has a switch statement. It's just spelled funny...

[...]

try:
raise value
except Case1:
print "did case 1"
except (Case2, Case3):
print "did either case 2 or 3"
else:
print "did default"

Not bad! Definitely worth considering.

I wrote a simple test that switched between 26 "enums" using three
techniques and measured execution times (2,600,000 switching
operations). I also measured the time with no switching and subtracted
that time from the test times.

Results of the competition (performed with python3.2.3):

1. DICT DISPATCH TABLE (0.2 µs/switch)

2. IF-ELSE CHAIN (850% slower than DICT DISPATCH TABLE)

3. TRY-EXCEPT (3500% slower than DICT DISPATCH TABLE)


Marko
 
R

Roy Smith

Marko Rauhamaa said:
Marko Rauhamaa said:
Roy Smith said:
Python already has a switch statement. It's just spelled funny...

[...]

try:
raise value
except Case1:
print "did case 1"
except (Case2, Case3):
print "did either case 2 or 3"
else:
print "did default"

Not bad! Definitely worth considering.

[...]
3. TRY-EXCEPT (3500% slower than DICT DISPATCH TABLE)

I'm glad to hear that. I hope nobody took me seriously when I suggested
this.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top