Can global variable be passed into Python function?

M

Marko Rauhamaa

Roy Smith said:
I'm glad to hear that. I hope nobody took me seriously when I
suggested this.

I actually have employed the idea before in a related but slightly
different use case.

Anyway, it's extremely close to the switch statement's use case and
should give some guidance if a proper switch statement is ever worked
into the language. What's killing the performance is the backtrace
generation and longjmp trickery. If an analogous syntax could be (A)
separated from BaseException and (B) compiled into a dict, we could have
a winner.


Marko
 
C

Chris Angelico

I actually have employed the idea before in a related but slightly
different use case.

Anyway, it's extremely close to the switch statement's use case and
should give some guidance if a proper switch statement is ever worked
into the language. What's killing the performance is the backtrace
generation and longjmp trickery. If an analogous syntax could be (A)
separated from BaseException and (B) compiled into a dict, we could have
a winner.

The trouble is, try/except fundamentally can't be compiled into a
dict, because Python's exception handling is based on subclasses.

try: func()
except FileNotFoundError: pass
except OSError: raise RuntimeError("Oops")
except Exception as e: log(e)
except: log("Aborting!"); raise
finally: log("Done")

How would you handle that with a dict? It's inherently ordered - a
FileNotFoundError would be caught by any one of those clauses, and
since there's no sane way to "pick the narrowest", the best way to
handle it is sequential evaluation. (Also, it's worth noting that
exception lists are not constants. It's possible to do downright
insane things like calling a function to figure out what exceptions to
handle. Yeah, that's pretty stupid, right there.)

A switch block that works with constants and equality *can* be turned
into a dict. If the constants are hashable, use them as the keys
directly; if they're not hashable and/or you want to use object
identity as the criterion (effectively like using 'is' rather than
'==' for your case statements), use id(x) as the keys, and make sure
you have other references to the objects. Then it'll be fine as a
straight-up dict.

If the switch block uses inequalities, then it suffers from the same
problem as the try/except block - it's inherently ordered, in case
(pun intended) there's a switched-on value that matches more than one.
(You could possibly optimize the int case, but that would be way WAY
too specific for a generic language structure.)

ChrisA
 
G

Grant Edwards

The quote you make from the C standard doesn't mention malloc, so
you're arguing different things.

No, I'm not. A pointer to a structure object and a pointer to it's
first field are the same. It doesn't matter where the object came
from.
It's not the compiler that casts the malloc return value to the
struct type.

That's irrelevent. The actual location of the memory containing the
struct object (static, stack, heap, shared) doesn't matter. The
address of the first field in a struture object _is_ the address of
the structure object.
 
M

Mark Lawrence

A switch block that works with constants and equality *can* be turned
into a dict. If the constants are hashable, use them as the keys
directly; if they're not hashable and/or you want to use object
identity as the criterion (effectively like using 'is' rather than
'==' for your case statements), use id(x) as the keys, and make sure
you have other references to the objects. Then it'll be fine as a
straight-up dict.

If the switch block uses inequalities, then it suffers from the same
problem as the try/except block - it's inherently ordered, in case
(pun intended) there's a switched-on value that matches more than one.
(You could possibly optimize the int case, but that would be way WAY
too specific for a generic language structure.)

ChrisA

You clearly don't get my point. I *DON'T* want to use stupid constants
and stupid equalities, I want to use identities. I *DON'T* want to use
stupid ==, I want to use 'is'. I *DON'T* care how many people with
years of experience of Python tell me that this is the wrong thing to
do, that is how I am going to do it. So, for the final time of asking,
how do I do the above with, and only with, the identity, even if you
stupidly keep on trying to tell me that this is wrong?
 
M

Mark Lawrence

That's irrelevent. The actual location of the memory containing the
struct object (static, stack, heap, shared) doesn't matter. The
address of the first field in a struture object _is_ the address of
the structure object.

You say struture, I'll say structure, let's call the whole thing off :)
 
M

Michael Torrie

<class 'int'>

So use ==. If it's later changed and you have to instead use 'is', you
can change your code.

I don't see why == wouldn't continue to work if os.POSIX_FADV_RANDOM
became an object of a different type.
 
D

Dave Angel

Chris Angelico said:
Sure, for some definition of "usable". Overhead such as block
size, freelist pointer etc., are obviously outside of the
returned block. But the array size that's specified in a call to
new [], and the vptr, are definitely inside the malloc'ed block,
and may be before the struct data.

Hmm. Last I was working with it, the array size to new[] was outside
the block, just as the block size to malloc(). The vptr is part of any
struct/class with virtual functions, and effectively acts as a hidden
class member, so you get one of those inside the block, and it's
included in sizeof.

//Allocated Space: The Final Frontier!
#include <stdio.h> //cout sucks :)

class Foo
{
int x;
int y;
int z;
};

class Bar
{
int x;
int y;
int z;
virtual int get_x() {return x;}
};

int main()
{
printf("sizeof(int) = %u\n",sizeof(int));
printf("sizeof(int*) = %u\n",sizeof(int*));
printf("sizeof(Foo) = %u\n",sizeof(Foo));
printf("sizeof(Bar) = %u\n",sizeof(Bar));
Foo *foo = new Foo[10];
printf("foo = %p/%p = %u\n",foo,foo+10,(char *)(foo+10)-(char *)foo);
Bar *bar = new Bar[10];
printf("bar = %p/%p = %u\n",bar,bar+10,(char *)(bar+10)-(char *)bar);
return 0;
}


rosuav@sikorsky:~$ g++ frontier.cpp && ./a.out
sizeof(int) = 4
sizeof(int*) = 8
sizeof(Foo) = 12
sizeof(Bar) = 24
foo = 0xf38010/0xf38088 = 120
bar = 0xf38090/0xf38180 = 240



The rules of structs are that they be contiguous, that they be laid
out sequentially, and that any padding needed between structures is at
the end of the previous one (which is why three of 4 bytes makes 12
bytes, but three of 4 bytes plus 8 bytes makes 24 - the eight-byte
pointer has to be aligned on a multiple of eight bytes, so having a
20-byte structure that starts with an 8-byte pointer is a no-no). The
allocated block of memory is, by definition, the same as the pointer
to its first element. As it happens, the pointer bar is not synonymous
with &bar->x, &bar->y, or &bar->z, which means the vptr is at the
beginning of bar, which makes sense; but the compiler's not obliged to
do that, and in some cases may choose not to - for instance, if bar
(with a virtual function) inherited from foo (with none), it might be
convenient to allow a pointer-cast to not change the value of the
pointer. (g++ 4.7.2 still puts the vptr at the beginning of bar in
that case, but other compilers or other versions may differ.)

Array size is outside the block, presumably before it, as &foo[0] is
by definition identical to foo, and there's no room inside the
structure for any spare data. Virtual function table is inside the
block because it's a hidden member of the object (like __class__ in
Python, only better hidden).

Array size is inside the malloc block, but outside the struct
block. As you can see if you try to delete without the brackets
when you used new [], some runtimes will crash.

This is not to say that there will always be these extra offsets,
just that they can be there.
 
C

Chris Angelico

You say struture, I'll say structure, let's call the whole thing off :)

:)

Note that, technically, Grant is correct as long as you grant (heh)
that a structure may have an invisible member, the virtual function
table pointer. C++ only (I don't believe C has virtual functions - but
it may have grown them in one of the newer standards), so in C, all
members are public.

With an array, the array's pointer *is* the same as the pointer to its
first member, because adding zero to a pointer does nothing, and x <->
&x[0] <-> &(*(x+0)).

ChrisA
 
C

Chris Angelico

Array size is inside the malloc block, but outside the struct
block. As you can see if you try to delete without the brackets
when you used new [], some runtimes will crash.

As in, you have to use "delete [] x" to correspond with "x = new
whatever[n]"? Yes, that's right, but that array size is earlier in
memory than x itself. I can pretend that x is the same as one declared
statically as "whatever x[n]", and it'll function the same way. When
new[] is implemented using malloc(), it'll be something like this:

{
data = malloc(n * sizeof(whatever) + sizeof n);
*(int *)data = n;
return ((int *)data)+1;
}

so in that case, the array size is inside the malloc'd block, but it's
still invisible to the calling function. A fully compliant C++
implementation could choose to store that elsewhere, in some kind of
lookup table - it could then easily catch bugs like "delete
malloc(1)", "delete [] malloc(1)", "delete [] new whatever", and
"delete new whatever[1]" (because the pointer given wouldn't be in the
'new' table or the 'new[]' table, respectively).

ChrisA
 
M

Marko Rauhamaa

Michael Torrie said:
I don't see why == wouldn't continue to work if os.POSIX_FADV_RANDOM
became an object of a different type.

It probably would.

If one were begging for trouble, one *could* define:

class ABC:
A = 1
B = 1.0
C = 1+0j

Now:

ABC.A == ABC.B
ABC.B == ABC.C
ABC.C == ABC.A

but:

ABC.A is not ABC.B
ABC.B is not ABC.C
ABC.C is not ABC.A


Marko
 
R

Roy Smith

Marko Rauhamaa said:
It probably would.

If one were begging for trouble, one *could* define:

class ABC:
A = 1
B = 1.0
C = 1+0j

Now:

ABC.A == ABC.B
ABC.B == ABC.C
ABC.C == ABC.A

but:

ABC.A is not ABC.B
ABC.B is not ABC.C
ABC.C is not ABC.A


Marko

On can do all sorts of bizarre things. If you wish to shoot yourself in
the foot, Python is happy to provide the gun.

class ABC(object):
_instance = None
def __new__(cls):
if cls._instance is None:
i = object.__new__(cls)
i.__class__ = ABC
cls._instance = i
return cls._instance

def __eq__(self, other):
return False

a = ABC()
b = ABC()

print a is b
print a == b
 
M

Michael Torrie

It probably would.

If one were begging for trouble, one *could* define:

class ABC:
A = 1
B = 1.0
C = 1+0j

And one could also set A=1 and B=1 if he was trying to be stupid. That
would fail the equality test and the identity test (in CPython). Seems
like this argument is getting a bit on the absurd side. The normal
idiom is to use equality checks to test state variables' *values*. I
don't know of any developer that would purposely try to break that when
defining a new module or class.

If Mark H wants to use an idiom that isn't conventional, or isn't widely
used, he is free to do so; I can't see much harm in it. But certainly
it's not the "normal" way that it's done in Python from what I can see.
 
M

Marko Rauhamaa

Michael Torrie said:
And one could also set A=1 and B=1 if he was trying to be stupid.
[...]
If Mark H wants to use an idiom that isn't conventional, or isn't
widely used, he is free to do so; I can't see much harm in it. But
certainly it's not the "normal" way that it's done in Python from what
I can see.

You might be referring to what I have proposed.

Note that the idiom is in use in some standard python modules
(socket.py, ftplib.py, argparse.py, optparse.py). It is used extensively
in sre_compile.py/sre_constants.py:

ANY = "any"
[...]
AT = "at"
[...]
CALL = "call"
[...]
IN = "in"

elif op is IN:
[...]
elif op is ANY:
[...]
elif op is CALL:
[...]
elif op is AT:


Marko
 
D

Dave Angel

Chris Angelico said:
}

so in that case, the array size is inside the malloc'd block, but it's
still invisible to the calling function.

Please quit using negative language when you're so vehemently
agreeing with me.

The data is sometimes not at the beginning of the malloc'ed block.
 
S

Steven D'Aprano

Then, the documentation is seriously flawed. It gives no hint whatsoever
on the nature of those objects.

"Seriously" flawed? I doubt it. It's a trivial, pedantic point, and I
expect that most practising Python programmers will consider it too
obvious to bother documenting the fact that flags meant for compatibility
with POSIX operating systems are ints.

On the other hand, perhaps I am wrong and it is a documentation bug. Feel
free to suggest a documentation patch on the bug tracker.

The values of those Python constants don't need to have any relationship
with those of the underlying operating system.

In theory, they could be different. In practice, no they won't. You
should be able to pass the Python constants directly to some C library
which expects to see ints. It's a thin wrapper, not a bridge.

Nobody has ever argued such a thing. I certainly haven't.

You may not have intended to, but by championing the use of "is", that is
precisely what you have done. Using "is" tests for *identity*, not value.
To get the behaviour you want, it requires those objects to be singletons.

However, nothing in the API spec gives you the right to call the
function with an integer.

But you do call the function with an integer. And if you don't, you get a
type error that explicitly tells you that an integer is needed:

py> os.posix_fadvise(open("/tmp/spam").fileno(), 0, 100, None)
Traceback (most recent call last):
File "<stdin>", line 1, in <module>
TypeError: an integer is required

That might well be true but is not explicitly or implicitly specified in
the documentation. (The os.SEEK_* constants are explicitly defined.)

Not everything needs to be documented explicitly. Would you rather the
Python developers spend their time fixing bugs and improving the
language, or ensuring that every trivial and obvious point is explicitly
documented?

If you feel this is not a trivial or obvious point, and that it needs
documenting, then feel free to contribute a documentation patch.
 
G

Grant Edwards

:)

Note that, technically, Grant is correct as long as you grant (heh)
that a structure may have an invisible member, the virtual function
table pointer. C++ only (I don't believe C has virtual functions -
but it may have grown them in one of the newer standards), so in C,
all members are public.

Yes. I was talking about C, not C++. I made that quite clear in
portions of my post that have been elided. In C there is no such
thing as a virtual table pointer.
 
C

Chris Angelico

Yes. I was talking about C, not C++. I made that quite clear in
portions of my post that have been elided. In C there is no such
thing as a virtual table pointer.

I wasn't certain of the newer C standards. C's been gaining all sorts
of features, not all of which are necessary, and it's entirely
possible based on my current knowledge that C specifies #include
<antigravity> ...

ChrisA
 
A

Albert van der Horst

Your brief summary, please, Mark?

Anyway, the first 1000 lines or so that I managed to read from that page
stated a valid principle, which however doesn't invalidate the existence
of a switch statement.

A colleague of mine taught me decades back that the whole point of OO
was the avoidance of if and switch statements. So if your code has an if
or switch statement, chances are you are doing something wrong.

I agree.

However, like all other maxims, that principle, too, has exceptions. Two
recurring examples are parsers/decoders and state machines. Sure, you
can implement states beautifully with objects/classes (and I do do that
when performance is not an issue), but having experimented with
different implementation techniques (in C, C++ and Java), I have
concluded that switch statements are often unbeatable in performance and
clarity.

I can't see why parsers decoders are any different.
The Pentium assembler in my ciforth's ``forth.lab'' library has not
a single if statement and I reckon it is a superior design.
(State is kept in an ai blackboard fashion in bitmaps.)
Forth has of course a built in "look it up, then execute it",
which could be regarded as a giant switch.
 
M

Marko Rauhamaa

(e-mail address removed)4all.nl (Albert van der Horst):
I can't see why parsers decoders are any different. The Pentium
assembler in my ciforth's ``forth.lab'' library has not a single if
statement and I reckon it is a superior design. (State is kept in an
ai blackboard fashion in bitmaps.) Forth has of course a built in
"look it up, then execute it", which could be regarded as a giant
switch.

Isn't this the idea of using Python dicts as hand-compiled switch
statements?


Marko
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,773
Messages
2,569,594
Members
45,119
Latest member
IrmaNorcro
Top