Subclassing built-in classes

[MonkeeSage]

I know that python doesn't allow extending built-in objects like the
str class; but you can subclass them using a class of the same name and
thus shadow them to get the same general effect (albeit you have to use
the explicit constructor rather than literals).

class str(str):
def display(self):
print self
str('blah').display()

I was just playing around and realized that assigning to
__builtins__.str (or if you prefer sys.modules['__builtin__'].str) has
the same effect.

class mystr(str):
def display(self):
print self
__builtins__.str = mystr
str('blah').display()

So that made me wonder...couldn't python (in theory) allow for literals
to use extended classes by using the object in __builtins__.<class> as
the class for literals? By default it would be the standard base class,
but it could also be a custom subclass. Would that be possible / easy /
worthwhile to do?

Regards,
Jordan

 

[Steve Holden]

I know that python doesn't allow extending built-in objects like the
str class; but you can subclass them using a class of the same name and
thus shadow them to get the same general effect (albeit you have to use
the explicit constructor rather than literals).

class str(str):
def display(self):
print self
str('blah').display()

I was just playing around and realized that assigning to
__builtins__.str (or if you prefer sys.modules['__builtin__'].str) has
the same effect.

class mystr(str):
def display(self):
print self
__builtins__.str = mystr
str('blah').display()

So that made me wonder...couldn't python (in theory) allow for literals
to use extended classes by using the object in __builtins__.<class> as
the class for literals? By default it would be the standard base class,
but it could also be a custom subclass. Would that be possible / easy /
worthwhile to do?

Unfortunately the literals are interpreted during bytecode generation,
before the compiled program is available, and your modifications to
__builtns__ haven't been made, so the answer is "no", I'm afraid.

regards
Steve

 

[MonkeeSage]

Unfortunately the literals are interpreted during bytecode generation,
before the compiled program is available, and your modifications to
__builtns__ haven't been made, so the answer is "no", I'm afraid.

Ah! That makes sense. I guess the only way to do it would be to add an
extra bit to every object to indicate whether it was constructed
literally and then re-initialize the object after compilation if that
bit is set. For some reason I just don't think that's gonna happen.

Thanks for taking the time to explain.

Regards,
Jordan

 

[Maric Michaud]

Le jeudi 05 octobre 2006 14:20, Steve Holden a écrit :

Unfortunately the literals are interpreted during bytecode generation,
before the compiled program is available, and your modifications to
__builtns__ haven't been made, so the answer is "no", I'm afraid.

But what prevents to interpret literals as a call to __builtins__ objects and
functions ? optimization ? what else ?

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Steve Holden]

Le jeudi 05 octobre 2006 14:20, Steve Holden a écrit :



But what prevents to interpret literals as a call to __builtins__ objects and
functions ? optimization ? what else ?

How can you modify __builtins__ before your program starts running? You
can't.

What does the interpreter have to do before the program runs? Translate
it into bytecode.

When are literals interpreted? During translation into bytecode.

regards
Steve

 

[Maric Michaud]

Le jeudi 05 octobre 2006 15:52, Steve Holden a écrit :

When are literals interpreted? During translation into bytecode.

agreed, but what's the problem with this ?

We can actually monkey patch all buitins like in this example :

In [1]: oldstr=str

In [2]: class mystr(str) :
...: def __new__(*a, **kw) :
...: print 'called : ', a, kw
...: return oldstr.__new__(*a, **kw)
...:
...:

In [3]: import __builtin__

In [4]: __builtin__.str = mystr
called : (<class '__main__.mystr'>, <ItplNS 'In
[${self.cache.prompt_count}]: ' >) {}
called : (<class '__main__.mystr'>, 5) {}
...


If the generated bytecode of {k:v} is more or less the same as the one
gernerated by dict(((k,v))), monkey patching dict will work for dict literals
too.

Also, this should work with scalars, 'a string' be translated in what
actually is produced by str('a string') (of course the internal code for
building scalars should still be there).

If that is feasible without big refactoring and do not introduce noticeable
performance loss is what I don't know, but it could be a nice feature of
__builtin__ module IMO (at less I expected it to work like this when I first
tried it).

--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[Steve Holden]

Le jeudi 05 octobre 2006 15:52, Steve Holden a écrit :

When are literals interpreted? During translation into bytecode.

agreed, but what's the problem with this ?

We can actually monkey patch all buitins like in this example :

In [1]: oldstr=str

In [2]: class mystr(str) :
...: def __new__(*a, **kw) :
...: print 'called : ', a, kw
...: return oldstr.__new__(*a, **kw)
...:
...:

In [3]: import __builtin__

In [4]: __builtin__.str = mystr
called : (<class '__main__.mystr'>, <ItplNS 'In
[${self.cache.prompt_count}]: ' >) {}
called : (<class '__main__.mystr'>, 5) {}
....


If the generated bytecode of {k:v} is more or less the same as the one
gernerated by dict(((k,v))), monkey patching dict will work for dict literals
too.

Also, this should work with scalars, 'a string' be translated in what
actually is produced by str('a string') (of course the internal code for
building scalars should still be there).

If that is feasible without big refactoring and do not introduce noticeable
performance loss is what I don't know, but it could be a nice feature of
__builtin__ module IMO (at less I expected it to work like this when I first
tried it).

C:\Steve\Projects\tgtest>python
Python 2.4.2 (#67, Sep 28 2005, 12:41:11) [MSC v.1310 32 bit (Intel)] on
win32
Type "help", "copyright", "credits" or "license" for more information.
Started with C:/Steve/.pythonrc ... def __new__(*a, **kw):
... print "called:", a, kw
...Readline internal error
Traceback (most recent call last):
File "C:\Python24\lib\site-packages\readline\Console.py", line 644,
in hook_wrapper_23
raise TypeError, 'readline must return a string.'
called: (<class '__main__.mystr'>, <exceptions.TypeError instance at
0x00B3E2B0>
) {}
Readline internal error
Traceback (most recent call last):
File "C:\Python24\lib\site-packages\readline\Console.py", line 644,
in hook_wrapper_23
raise TypeError, 'readline must return a string.'
called: (<class '__main__.mystr'>, <exceptions.TypeError instance at
0x00B4D288>
) {}

Seems like there might be a few glitches to debug still ... or perhaps
it's my fault for using readline? Let's try under Cygwin as a program:

sholden@bigboy ~/Projects/Python
$ cat test39.py
myStr = "This is a string and it alwyas will be"

oldstr=str

class mystr(str) :
def __new__(*a, **kw) :
print 'called : ', a, kw
return oldstr.__new__(*a, **kw)

import __builtin__

__builtin__.str = mystr

print type(myStr)
newStr = "This is another string and it always will be"
print type(newStr)
strStr = str(newStr)
print type(strStr)

sholden@bigboy ~/Projects/Python
$ python test39.py
<type 'str'>
<type 'str'>
called : (<class '__main__.mystr'>, 'This is another string and it
always will be') {}
<class '__main__.mystr'>

So, what are you trying to say? The type of the literal strings is
clearly just what it always was. Where, when and how exactly does this
magical monkey patch affect literals?

regards
Steve

 

[Maric Michaud]

Le jeudi 05 octobre 2006 20:24, Steve Holden a écrit :

  >>> class mystr(oldstr):
  ...   def __new__(*a, **kw):
  ...     print "called:", a, kw
  ...

you don't return the string here...

  >>> import __builtin__
  >>> __builtin__.str = mystr
  >>>
Readline internal error
Traceback (most recent call last):
   File "C:\Python24\lib\site-packages\readline\Console.py", line 644,
in hook_wrapper_23
     raise TypeError, 'readline must return a string.'

...It should explain this error.

So, what are you trying to say? The type of the literal strings is
clearly just what it always was. Where, when and how exactly does this
magical monkey patch affect literals?

ooops, maybe we aren't on the same plan.

As the first post said "...couldn't python (in theory)...", I was discussing
if it would be possible for python (in some future version) to manage the
literals so that they use the constructors in the __builtin__ module, I
didn't say it works actually (I'm aware it's not the case).


--
_____________

Maric Michaud
_____________

Aristote - www.aristote.info
3 place des tapis
69004 Lyon
Tel: +33 426 880 097

 

[MonkeeSage]

As the first post said "...couldn't python (in theory)...", I was discussing
if it would be possible for python (in some future version) to manage the
literals so that they use the constructors in the __builtin__ module, I
didn't say it works actually (I'm aware it's not the case).

Hi Maric,

I think the problem Steve was pointing out is this: how do you make a
literal use a constructor that is not available until *after* the
literal has been constructed? In other words, python currently does
this: 1) parses plaintext into bytecode (at which point literals are
constructed), 2) executes bytcode (at which point custom constructors
are available). The only way I can see to do what I asked is to, during
bytecode compilation, mark a literal in the AST as being a literal (+1
bits, at least, on every literal until it is constructed) but not
actually constructing it until the point of execution (+? time to go
back through the object space looking for marked objects and
constructing them). I don't know enough about python internals to know
if that would be a problem, but I seriously doubt it would be
implemented without some pressing use cases. So it is possible, but
probably not plausible.

Regards,
Jordan

 

[Gabriel Genellina]

As the first post said "...couldn't python (in theory)...", I was discussing
if it would be possible for python (in some future version) to manage the
literals so that they use the constructors in the __builtin__ module, I
didn't say it works actually (I'm aware it's not the case).

The idea looks crazy for me... You suggest that code like this:

x = 12 + 6.0 - len('ABCD'

would be evaluated at run time as it were:

x = int('12') + float('6.0') - len(str('ABCD'))

Certainly would slow down the whole execution time *a*lot*, with no
benefit for almost nobody, if *every* reference to *any* literal in
the code calls a python function at run time.
And unless you return *exactly* the same object as now, almost all
code would break!
Do you have any useful usage for this?


--
Gabriel Genellina
Softlab SRL





__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

 

[hanumizzle]

The idea looks crazy for me... You suggest that code like this:

x = 12 + 6.0 - len('ABCD'

would be evaluated at run time as it were:

x = int('12') + float('6.0') - len(str('ABCD'))

Certainly would slow down the whole execution time *a*lot*, with no
benefit for almost nobody, if *every* reference to *any* literal in
the code calls a python function at run time.
And unless you return *exactly* the same object as now, almost all
code would break!
Do you have any useful usage for this?

Sometimes I've had weird ideas that I thought might be useful, but
they turned out to be doo doo. On other occasions, the SWAG paid off
(e.g., vesa driver runs faster than accelerated via driver for
compositing in Xorg) It's all a matter of proposing and disposing, and
mistakes happen.

Somehow, I missed Python's round() function and came up with
convoluted solution involving decimal. Gee duh, Theerasak

-- Theerasak

 

[Gabriel Genellina]

Somehow, I missed Python's round() function and came up with
convoluted solution involving decimal. Gee duh, Theerasak

Ah! So this is applicable:
Describe the goal, not the step
http://catb.org/~esr/faqs/smart-questions.html#goal


--
Gabriel Genellina
Softlab SRL





__________________________________________________
Preguntá. Respondé. Descubrí.
Todo lo que querías saber, y lo que ni imaginabas,
está en Yahoo! Respuestas (Beta).
¡Probalo ya!
http://www.yahoo.com.ar/respuestas

 

[Theerasak Photha]

Ah! So this is applicable:
Describe the goal, not the step
http://catb.org/~esr/faqs/smart-questions.html#goal

Believe it or not, I did look for round() but by virtue of some
strange brain fart I didn't find it until later.

-- Theerasak