Python 2.6 Deprecation Warnings with __new__ — Can someone explain why?

[rh0dium]

Hi all,

I have a basic Monostate with Python 2.6.

class Borg(object):
__shared_state = {}
def __new__(cls, *args, **kwargs):
self = object.__new__(cls, *args, **kwargs)
self.__dict__ = cls.__shared_state
return self

def __init__(self, *args, **kwargs):
noSend = kwargs.get("noSend", False)
reportLevel = kwargs.get("reportLevel", 30)
reportMethods = kwargs.get("reportMethods", "BaseReport")
contacts= kwargs.get("contacts", None)

a = Borg(contacts="Foo", noSend="Bar", )

Which happily gives me the following Deprecation warning..

untitled:4: DeprecationWarning: object.__new__() takes no parameters
self = object.__new__(cls, *args, **kwargs)

After a bit of googling I find this is attached to Bug #1683368. What
I can't figure out is what Guidos answer means. FWIW - It's
complaining about the following line

self = object.__new__(cls, *args, **kwargs)

Which appears (to me) to be OK. Can someone explain in laymens terms
why this is a problem. I understand that "this is inconsistent with
other built-ins, like list" but I'm not sure I understand why. Would
someone explain this show me the right way to do it? I have read
Guido's answer on this but I guess I just don't understand his
reasoning. In short Guido said the following:

"The message means just what it says. There's no point in
calling object.new() with more than a class parameter, and any code
that did so was just dumping those args into a black hole.

The only time when it makes sense for object.new() to ignore extra
arguments is when it's not being overridden, but init is being
overridden -- then you have a completely default new and the checking
of constructor arguments is relegated to init."

In my case the args that it dumps them into a black hold is simply not
true. I want an unknown set of args and kwargs to simply be forwarded
onto init. So what's the problem with this??


Thanks

 

[Carl Banks]

Hi all,

I have a basic Monostate with Python 2.6.

class Borg(object):
    __shared_state = {}
    def __new__(cls, *args, **kwargs):
        self = object.__new__(cls, *args, **kwargs)
        self.__dict__ = cls.__shared_state
        return self

    def __init__(self, *args, **kwargs):
        noSend = kwargs.get("noSend", False)
        reportLevel = kwargs.get("reportLevel", 30)
        reportMethods = kwargs.get("reportMethods", "BaseReport")
        contacts= kwargs.get("contacts", None)

a = Borg(contacts="Foo", noSend="Bar", )

Before we get into object semantics, I'm not sure why you'd need to
override __new__ for Borg pattern, unless they're working around some
edge cases or something.
For that matter you shouldn't need args and kwargs either, you know
what the arguments to your function are.

This should suffice for you:

class Borg(object):
__shared_state = {}
def __init__(self, noSend=False,reportLevel=30,
reportMethods="BaseReport",
contacts=None):
self.__dict__ = self.__shared_state
self.noSend = noSend
self.reportLevel = reportLevel
self.reportMethods = reportMethods
self.contacts = contacts

This (as well as your class) will update the shared state for all Borg
objects whenever you instantiate a new Borg object. That might be
what you want, but be aware.

Which happily gives me the following Deprecation warning..

untitled:4: DeprecationWarning: object.__new__() takes no parameters
self = object.__new__(cls, *args, **kwargs)

After a bit of googling I find this is attached to Bug #1683368. What
I can't figure out is what Guidos answer means. FWIW - It's
complaining about the following line

self = object.__new__(cls, *args, **kwargs)

Which appears (to me) to be OK. Can someone explain in laymens terms
why this is a problem.

I don't mean to be flippant, but the reason you're seeing this error
is because object.__new__() takes no parameters.

There's not much more to it than that.

I'll explain below.

I understand that "this is inconsistent with
other built-ins, like list" but I'm not sure I understand why. Would
someone explain this show me the right way to do it?
object.__new__(cls)


 I have read
Guido's answer on this but I guess I just don't understand his
reasoning.  In short Guido said the following:

  "The message means just what it says. There's no point in
calling object.new() with more than a class parameter, and any code
that did so was just dumping those args into a black hole.

The only time when it makes sense for object.new() to ignore extra
arguments is when it's not being overridden, but init is being
overridden -- then you have a completely default new and the checking
of constructor arguments is relegated to init."

In my case the args that it dumps them into a black hold is simply not
true.

Actually it is.

 I want an unknown set of args and kwargs to simply be forwarded
onto init.  So what's the problem with this??

The problem is that __new__ doesn't forward the arguments to
__init__. Look at what happens when you call this class:

class X(object):
def __new__(cls,*args,**kwargs):
return object.__new__(cls)
def __init__(self,*args,**kwargs):
print args, kwargs

X(1,2,q=3,g=4)


Note that *args and **kwargs are not passed to object.__new__, but
nevertheless __init__ is still called with them.

What's actually happens is that the type object's __call__ method
passes the same arguments to both __new__ and __init__.

The fact is, object.__new__ does nothing at all with the args and
kwargs arguments, it just ignores them. So, when calling
object.__new__, don't pass them to it.


Carl Banks

 

[Terry Reedy]

Consider this:

def blackhole(*args, **kwds): pass

The fact that it accept args that it ignores could be considered
misleading or even a bug. Now modify it to do something useful, like
return a new, naked, immutable object that is the same for every call
except for identity, and which still totally ignores the args as
irrelavant. Call it object.__new__. It is just as misleading, if not
more so.

In 3.x, the mistake has been fixed.Traceback (most recent call last):
File "<pyshell#9>", line 1, in <module>
object(1)
TypeError: object.__new__() takes no parameters

Terry Jan Reedy

 

[Dieter Maurer]

Consider this:

def blackhole(*args, **kwds): pass

The fact that it accept args that it ignores could be considered
misleading or even a bug.

Maybe, it could. But, it is by no means necessary.

In mathematics, there is a set of important functions which behave precisely
as described above (there ignore their arguments); they are called
"constant functions"....

 

[rh0dium]

Carl,

First off - Thanks your post was exactly the kind of informative
example driven learnings that help — Thanks!!

Before we get into object semantics, I'm not sure why you'd need to
override __new__ for Borg pattern, unless they're working around some
edge cases or something.
For that matter you shouldn't need args and kwargs either, you know
what the arguments to your function are.

Good point — admitadly I blindly followed Alex M's advice. I am still
working my way through this. Point taken.

This should suffice for you:

class Borg(object):
    __shared_state = {}
    def __init__(self, noSend=False,reportLevel=30,
                 reportMethods="BaseReport",
                 contacts=None):
        self.__dict__ = self.__shared_state
        self.noSend = noSend
        self.reportLevel = reportLevel
        self.reportMethods = reportMethods
        self.contacts = contacts

This (as well as your class) will update the shared state for all Borg
objects whenever you instantiate a new Borg object.  That might be
what you want, but be aware.

Now the real question I have on this is scalability. The real
advantage to using *args and **kwargs is that down the road (through
inheritance/polymorphism) I may not know what I'm being given (isn't
that the essence of duck-typing). My long standing belief is that by
using *args and **kwargs I plan for future additions with minimal
changes to my code. So doesn't restricting this just defeat the
purpose?

Actually it is.


The problem is that __new__ doesn't forward the arguments to
__init__.  Look at what happens when you call this class:

class X(object):
    def __new__(cls,*args,**kwargs):
        return object.__new__(cls)
    def __init__(self,*args,**kwargs):
        print args, kwargs

X(1,2,q=3,g=4)

Note that *args and **kwargs are not passed to object.__new__, but
nevertheless __init__ is still called with them.

What's actually happens is that the type object's __call__ method
passes the same arguments to both __new__ and __init__.

The fact is, object.__new__ does nothing at all with the args and
kwargs arguments, it just ignores them.  So, when calling
object.__new__, don't pass them to it.

So what is the point of using __new__? After a bit of reading
http://docs.python.org/reference/datamodel.html 3.4.1 and I think I
understand. More to the point you made above. The whole necessity of
__new__ above makes most of this moot


Once again thanks Carl for your great response!!

Steven

 

[Carl Banks]

Now the real question I have on this is scalability.  The real
advantage to using *args and **kwargs is that down the road (through
inheritance/polymorphism) I may not know what I'm being given (isn't
that the essence of duck-typing).

No, duck-typing means you don't necessarily know the types of
arguments you're getting. Not knowing the number, pattern, or usage
of the arguments you're getting is something else.

My long standing belief is that by
using *args and **kwargs I plan for future additions with minimal
changes to my code.  So doesn't restricting this just defeat the
purpose?

If that's your intended purpose down the road, feel free to do it.
(Not sure why you would do that with a singleton, but that's your
business.)

Personally, I wouldn't recommend replacing regular arguments with
*args and **kwargs unless there was a specific reason to believe that
this class would be used in a heirarchy where it's expected to pass on
arguments it doesn't know about. If it just "might someday" be used
that way, I'd say it's a lot of unnecessary caution.

One place I do recommend using *args and **kwargs is with mixin
classes, because mixins usually are required to pass along unknown
arguments, not just "might be someday".

So what is the point of using __new__?

It's mostly for types written in C, or for subclassing types written
in C. Advanced programmers can take advantage of it to do some
interesting things, but most of the time __init__ suffices.


Carl Banks

 

[Terry Reedy]

..__new__ creates new objects. It also inializes 'immutable' objects.

It's mostly for types written in C, or for subclassing types written
in C.

Specifically, for subclassing immutable classes where one wants
initialization behavior different from that provided by the superclass.

> Advanced programmers can take advantage of it to do some
interesting things, but most of the time __init__ suffices.

Initialization of mutables should be done in .__init__. Most
user-defined classes define mutables.

Terry Jan Reedy

 

[Simon Forman]

Now the real question I have on this is scalability.  The real

What you're describing isn't "scalability". It could be called "extensibility".

advantage to using *args and **kwargs is that down the road (through
inheritance/polymorphism) I may not know what I'm being given (isn't
that the essence of duck-typing). My long standing belief is that by
using *args and **kwargs I plan for future additions with minimal
changes to my code.  So doesn't restricting this just defeat the
purpose?

YAGNI?