use of __new__ to permit "dynamic" completion within (any?) IDE ?

Discussion in 'Python' started by gst, Dec 7, 2010.

  1. gst

    gst Guest

    Hi,

    I met a situation where I was passing an object created in/with an
    upper level module class to a lower level module class' instance in
    one of its __init__ argument and saving a ref of the upper object in
    that lower level class' new instance.

    But in my IDE I want the completion to also work from within the lower
    level module when it's refering to the object passed from the upper
    level:


    Well, I'm sure I'm not very clear in my terms (and probably a bit long
    in the sentence) so here it is in code:


    files:
    module1.py
    subpackage/module2.py


    file module1.py:

    from subpackage.module2 import class2

    class class1(object):

    def __new__(cls, _self=None, *args, **kwargs):
    if _self: ## we've been passed an instance already
    initialyzed
    ## so directly return it instead of creating a
    new object.
    return _self
    return object.__new__(cls)

    def __init__(self, _self=None, *args, **kwargs):
    if _self: ## we've been passed an instance already
    initialyzed
    ## so directly returns
    ## assert(self is _self) ?
    return
    self.object2 = class2(object1=self, "blip", "blop")
    # others init


    file module2.py:

    class class2(object):

    def __init__(self, object1, *args, **kwargs):

    from ..module1 import class1

    self.object1 = class1(_self=object1) ## instead of:
    self.object1 = object1

    ## others functions and/or init..
    ## where now I've completion working on self.object1 :
    ## if I add(or remove) fields/methods in module1 (and save) then
    ## I have them available(or disappeared) in the completion when
    executed from this submodule.
    ## This ofcourse permits to save me of remembering all of the
    class1 attributes/methods when I'm working with self.object1 from
    within one of class2 methods.


    What do you think of this ?

    I guess there can be others ways of doing this.. ?

    Thanks,

    Regards,

    Greg.

    note: I'm using Eclipse 3.5.2 with pydev so (I don't really know how
    others IDE can handle this case) .
     
    gst, Dec 7, 2010
    #1
    1. Advertising

  2. Re: use of __new__ to permit "dynamic" completion within (any?)IDE ?

    On Tue, 07 Dec 2010 07:52:06 -0800, gst wrote:

    > Hi,
    >
    > I met a situation where I was passing an object created in/with an upper
    > level module class to a lower level module class' instance in one of its
    > __init__ argument and saving a ref of the upper object in that lower
    > level class' new instance.
    >
    > But in my IDE I want the completion to also work from within the lower
    > level module when it's refering to the object passed from the upper
    > level:


    "The completion"? What do you mean?


    > Well, I'm sure I'm not very clear in my terms (and probably a bit long
    > in the sentence) so here it is in code:


    I'm afraid I have to agree with your, your description is not clear to me.

    Unfortunately, neither is your code, because I don't understand *why* you
    do the things you do.


    > files:
    > module1.py
    > subpackage/module2.py
    >
    >
    > file module1.py:
    >
    > from subpackage.module2 import class2


    Is it relevant that class2 comes from another module? Would your concept
    work if class1 and class2 were defined in the same file?

    By the way, it's conventional (but not compulsory) to name classes with
    an initial capital letter, and instances in all lowercase.



    > class class1(object):
    >
    > def __new__(cls, _self=None, *args, **kwargs):
    > if _self: ## we've been passed an instance already
    > initialyzed
    > ## so directly return it instead of creating a
    > new object.
    > return _self
    > return object.__new__(cls)


    Depending on your application, this may be an unsafe assumption. If the
    caller says class1("spam"), your class will return "spam". This may or
    may not be what you want.


    > def __init__(self, _self=None, *args, **kwargs):
    > if _self: ## we've been passed an instance already
    > initialyzed
    > ## so directly returns
    > ## assert(self is _self) ?
    > return
    > self.object2 = class2(object1=self, "blip", "blop") # others
    > init


    Okay, this seems like a fairly straightforward case of adding an
    attribute to your instance that includes a reference to itself. No big
    deal... there's nothing wrong with this, although you are creating a
    reference cycle, which is somewhat of a (mild) code-smell.


    > file module2.py:
    >
    > class class2(object):
    >
    > def __init__(self, object1, *args, **kwargs):
    >
    > from ..module1 import class1


    Now you have a circular import, and that's a pretty major code smell.
    That's dangerous. Try to avoid it. Perhaps the easiest way to avoid it is
    to place class1 and class2 in the same module, and stop writing Java :)


    > self.object1 = class1(_self=object1) ## instead of:
    > self.object1 = object1
    >
    > ## others functions and/or init..
    > ## where now I've completion working on self.object1 : ## if I
    > add(or remove) fields/methods in module1 (and save) then ## I have
    > them available(or disappeared) in the completion when
    > executed from this submodule.
    > ## This ofcourse permits to save me of remembering all of the
    > class1 attributes/methods when I'm working with self.object1 from within
    > one of class2 methods.


    I'm afraid I don't understand what you mean here in these comments. What
    do you mean, "save me of (from?) remembering all the class1 attributes/
    methods..." -- how does it save you from knowing the methods? Whether you
    write this:

    instance = class1()
    instance.method()

    or this:

    instance = class1()
    another_instance = class2(instance)
    another_instance.object1.method()

    you still need to know the name "method".

    It seems to me that this scheme is unnecessarily complex, convoluted and
    confusing, for very little gain. Perhaps you could try explaining what
    you hope to accomplish, giving examples?


    --
    Steven
     
    Steven D'Aprano, Dec 8, 2010
    #2
    1. Advertising

  3. gst

    gst Guest

    Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    On 8 déc, 11:45, Steven D'Aprano <steve
    > wrote:
    > On Tue, 07 Dec 2010 07:52:06 -0800, gst wrote:
    > > Hi,

    >
    > > But in my IDE I want the completion to also work from within the lower
    > > level module when it's refering to the object passed from the upper
    > > level:

    >
    > "The completion"? What do you mean?


    Hi,

    I mean the autocompletion feature of my IDE :

    when I type "object." then on the "." it shows me possible members/
    methods of the 'object'. depending on what my IDE can guess about the
    object possible type(s) ; in the worst case it can't guess nothing
    about it and that's my point below..


    > > Well, I'm sure I'm not very clear in my terms (and probably a bit long
    > > in the sentence) so here it is in code:

    >
    > I'm afraid I have to agree with your, your description is not clear to me..


    I hope I'm now ;)

    > Unfortunately, neither is your code, because I don't understand *why* you
    > do the things you do.


    damn. ok let's go..


    > > file module1.py:

    >
    > > from subpackage.module2 import class2

    >
    > Is it relevant that class2 comes from another module?


    yes. hmmm or maybe not completely.. I think it only highly depends on
    what the IDE can guess on an object type used inside some methods
    depending on the files hierarchy..


    > Would your concept work if class1 and class2 were defined in the same file?


    I guess yes also.


    > > class class1(object):

    >
    > >     def __new__(cls, _self=None, *args, **kwargs):
    > >         if _self:  ## we've been passed an instance already
    > > initialyzed
    > >                      ## so directly return it instead of creating a
    > > new object.
    > >             return _self
    > >         return object.__new__(cls)

    >
    > Depending on your application, this may be an unsafe assumption. If the
    > caller says class1("spam"), your class will return "spam". This may or
    > may not be what you want.


    I'm always passing the good object type when I do this.


    > >     def __init__(self, _self=None, *args, **kwargs):
    > >         if _self:  ## we've been passed an instance already
    > > initialyzed
    > >                      ## so directly returns
    > >             ## assert(self is _self) ?
    > >             return
    > >         self.object2 = class2(object1=self, "blip", "blop") # others
    > >         init

    >
    > Okay, this seems like a fairly straightforward case of adding an
    > attribute to your instance that includes a reference to itself. No big
    > deal... there's nothing wrong with this, although you are creating a
    > reference cycle, which is somewhat of a (mild) code-smell.


    that is a worry effectively. (hmm when you say it's a case of adding
    an attribute that includes a ref to itself it's not really that (or i
    badly understand) : it's adding an attribute to a lower level class
    instance that contains a ref to an upper level class instance (which
    also contains a ref to the lower level class' instance and that's the
    cycle (and that's a bit my worry))

    > > file module2.py:

    >
    > > class class2(object):

    >
    > >     def __init__(self, object1, *args, **kwargs):

    >
    > >         from ..module1 import class1

    >
    > Now you have a circular import, and that's a pretty major code smell.
    > That's dangerous. Try to avoid it. Perhaps the easiest way to avoid it is
    > to place class1 and class2 in the same module, and stop writing Java :)


    damn if this looks like Java it's really not my will ; I've not made
    Java anymore for probably 10 years !

    I'll try by putting class1 and class2 in same module (or at same
    directory level first) and see if that can do it..

    >
    > >         self.object1 = class1(_self=object1)    ## instead of:
    > > self.object1 = object1

    >
    > >     ## others functions and/or init..
    > >     ## where  now I've completion working on self.object1 : ## if I
    > >     add(or remove) fields/methods in module1 (and save) then ## I have
    > >     them available(or disappeared) in the completion when
    > > executed from this submodule.
    > >     ## This ofcourse permits to save me of remembering all of the
    > > class1 attributes/methods when I'm working with self.object1 from within
    > > one of class2 methods.

    >
    > I'm afraid I don't understand what you mean here in these comments. What
    > do you mean, "save me of (from?) remembering all the class1 attributes/
    > methods..." -- how does it save you from knowing the methods? Whether you
    > write this:
    >
    > instance = class1()
    > instance.method()
    >
    > or this:
    >
    > instance = class1()
    > another_instance = class2(instance)
    > another_instance.object1.method()
    >
    > you still need to know the name "method".
    >
    > It seems to me that this scheme is unnecessarily complex, convoluted and
    > confusing, for very little gain. Perhaps you could try explaining what
    > you hope to accomplish, giving examples?


    Effectively I recognise that this seems like a bit silly/odd .. but as
    it's working I wanted some advises so ;)

    As I told I try to have some kind of "ultimate" autocompletion, always
    working for this kind of case (I know exactly that a certain member of
    a certain class' instance will always have the same type but my IDE
    doesn't seem to guess/know it.. ; I guess the best answer would be :
    bad IDE, change IDE but I'm guessing that others IDE can't do a lot
    better in this case.

    Anyway many thanks for your reply ; I'll do some further tests with
    your comments already..

    regards,

    greg.

    nb: so this "hack" is only relevant during dev ; once the project
    would be finished the "hack" could be removed (i.e : in class2 init I
    would directly do : self.object1 = object1)
     
    gst, Dec 8, 2010
    #3
  4. gst

    Steve Holden Guest

    Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    On 12/8/2010 1:01 PM, gst wrote:
    > As I told I try to have some kind of "ultimate" autocompletion, always
    > working for this kind of case (I know exactly that a certain member of
    > a certain class' instance will always have the same type but my IDE
    > doesn't seem to guess/know it.. ; I guess the best answer would be :
    > bad IDE, change IDE but I'm guessing that others IDE can't do a lot
    > better in this case.
    >

    If you'd told us which IDE you were using we might have offered better
    advice, but you seem to want to keep that a secret ("my IDE" tells us
    nothing).

    regards
    Steve
    --
    Steve Holden +1 571 484 6266 +1 800 494 3119
    PyCon 2011 Atlanta March 9-17 http://us.pycon.org/
    See Python Video! http://python.mirocommunity.org/
    Holden Web LLC http://www.holdenweb.com/
     
    Steve Holden, Dec 8, 2010
    #4
  5. gst

    gst Guest

    Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    On 8 déc, 14:09, Steve Holden <> wrote:
    >
    > If you'd told us which IDE you were using we might have offered better
    > advice, but you seem to want to keep that a secret ("my IDE" tells us
    > nothing).
    >


    sorry it was totally bottom of my first message :

    > note: I'm using Eclipse 3.5.2 with pydev so (I don't really know how

    others IDE can handle this case) .

    now I realyze that my question has not a lot to do with python
    language itself and I should resubmit that probably on pydev forums/..

    but as I told I wanted to have some advises on the "good" (or not) use
    of this "way of doing" with python itself. You already explained me
    it's not the best so far.. and I agree although during (my) dev it
    helps ;) (but I do this only with very few classes that have lot of
    attributes/methods ofcourse).

    regards,

    greg.
     
    gst, Dec 8, 2010
    #5
  6. Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    gst wrote:
    > greg.
    >
    > nb: so this "hack" is only relevant during dev ; once the project
    > would be finished the "hack" could be removed (i.e : in class2 init I
    > would directly do : self.object1 = object1)
    >


    Expect some bugs then on the 'release' version.
    I'm not sure I understood everything you mentioned in your OP, however
    writing python code on class creation to make Eclipse completion 'work'
    is ... a very bad idea IMO.
    Just don't do it.

    quoting eclipse page:

    "Pydev [...] uses advanced type inference techniques to provide features
    such code completion and code analysis"

    I don't know exactly what's hidden behind this marketing stuff. Did you
    try to document your method with a markup language supported by Eclipse
    (if there is any)?

    class class2(object):
    def __init__(self, object1, *args, **kwargs):
    """blablabla

    @param object1: a L{class1} reference
    """
    self.object1 = object1

    The docstring would be the only way for an IDE to infer a argument type.
    Note that I used the epydoc markup language because it's the only one I
    know but it's unlikely supported by Eclipse. You better try
    reStructuredText.

    JM
     
    Jean-Michel Pichavant, Dec 8, 2010
    #6
  7. gst

    gst Guest

    Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    On 8 déc, 15:51, Jean-Michel Pichavant <> wrote:

    Hi,


    > gst wrote:
    > > nb: so this "hack" is only relevant during dev ; once the project
    > > would be finished the "hack" could be removed (i.e : in class2 init I
    > > would directly do : self.object1 = object1)

    >
    > Expect some bugs then on the 'release' version.


    even with the "assert(self is _self)" in the "if _self:" condition in
    the new and/or init methods of the class1 ?


    > I'm not sure I understood everything you mentioned in your OP, however
    > writing python code on class creation to make Eclipse completion 'work'
    > is ... a very bad idea IMO.
    > Just don't do it.


    yes I'm trying to find better ways for having completion works out of
    the box in this case.. (that would be a high help to my opinion).


    > quoting eclipse page:
    >
    > "Pydev [...] uses advanced type inference techniques to provide features
    > such code completion and code analysis"
    >
    > I don't know exactly what's hidden behind this marketing stuff. Did you
    > try to document your method with a markup language supported by Eclipse
    > (if there is any)?


    pydev completion apparently is restricted (as far as i see) to some
    very specific cases (basically it works when you import a module at
    top of another one and that you instantiate objects from the imported
    module within the init methods of the classes of the module which is
    importing the other one (but so "circular" references created on some
    objects from the first module (the one imported) in the second one
    won't have the completion working for them (that's what I see)).


    > class class2(object):
    >     def __init__(self, object1, *args, **kwargs):
    >     """blablabla
    >
    >     @param object1: a L{class1} reference
    >     """
    >     self.object1 = object1
    >
    > The docstring would be the only way for an IDE to infer a argument type.
    > Note that I used the epydoc markup language because it's the only one I
    > know but it's unlikely supported by Eclipse. You better try
    > reStructuredText.
    >
    > JM


    well, I'm not very used to docstrings in fact but it's effectively a
    way to achieve this ; well certainly better than doing this kind of
    "ugly" hack so.
    I'll have a try at that.

    Thanks for your reply,

    regards,

    greg.
     
    gst, Dec 8, 2010
    #7
  8. gst

    gst Guest

    Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    On 8 déc, 16:20, gst <> wrote:
    > On 8 déc, 15:51, Jean-Michel Pichavant <> wrote:
    >
    > Hi,
    >
    > > gst wrote:
    > > > nb: so this "hack" is only relevant during dev ; once the project
    > > > would be finished the "hack" could be removed (i.e : in class2 init I
    > > > would directly do : self.object1 = object1)

    >
    > > Expect some bugs then on the 'release' version.

    >
    > even with the "assert(self is _self)" in the "if _self:" condition in
    > the new and/or  init methods of the class1 ?


    note: I don't know if that matters but I'm using __slots__ in this
    class1 ..

    greg.
     
    gst, Dec 8, 2010
    #8
  9. gst

    gst Guest

    Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    On 8 déc, 16:20, gst <> wrote:

    > even with the "assert(self is _self)" in the "if _self:" condition in
    > the new and/or  init methods of the class1 ?


    damn : in the init method only of course..
     
    gst, Dec 8, 2010
    #9
  10. Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?


    >> quoting eclipse page:
    >>
    >> "Pydev [...] uses advanced type inference techniques to provide features
    >> such code completion and code analysis"
    >>
    >> I don't know exactly what's hidden behind this marketing stuff. Did you
    >> try to document your method with a markup language supported by Eclipse
    >> (if there is any)?
    >>

    >
    > pydev completion apparently is restricted (as far as i see) to some
    > very specific cases (basically it works when you import a module at
    > top of another one and that you instantiate objects from the imported
    > module within the init methods of the classes of the module which is
    > importing the other one (but so "circular" references created on some
    > objects from the first module (the one imported) in the second one
    > won't have the completion working for them (that's what I see)).
    >


    To understand what is possible and what is not, you simply need to think
    like a completion feature.

    case 1:
    self.object = MyClass()

    self.object is a MyClass instance, easy stuff I need to parse MyClass
    and get the list of attributes/method

    case 2:

    self.object = object1

    There may be simply no way to know the type/class of object1, object1
    could be even of an inconsistent type, sometimes None, int, MyClass, who
    knows ?
    Your trick worked because you found a way to write self.object =
    MyClass(object1) and getting barely the same effect than self.object =
    object1.

    I'm still thinking that's it's a bad idea. In any case your IDE can
    still be fooled by the dynamic of python (like pylint is)

    How to write python code then ? Well I guess most people either knows
    all the attributes by heart, have the documentation, or a split screen
    to the class definition, possibly folded using advanced text editor
    feature. You need to forget about code completion with any dynamic
    language. I also sometimes use an ipython shell to instanciate objects
    and inspect their attributes.

    JM
     
    Jean-Michel Pichavant, Dec 8, 2010
    #10
  11. gst

    nn Guest

    Re: use of __new__ to permit "dynamic" completion within (any?) IDE ?

    On Dec 7, 10:52 am, gst <> wrote:
    > Hi,
    >
    > I met a situation where I was passing an object created in/with an
    > upper level module class to a lower level module class' instance in
    > one of its __init__ argument and saving a ref of the upper object in
    > that lower level class' new instance.
    >
    > But in my IDE I want the completion to also work from within the lower
    > level module when it's refering to the object passed from the upper
    > level:
    >
    > Well, I'm sure I'm not very clear in my terms (and probably a bit long
    > in the sentence) so here it is in code:
    >
    > files:
    > module1.py
    > subpackage/module2.py
    >
    > file module1.py:
    >
    > from subpackage.module2 import class2
    >
    > class class1(object):
    >
    >     def __new__(cls, _self=None, *args, **kwargs):
    >         if _self:  ## we've been passed an instance already
    > initialyzed
    >                      ## so directly return it instead of creating a
    > new object.
    >             return _self
    >         return object.__new__(cls)
    >
    >     def __init__(self, _self=None, *args, **kwargs):
    >         if _self:  ## we've been passed an instance already
    > initialyzed
    >                      ## so directly returns
    >             ## assert(self is _self) ?
    >             return
    >         self.object2 = class2(object1=self, "blip", "blop")
    >         # others init
    >
    > file module2.py:
    >
    > class class2(object):
    >
    >     def __init__(self, object1, *args, **kwargs):
    >
    >         from ..module1 import class1
    >
    >         self.object1 = class1(_self=object1)    ## instead of:
    > self.object1 = object1
    >
    >     ## others functions and/or init..
    >     ## where  now I've completion working on self.object1 :
    >     ## if I add(or remove) fields/methods in module1 (and save) then
    >     ## I have them available(or disappeared) in the completion when
    > executed from this submodule.
    >     ## This ofcourse permits to save me of remembering all of the
    > class1 attributes/methods when I'm working with self.object1 from
    > within one of class2 methods.
    >
    > What do you think of this ?
    >
    > I guess there can be others ways of doing this..  ?
    >
    > Thanks,
    >
    > Regards,
    >
    > Greg.
    >
    > note: I'm using Eclipse 3.5.2 with pydev so (I don't really know how
    > others IDE can handle this case) .


    You might want to test with Wing IDE http://wingware.com/
    using the 10 day trial version to see if that works just for kicks.
    I've heard it has the best autocomplete of all IDEs out there.
     
    nn, Dec 8, 2010
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Steven Bethard
    Replies:
    2
    Views:
    459
    Steven Bethard
    Feb 16, 2005
  2. BillJosephson
    Replies:
    4
    Views:
    291
    BillJosephson
    Nov 6, 2006
  3. Lighter
    Replies:
    3
    Views:
    406
    noone
    Aug 10, 2006
  4. Replies:
    0
    Views:
    398
  5. ziggs
    Replies:
    2
    Views:
    189
    Henry Robinson
    Apr 21, 2004
Loading...

Share This Page