Class extension confusion :(

Discussion in 'Python' started by r0g, Nov 10, 2010.

  1. r0g

    r0g Guest

    I have a subclass of BaseHHTPRequestHandler which uses a dictonary
    "paths" and a function "api_call" which are defined in the main
    namespace of the module. I'd rather I was able to pass these object to
    the constructor and store them as data attributes "self.paths" and
    "self.api_call" but I'm not sure how to do that properly. My
    understanding is that one may extend a constructor by defining it's
    __init__ method, calling the parents constructor and then adding ones
    own attributes to taste. What I don't understand is where or how I am
    supposed to get these extra constructor arguments into the class given
    that I don't instantiate it myself, it is seemingly instantiated by
    HTTPServer class that I pass it to e.g.

    httpd = HTTPServer(server_address, PlainAJAXRequestHandler)

    I wondered if I ought to instantiate an instance of
    PlainAJAXRequestHandler, set the attributes (either manually or by
    extending it's constructor) and pass that to HTTPServer but I figured it
    expects a class not an instance as it probably wants to spawn one
    instance for each request so that would be a non starter. Might I need
    to subclass HTTPServer, find the bit that instantiates the request
    handler and override that so it passes it's constructor more parameters?
    Right now I'm pretty confused, can somebody please tell me how I might
    accomplish this, what I'm failing to grasp or point me to the docs that
    explain it - I've spent the last hour or two plowing through docs to no
    avail, I guess it's a case of keyword ignorance on my part! Code follows...

    Thanks for reading!

    Roger.



    class PlainAJAXRequestHandler(BaseHTTPRequestHandler):

    paths = { "/": pages.main,
    "/jqtest/": pages.jqtest
    }

    def do_GET(self):

    # Handle JSON api calls
    if self.path[:6] == "/ajax?":
    getvars = urlparse.parse_qs( self.path[6:] )
    api_key = getvars[ "api" ][0]
    json_string = getvars[ "qry" ][0]
    json_object = json.loads( json_string )
    response = api_call( api_key, json_object )
    if response:
    self.send_response(200)
    self.send_header("Content-type", "application/json")
    self.end_headers()
    self.wfile.write( response )
    else:
    self.send_response(404)
    self.end_headers()
    return

    # Handle web pages
    try:
    page = self.paths[self.path]()
    except KeyError:
    self.send_response(404)
    self.end_headers()
    self.wfile.write( "404 - Document not found!" )
    return
    self.send_response(200)
    self.send_header("Content-type", "text/html")
    self.end_headers()
    self.wfile.write( page )
    return
     
    r0g, Nov 10, 2010
    #1
    1. Advertising

  2. r0g

    Peter Otten Guest

    r0g wrote:

    > I have a subclass of BaseHHTPRequestHandler which uses a dictonary
    > "paths" and a function "api_call" which are defined in the main
    > namespace of the module. I'd rather I was able to pass these object to
    > the constructor and store them as data attributes "self.paths" and
    > "self.api_call" but I'm not sure how to do that properly. My
    > understanding is that one may extend a constructor by defining it's
    > __init__ method, calling the parents constructor and then adding ones
    > own attributes to taste. What I don't understand is where or how I am
    > supposed to get these extra constructor arguments into the class given
    > that I don't instantiate it myself, it is seemingly instantiated by
    > HTTPServer class that I pass it to e.g.
    >
    > httpd = HTTPServer(server_address, PlainAJAXRequestHandler)
    >
    > I wondered if I ought to instantiate an instance of
    > PlainAJAXRequestHandler, set the attributes (either manually or by
    > extending it's constructor) and pass that to HTTPServer but I figured it
    > expects a class not an instance as it probably wants to spawn one
    > instance for each request so that would be a non starter. Might I need
    > to subclass HTTPServer, find the bit that instantiates the request
    > handler and override that so it passes it's constructor more parameters?
    > Right now I'm pretty confused, can somebody please tell me how I might
    > accomplish this, what I'm failing to grasp or point me to the docs that
    > explain it - I've spent the last hour or two plowing through docs to no
    > avail, I guess it's a case of keyword ignorance on my part! Code
    > follows...


    Try passing a factory function instead of a class. Untested:

    from functools import partial

    class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
    def __init__(self, api_call, paths, *args, **kw):
    BaseHTTPRequestHandler.__init__(self, *args, **kw)
    self.api_call = api_call
    self.paths = paths

    paths = ...
    api_call = ...

    httpd = HTTPServer(
    server_address,
    partial(PlainAJAXRequestHandler, paths, api_call))
     
    Peter Otten, Nov 10, 2010
    #2
    1. Advertising

  3. r0g

    r0g Guest

    On 10/11/10 09:52, Peter Otten wrote:
    > r0g wrote:
    >
    >> I have a subclass of BaseHHTPRequestHandler which uses a dictonary
    >> "paths" and a function "api_call" which are defined in the main
    >> namespace of the module. I'd rather I was able to pass these object to
    >> the constructor and store them as data attributes "self.paths" and
    >> "self.api_call" but I'm not sure how to do that properly. My
    >> understanding is that one may extend a constructor by defining it's
    >> __init__ method, calling the parents constructor and then adding ones
    >> own attributes to taste. What I don't understand is where or how I am
    >> supposed to get these extra constructor arguments into the class given
    >> that I don't instantiate it myself, it is seemingly instantiated by
    >> HTTPServer class that I pass it to e.g.
    >>
    >> httpd = HTTPServer(server_address, PlainAJAXRequestHandler)
    >>
    >> I wondered if I ought to instantiate an instance of
    >> PlainAJAXRequestHandler, set the attributes (either manually or by
    >> extending it's constructor) and pass that to HTTPServer but I figured it
    >> expects a class not an instance as it probably wants to spawn one
    >> instance for each request so that would be a non starter. Might I need
    >> to subclass HTTPServer, find the bit that instantiates the request
    >> handler and override that so it passes it's constructor more parameters?
    >> Right now I'm pretty confused, can somebody please tell me how I might
    >> accomplish this, what I'm failing to grasp or point me to the docs that
    >> explain it - I've spent the last hour or two plowing through docs to no
    >> avail, I guess it's a case of keyword ignorance on my part! Code
    >> follows...

    >
    > Try passing a factory function instead of a class. Untested:
    >
    > from functools import partial
    >
    > class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
    > def __init__(self, api_call, paths, *args, **kw):
    > BaseHTTPRequestHandler.__init__(self, *args, **kw)
    > self.api_call = api_call
    > self.paths = paths
    >
    > paths = ...
    > api_call = ...
    >
    > httpd = HTTPServer(
    > server_address,
    > partial(PlainAJAXRequestHandler, paths, api_call))
    >



    Great, that looks ideal Peter, thanks very much :)

    One more question quick question if you have time... I actually subclass
    the class in question later on to provide an SSL secured connection, is
    it possible (or even advisable) to do the following to spare me having
    to modify the derived classes init (and any subsequent classes' inits)
    too?...

    PlainAJAXRequestHandler = partial(PlainAJAXRequestHandler, paths, api_call))

    ^??? Remap name to new extended class

    class SecureAJAXRequestHandler(PlainAJAXRequestHandler):
    def setup(self):
    self.connection = self.request
    self.rfile = socket._fileobject(self.request, "rb", self.rbufsize)
    self.wfile = socket._fileobject(self.request, "wb", self.wbufsize)

    ^??? therefore no init needed?


    Cheers,

    Roger
     
    r0g, Nov 11, 2010
    #3
  4. r0g

    Peter Otten Guest

    r0g wrote:

    > On 10/11/10 09:52, Peter Otten wrote:
    >> r0g wrote:
    >>
    >>> I have a subclass of BaseHHTPRequestHandler which uses a dictonary
    >>> "paths" and a function "api_call" which are defined in the main
    >>> namespace of the module. I'd rather I was able to pass these object to
    >>> the constructor and store them as data attributes "self.paths" and
    >>> "self.api_call" but I'm not sure how to do that properly. My
    >>> understanding is that one may extend a constructor by defining it's
    >>> __init__ method, calling the parents constructor and then adding ones
    >>> own attributes to taste. What I don't understand is where or how I am
    >>> supposed to get these extra constructor arguments into the class given
    >>> that I don't instantiate it myself, it is seemingly instantiated by
    >>> HTTPServer class that I pass it to e.g.
    >>>
    >>> httpd = HTTPServer(server_address, PlainAJAXRequestHandler)
    >>>
    >>> I wondered if I ought to instantiate an instance of
    >>> PlainAJAXRequestHandler, set the attributes (either manually or by
    >>> extending it's constructor) and pass that to HTTPServer but I figured it
    >>> expects a class not an instance as it probably wants to spawn one
    >>> instance for each request so that would be a non starter. Might I need
    >>> to subclass HTTPServer, find the bit that instantiates the request
    >>> handler and override that so it passes it's constructor more parameters?
    >>> Right now I'm pretty confused, can somebody please tell me how I might
    >>> accomplish this, what I'm failing to grasp or point me to the docs that
    >>> explain it - I've spent the last hour or two plowing through docs to no
    >>> avail, I guess it's a case of keyword ignorance on my part! Code
    >>> follows...

    >>
    >> Try passing a factory function instead of a class. Untested:
    >>
    >> from functools import partial
    >>
    >> class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
    >> def __init__(self, api_call, paths, *args, **kw):
    >> BaseHTTPRequestHandler.__init__(self, *args, **kw)
    >> self.api_call = api_call
    >> self.paths = paths
    >>
    >> paths = ...
    >> api_call = ...
    >>
    >> httpd = HTTPServer(
    >> server_address,
    >> partial(PlainAJAXRequestHandler, paths, api_call))
    >>

    >
    >
    > Great, that looks ideal Peter, thanks very much :)
    >
    > One more question quick question if you have time... I actually subclass
    > the class in question later on to provide an SSL secured connection, is
    > it possible (or even advisable) to do the following to spare me having
    > to modify the derived classes init (and any subsequent classes' inits)
    > too?...
    >
    > PlainAJAXRequestHandler = partial(PlainAJAXRequestHandler, paths,
    > api_call))
    >
    > ^??? Remap name to new extended class
    >
    > class SecureAJAXRequestHandler(PlainAJAXRequestHandler):


    That's not possible; you are trying to subclass a function.
    What you can do (again untested):

    def make_handler_class(paths, api_call):
    class MyHandler(BaseHTTPRequestHandler):
    paths = paths
    api_call = staticmethod(api_call)
    return MyHandler

    PlainAJAXRequestHandler = make_handler_class(paths, api_call)

    class SecureAJAXRequestHandler(PlainAJAXRequestHandler):
    ...

    Peter
     
    Peter Otten, Nov 11, 2010
    #4
  5. r0g

    r0g Guest

    On 10/11/10 09:52, Peter Otten wrote:
    > class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
    > def __init__(self, api_call, paths, *args, **kw):
    > BaseHTTPRequestHandler.__init__(self, *args, **kw)
    > self.api_call = api_call
    > self.paths = paths



    Hmm, the plot thickens! I always thought you had to call the parent
    constructor first (as above) when extending a constructor (not that I've
    had occasion to do that in a long time), but it turns out when I do this
    the lines below it never get executed and when I move them above that
    line they seem to work fine so it appears I was wrong about that. I've
    tried typing many variants of "python class extend constructor" into
    google over the last few days but I'm damned if I can find the docs
    explaining this. I'm sure I found them several years back when I first
    too up python, maybe by google-fu is on the wane!

    Anyway, that's not my main question, this is... The extra names that I
    pass to functools.partial seem to be bound permanently into the
    namespace of my class now i.e. I can reference them as 'api_call' and
    'paths' anywhere in the classes' methods as opposed to having to assign
    them in the constructor and reference them as 'self.api_call' and
    'self.paths'. I'm not 100% how that's working but in practical terms it
    suggests to two lines assigning those names to data attributes are
    redundant as I can access them anywhere anyway. Indeed, I've commented
    them out and my app still seems to work fine so...

    Question A) Are there any good reasons why I shouldn't just do that?
    (other than B!)

    Question B) The only reason I can think of so far is that I don't have a
    clear picture of how those names came to end up in that scope, it seems
    very convenient but I'm worried it's black magic of some sort! Could
    anyone explain or point me to the right docs please?

    Please shout if you would like to see the source.

    Thanks muchly :)


    Roger.
     
    r0g, Nov 11, 2010
    #5
  6. r0g

    Peter Otten Guest

    r0g wrote:

    > On 10/11/10 09:52, Peter Otten wrote:
    >> class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
    >> def __init__(self, api_call, paths, *args, **kw):
    >> BaseHTTPRequestHandler.__init__(self, *args, **kw)
    >> self.api_call = api_call
    >> self.paths = paths

    >
    >
    > Hmm, the plot thickens! I always thought you had to call the parent
    > constructor first (as above) when extending a constructor (not that I've
    > had occasion to do that in a long time), but it turns out when I do this
    > the lines below it never get executed and when I move them above that
    > line they seem to work fine so it appears I was wrong about that. I've


    I didn't believe you until I had a look into the source. The meat is in
    SocketServer.py:

    class BaseRequestHandler:

    [snip]

    def __init__(self, request, client_address, server):
    self.request = request
    self.client_address = client_address
    self.server = server
    try:
    self.setup()
    self.handle()
    self.finish()
    finally:
    sys.exc_traceback = None # Help garbage collection

    def setup(self):
    pass

    def handle(self):
    pass

    def finish(self):
    pass


    As you can see this doesn't use __init__() just to set up the instance, it
    makes it the only method that is called by client code.

    That's an unusual design decision, to say the least.

    > tried typing many variants of "python class extend constructor" into
    > google over the last few days but I'm damned if I can find the docs
    > explaining this. I'm sure I found them several years back when I first
    > too up python, maybe by google-fu is on the wane!
    >
    > Anyway, that's not my main question, this is... The extra names that I
    > pass to functools.partial seem to be bound permanently into the
    > namespace of my class now i.e. I can reference them as 'api_call' and
    > 'paths' anywhere in the classes' methods as opposed to having to assign
    > them in the constructor and reference them as 'self.api_call' and
    > 'self.paths'. I'm not 100% how that's working but in practical terms it
    > suggests to two lines assigning those names to data attributes are
    > redundant as I can access them anywhere anyway. Indeed, I've commented
    > them out and my app still seems to work fine so...
    >
    > Question A) Are there any good reasons why I shouldn't just do that?
    > (other than B!)


    No. Use either

    class B(A):
    path = ...

    or

    class B(A):
    def __init__(self, path, *args, **kw):
    self.path = path
    A.__init__(self, *args, **kw)

    not both.

    > Question B) The only reason I can think of so far is that I don't have a
    > clear picture of how those names came to end up in that scope, it seems
    > very convenient but I'm worried it's black magic of some sort! Could
    > anyone explain or point me to the right docs please?


    Python looks for attributes in the instance first, and then in the class as
    a fallback. You only need to put them in the instance if you expect that you
    want a different value for every instance.

    Peter
     
    Peter Otten, Nov 11, 2010
    #6
  7. r0g

    r0g Guest

    On 11/11/10 09:34, Peter Otten wrote:
    > r0g wrote:
    >
    >> Question B) The only reason I can think of so far is that I don't have a
    >> clear picture of how those names came to end up in that scope, it seems
    >> very convenient but I'm worried it's black magic of some sort! Could
    >> anyone explain or point me to the right docs please?

    >
    > Python looks for attributes in the instance first, and then in the class as
    > a fallback. You only need to put them in the instance if you expect that you
    > want a different value for every instance.
    >
    > Peter
    >



    Thanks Peter, I think I understand now, I'll quickly explain my picture
    of things though so you or some other denizens of the mighty
    comp.lang.python can correct me if I'm wrong!...

    I can access the parameters I pass to __init__ within the classes'
    methods without using self. as these methods are run within the
    constructor itself and are therefore within it's local scope.

    That also explains why I had to call the constructor _after_ creating
    the new data attributes to have them be included. My initial belief that
    one has to call the parent constructor as the first action in the
    extended constructor is not technically valid, in most cases it can be
    called at any point but in situations like the above it can mess things
    up so I ought to put it at the end of my constructors for it to always
    work. Are there any drawbacks to calling it last or is that how it is
    supposed to work?

    The BaseHTTPRequestHandler seemed curiously constructed to me at first
    too but I think I can see why now... as it's a handler it's only ever
    meant to be a transient thing and it requires no external parameters
    (unless you're me!) as it's always called by HTTPServer which has all
    the info necessary so there's be no sense requiring users to create an
    instance then call a method to get the servers response, you may as well
    just bundle it all into one. Also as it might be called hundreds or
    thousands of times in quick succession it would be important to have it
    finish and get garbage collected.

    I also see that I ought to at least assign these extended params to data
    attributes and access them via self. like I would normally do as I can't
    guarantee that implementation of BaseHTTPREquestHandler will remain
    constant over time. If I understand correctly it may also be possible
    (and more efficient) to use setattr() to inject the parameters I want
    into the class as class attributes before use, rather than assigning
    them to data attributes every time I instantiate an new instance.

    Actually looking at the code from BaseRequestHandler it seems I may have
    overlooked the proper way of associating a callback function anyway. It
    looks like I should just define self.handle() in my subclass so that's
    one less parameter to worry about, although I think I may still need to
    use the factory function method you showed me to get the "paths"
    dictionary in there.

    Thanks so much for all your help, I really appreciate it and, assuming
    I'm not totally wrong about all of the above, I find it reassuring that
    I'm not going mad!

    Cheers,

    Roger.
     
    r0g, Nov 11, 2010
    #7
  8. r0g

    r0g Guest

    On 11/11/10 19:34, r0g wrote:
    > On 11/11/10 09:34, Peter Otten wrote:
    >> r0g wrote:

    > If I understand correctly it may also be possible
    > (and more efficient) to use setattr() to inject the parameters I want
    > into the class as class attributes before use, rather than assigning
    > them to data attributes every time I instantiate an new instance.


    Ah, it's even simpler than that, turns out I've been significantly
    overcomplicating things, I can just set the class attributes by calling
    classname.attributename = whatever once at the start of my script. The
    one thing I might need to change as the server runs is the paths
    dictionary but dicts are mutable so that's no problem and I don't need
    to inject any data into the instances at all. I'm guessing I can
    subclass that easily now too - happy days!

    Well, far from wasted time it's been a very educational diversion, cheers :)


    Roger.
     
    r0g, Nov 11, 2010
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. E11
    Replies:
    1
    Views:
    4,786
    Thomas Weidenfeller
    Oct 12, 2005
  2. Anand
    Replies:
    3
    Views:
    898
    Tim Daneliuk
    Nov 8, 2003
  3. Christian Seberino
    Replies:
    3
    Views:
    1,188
    Christian Seberino
    Feb 5, 2004
  4. Steve
    Replies:
    3
    Views:
    26,997
    Steve
    Aug 25, 2006
  5. Replies:
    1
    Views:
    96
Loading...

Share This Page