Class extension confusion :(

[r0g]

I have a subclass of BaseHHTPRequestHandler which uses a dictonary
"paths" and a function "api_call" which are defined in the main
namespace of the module. I'd rather I was able to pass these object to
the constructor and store them as data attributes "self.paths" and
"self.api_call" but I'm not sure how to do that properly. My
understanding is that one may extend a constructor by defining it's
__init__ method, calling the parents constructor and then adding ones
own attributes to taste. What I don't understand is where or how I am
supposed to get these extra constructor arguments into the class given
that I don't instantiate it myself, it is seemingly instantiated by
HTTPServer class that I pass it to e.g.

httpd = HTTPServer(server_address, PlainAJAXRequestHandler)

I wondered if I ought to instantiate an instance of
PlainAJAXRequestHandler, set the attributes (either manually or by
extending it's constructor) and pass that to HTTPServer but I figured it
expects a class not an instance as it probably wants to spawn one
instance for each request so that would be a non starter. Might I need
to subclass HTTPServer, find the bit that instantiates the request
handler and override that so it passes it's constructor more parameters?
Right now I'm pretty confused, can somebody please tell me how I might
accomplish this, what I'm failing to grasp or point me to the docs that
explain it - I've spent the last hour or two plowing through docs to no
avail, I guess it's a case of keyword ignorance on my part! Code follows...

Thanks for reading!

Roger.



class PlainAJAXRequestHandler(BaseHTTPRequestHandler):

paths = { "/": pages.main,
"/jqtest/": pages.jqtest
}

def do_GET(self):

# Handle JSON api calls
if self.path[:6] == "/ajax?":
getvars = urlparse.parse_qs( self.path[6:] )
api_key = getvars[ "api" ][0]
json_string = getvars[ "qry" ][0]
json_object = json.loads( json_string )
response = api_call( api_key, json_object )
if response:
self.send_response(200)
self.send_header("Content-type", "application/json")
self.end_headers()
self.wfile.write( response )
else:
self.send_response(404)
self.end_headers()
return

# Handle web pages
try:
page = self.paths[self.path]()
except KeyError:
self.send_response(404)
self.end_headers()
self.wfile.write( "404 - Document not found!" )
return
self.send_response(200)
self.send_header("Content-type", "text/html")
self.end_headers()
self.wfile.write( page )
return

 

[Peter Otten]

I have a subclass of BaseHHTPRequestHandler which uses a dictonary
"paths" and a function "api_call" which are defined in the main
namespace of the module. I'd rather I was able to pass these object to
the constructor and store them as data attributes "self.paths" and
"self.api_call" but I'm not sure how to do that properly. My
understanding is that one may extend a constructor by defining it's
__init__ method, calling the parents constructor and then adding ones
own attributes to taste. What I don't understand is where or how I am
supposed to get these extra constructor arguments into the class given
that I don't instantiate it myself, it is seemingly instantiated by
HTTPServer class that I pass it to e.g.

httpd = HTTPServer(server_address, PlainAJAXRequestHandler)

I wondered if I ought to instantiate an instance of
PlainAJAXRequestHandler, set the attributes (either manually or by
extending it's constructor) and pass that to HTTPServer but I figured it
expects a class not an instance as it probably wants to spawn one
instance for each request so that would be a non starter. Might I need
to subclass HTTPServer, find the bit that instantiates the request
handler and override that so it passes it's constructor more parameters?
Right now I'm pretty confused, can somebody please tell me how I might
accomplish this, what I'm failing to grasp or point me to the docs that
explain it - I've spent the last hour or two plowing through docs to no
avail, I guess it's a case of keyword ignorance on my part! Code
follows...

Try passing a factory function instead of a class. Untested:

from functools import partial

class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
def __init__(self, api_call, paths, *args, **kw):
BaseHTTPRequestHandler.__init__(self, *args, **kw)
self.api_call = api_call
self.paths = paths

paths = ...
api_call = ...

httpd = HTTPServer(
server_address,
partial(PlainAJAXRequestHandler, paths, api_call))

 

[r0g]

Try passing a factory function instead of a class. Untested:

from functools import partial

class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
def __init__(self, api_call, paths, *args, **kw):
BaseHTTPRequestHandler.__init__(self, *args, **kw)
self.api_call = api_call
self.paths = paths

paths = ...
api_call = ...

httpd = HTTPServer(
server_address,
partial(PlainAJAXRequestHandler, paths, api_call))

Great, that looks ideal Peter, thanks very much

One more question quick question if you have time... I actually subclass
the class in question later on to provide an SSL secured connection, is
it possible (or even advisable) to do the following to spare me having
to modify the derived classes init (and any subsequent classes' inits)
too?...

PlainAJAXRequestHandler = partial(PlainAJAXRequestHandler, paths, api_call))

^??? Remap name to new extended class

class SecureAJAXRequestHandler(PlainAJAXRequestHandler):
def setup(self):
self.connection = self.request
self.rfile = socket._fileobject(self.request, "rb", self.rbufsize)
self.wfile = socket._fileobject(self.request, "wb", self.wbufsize)

^??? therefore no init needed?


Cheers,

Roger

 

[Peter Otten]

Great, that looks ideal Peter, thanks very much

One more question quick question if you have time... I actually subclass
the class in question later on to provide an SSL secured connection, is
it possible (or even advisable) to do the following to spare me having
to modify the derived classes init (and any subsequent classes' inits)
too?...

PlainAJAXRequestHandler = partial(PlainAJAXRequestHandler, paths,
api_call))

^??? Remap name to new extended class

class SecureAJAXRequestHandler(PlainAJAXRequestHandler):

That's not possible; you are trying to subclass a function.
What you can do (again untested):

def make_handler_class(paths, api_call):
class MyHandler(BaseHTTPRequestHandler):
paths = paths
api_call = staticmethod(api_call)
return MyHandler

PlainAJAXRequestHandler = make_handler_class(paths, api_call)

class SecureAJAXRequestHandler(PlainAJAXRequestHandler):
...

Peter

 

[r0g]

class PlainAJAXRequestHandler(BaseHTTPRequestHandler):
def __init__(self, api_call, paths, *args, **kw):
BaseHTTPRequestHandler.__init__(self, *args, **kw)
self.api_call = api_call
self.paths = paths

Hmm, the plot thickens! I always thought you had to call the parent
constructor first (as above) when extending a constructor (not that I've
had occasion to do that in a long time), but it turns out when I do this
the lines below it never get executed and when I move them above that
line they seem to work fine so it appears I was wrong about that. I've
tried typing many variants of "python class extend constructor" into
google over the last few days but I'm damned if I can find the docs
explaining this. I'm sure I found them several years back when I first
too up python, maybe by google-fu is on the wane!

Anyway, that's not my main question, this is... The extra names that I
pass to functools.partial seem to be bound permanently into the
namespace of my class now i.e. I can reference them as 'api_call' and
'paths' anywhere in the classes' methods as opposed to having to assign
them in the constructor and reference them as 'self.api_call' and
'self.paths'. I'm not 100% how that's working but in practical terms it
suggests to two lines assigning those names to data attributes are
redundant as I can access them anywhere anyway. Indeed, I've commented
them out and my app still seems to work fine so...

Question A) Are there any good reasons why I shouldn't just do that?
(other than B!)

Question B) The only reason I can think of so far is that I don't have a
clear picture of how those names came to end up in that scope, it seems
very convenient but I'm worried it's black magic of some sort! Could
anyone explain or point me to the right docs please?

Please shout if you would like to see the source.

Thanks muchly


Roger.

 

[Peter Otten]

Hmm, the plot thickens! I always thought you had to call the parent
constructor first (as above) when extending a constructor (not that I've
had occasion to do that in a long time), but it turns out when I do this
the lines below it never get executed and when I move them above that
line they seem to work fine so it appears I was wrong about that. I've

I didn't believe you until I had a look into the source. The meat is in
SocketServer.py:

class BaseRequestHandler:

[snip]

def __init__(self, request, client_address, server):
self.request = request
self.client_address = client_address
self.server = server
try:
self.setup()
self.handle()
self.finish()
finally:
sys.exc_traceback = None # Help garbage collection

def setup(self):
pass

def handle(self):
pass

def finish(self):
pass


As you can see this doesn't use __init__() just to set up the instance, it
makes it the only method that is called by client code.

That's an unusual design decision, to say the least.

tried typing many variants of "python class extend constructor" into
google over the last few days but I'm damned if I can find the docs
explaining this. I'm sure I found them several years back when I first
too up python, maybe by google-fu is on the wane!

Anyway, that's not my main question, this is... The extra names that I
pass to functools.partial seem to be bound permanently into the
namespace of my class now i.e. I can reference them as 'api_call' and
'paths' anywhere in the classes' methods as opposed to having to assign
them in the constructor and reference them as 'self.api_call' and
'self.paths'. I'm not 100% how that's working but in practical terms it
suggests to two lines assigning those names to data attributes are
redundant as I can access them anywhere anyway. Indeed, I've commented
them out and my app still seems to work fine so...

Question A) Are there any good reasons why I shouldn't just do that?
(other than B!)

No. Use either

class B(A):
path = ...

or

class B(A):
def __init__(self, path, *args, **kw):
self.path = path
A.__init__(self, *args, **kw)

not both.

Question B) The only reason I can think of so far is that I don't have a
clear picture of how those names came to end up in that scope, it seems
very convenient but I'm worried it's black magic of some sort! Could
anyone explain or point me to the right docs please?

Python looks for attributes in the instance first, and then in the class as
a fallback. You only need to put them in the instance if you expect that you
want a different value for every instance.

Peter

 

[r0g]

Python looks for attributes in the instance first, and then in the class as
a fallback. You only need to put them in the instance if you expect that you
want a different value for every instance.

Peter

Thanks Peter, I think I understand now, I'll quickly explain my picture
of things though so you or some other denizens of the mighty
comp.lang.python can correct me if I'm wrong!...

I can access the parameters I pass to __init__ within the classes'
methods without using self. as these methods are run within the
constructor itself and are therefore within it's local scope.

That also explains why I had to call the constructor _after_ creating
the new data attributes to have them be included. My initial belief that
one has to call the parent constructor as the first action in the
extended constructor is not technically valid, in most cases it can be
called at any point but in situations like the above it can mess things
up so I ought to put it at the end of my constructors for it to always
work. Are there any drawbacks to calling it last or is that how it is
supposed to work?

The BaseHTTPRequestHandler seemed curiously constructed to me at first
too but I think I can see why now... as it's a handler it's only ever
meant to be a transient thing and it requires no external parameters
(unless you're me!) as it's always called by HTTPServer which has all
the info necessary so there's be no sense requiring users to create an
instance then call a method to get the servers response, you may as well
just bundle it all into one. Also as it might be called hundreds or
thousands of times in quick succession it would be important to have it
finish and get garbage collected.

I also see that I ought to at least assign these extended params to data
attributes and access them via self. like I would normally do as I can't
guarantee that implementation of BaseHTTPREquestHandler will remain
constant over time. If I understand correctly it may also be possible
(and more efficient) to use setattr() to inject the parameters I want
into the class as class attributes before use, rather than assigning
them to data attributes every time I instantiate an new instance.

Actually looking at the code from BaseRequestHandler it seems I may have
overlooked the proper way of associating a callback function anyway. It
looks like I should just define self.handle() in my subclass so that's
one less parameter to worry about, although I think I may still need to
use the factory function method you showed me to get the "paths"
dictionary in there.

Thanks so much for all your help, I really appreciate it and, assuming
I'm not totally wrong about all of the above, I find it reassuring that
I'm not going mad!

Cheers,

Roger.

 

[r0g]

If I understand correctly it may also be possible
(and more efficient) to use setattr() to inject the parameters I want
into the class as class attributes before use, rather than assigning
them to data attributes every time I instantiate an new instance.

Ah, it's even simpler than that, turns out I've been significantly
overcomplicating things, I can just set the class attributes by calling
classname.attributename = whatever once at the start of my script. The
one thing I might need to change as the server runs is the paths
dictionary but dicts are mutable so that's no problem and I don't need
to inject any data into the instances at all. I'm guessing I can
subclass that easily now too - happy days!

Well, far from wasted time it's been a very educational diversion, cheers


Roger.