design question, metaclasses?

[Darren Dale]

I am working on a project that provides a high level interface to hdf5
files by implementing a thin wrapper around h5py. I would like to
generalize the project so the same API can be used with other formats,
like netcdf or ascii files. The format specific code exists in File,
Group and Dataset classes, which I could reimplement for each format.
But there are other classes deriving from Group and Dataset which do
not contain any format-specific code, and I would like to find a way
to implement the functionality once and apply uniformly across
supported formats. This is really abstract, but I was thinking of
something along the lines of:

format1.Group # implementation of group in format1
format2.Group # ...
Base.DerivedGroup # base implementation of DerivedGroup, not directly
useful
format1.DerivedGroup = Base.DerivedGroup(format1.Group) # useful
format2.DerivedGroup = Base.DerivedGroup(format2.Group) # useful

Could anyone please offer a comment, is this an appropriate use of
metaclassing, or is there maybe an easier/better alternative?

 

[Darren Dale]

I am working on a project that provides a high level interface to hdf5
files by implementing a thin wrapper around h5py. I would like to
generalize the project so the same API can be used with other formats,
like netcdf or ascii files. The format specific code exists in File,
Group and Dataset classes, which I could reimplement for each format.
But there are other classes deriving from Group and Dataset which do
not contain any format-specific code, and I would like to find a way
to implement the functionality once and apply uniformly across
supported formats. This is really abstract, but I was thinking of
something along the lines of:

format1.Group # implementation of group in format1
format2.Group # ...
Base.DerivedGroup # base implementation of DerivedGroup, not directly
useful
format1.DerivedGroup = Base.DerivedGroup(format1.Group) # useful
format2.DerivedGroup = Base.DerivedGroup(format2.Group) # useful

Could anyone please offer a comment, is this an appropriate use of
metaclassing, or is there maybe an easier/better alternative?

I don't fully understand metaclasses, but I think I have convinced
myself that they are not what I was looking for. I think this will do
what I want it to:

class Group1(object):

def origin(self):
return "Group1"


class Group2(object):

def origin(self):
return "Group2"


def _SubGroup(superclass):

class SubGroup(superclass):
pass

return SubGroup


SubGroup = _SubGroup(Group2)
sub_group = SubGroup()

print sub_group.origin()

 

[Aaron Brady]

_
_

class Group1(object):

    def origin(self):
        return "Group1"

class Group2(object):

    def origin(self):
        return "Group2"

def _SubGroup(superclass):

    class SubGroup(superclass):
        pass

    return SubGroup

SubGroup = _SubGroup(Group2)
sub_group = SubGroup()

print sub_group.origin()

You can create new types in one statement:

SubGroup= type( "SubGroup", ( BaseGroup, ), { } )

 

[Darren Dale]

You can create new types in one statement:

SubGroup= type( "SubGroup", ( BaseGroup, ), { } )

But how can I implement the *instance* behavior of SubGroup with this
example? In my original example:

format1.Group # implementation of group in format1
format2.Group # implementation of group in format2
Base.DerivedGroup # base implementation of DerivedGroup, must subclass
a group
format1.DerivedGroup = Base.DerivedGroup(format1.Group) # useful
format2.DerivedGroup = Base.DerivedGroup(format2.Group) # useful

I'm trying to achieve uniform behavior of my derived groups across
supported formats. My derived groups are abstracted such that they do
not need to be reimplemented for each format, I only need to implement
Group for each format. This is a real mind bender for me, even my
factory function gets hairy because I have additional classes that
derive from DerivedGroup. Maybe what I need is the ability to provide
context at import time, is that possible?

 

[Kay Schluehr]

I am working on a project that provides a high level interface to hdf5
files by implementing a thin wrapper around h5py.
I would like to
generalize the project so the same API can be used with other formats,
like netcdf or ascii files. The format specific code exists in File,
Group and Dataset classes, which I could reimplement for each format.
But there are other classes deriving from Group and Dataset which do
not contain any format-specific code, and I would like to find a way
to implement the functionality once and apply uniformly across
supported formats.

Seems like you are doing it wrong. The classical OO approach is to add
more details / refining classes in subclasses instead of doing it the
other way round and derive the less specific classes from the more
specific ones.

 

[Darren Dale]

Seems like you are doing it wrong. The classical OO approach is to add
more details / refining classes in subclasses instead of doing it the
other way round and derive the less specific classes from the more
specific ones.

I think I am following the classical OO approach, refining details in
subclasses. I just want a given subclass implementation describing a
complex dataset to be able to work on top of multiple hierarchical
file formats (like NetCDF or HDF5) by deriving from either NetCDF or
HDF5 base classes that have an identical API. Those base classes
encapsulate all the format-specific details, the subclasses allow a
uniform image to be handled differently than a nonuniform image with a
mask (for example). Maybe I should be delegating rather than
subclassing.

 

[Aaron Brady]

I think I am following the classical OO approach, refining details in
subclasses. I just want a given subclass implementation describing a
complex dataset to be able to work on top of multiple hierarchical
file formats (like NetCDF or HDF5) by deriving from either NetCDF or
HDF5 base classes that have an identical API. Those base classes
encapsulate all the format-specific details, the subclasses allow a
uniform image to be handled differently than a nonuniform image with a
mask (for example). Maybe I should be delegating rather than
subclassing.

You made me think of the 'mixin' design pattern, also known as
multiple inheritance. The subclass derives from multiple classes.
One is the domain-specific description of the data class; the other is
the file format.

class NetCDF_DataA_Writer( NetCDF_Writer, DataA_Description ):
pass

The two classes can't have conflicting method names, however; or
you'll have to resolve them by hand, or assign them unique names. In
this example, the subclass would favor the first super class, and you
could access the second super class with special names. It kind of
makes uniform access unreliable.

class NetCDF_DataA_Writer( NetCDF_Writer, DataA_Description ):
data_write= DataA_Description.write
file_write= NetCDF_Writer.write

The standard library has a couple of examples of this, as I recall,
such as the UDPServerMixin (sp).

 

[Piet van Oostrum]

DD> I don't fully understand metaclasses, but I think I have convinced
DD> myself that they are not what I was looking for. I think this will do
DD> what I want it to:

DD> class Group1(object):

DD> def origin(self):
DD> return "Group1"

DD> class Group2(object):

DD> def origin(self):
DD> return "Group2"

DD> def _SubGroup(superclass):

DD> class SubGroup(superclass):
DD> pass

DD> return SubGroup

DD> SubGroup = _SubGroup(Group2)

What is the difference of this with:

class SubGroup(Group2):
pass
?

DD> sub_group = SubGroup()

DD> print sub_group.origin()

From your description I find it very difficult to understand what you
want. But I have some feeling that multiple inheritance may be something
you could use.