A lazy-committing database object with curry?

[Tim Lesher]

I'm writing a database helper class that represents records in a SQL
database as objects. I want to be able to instantiate the objects
(from the database), play with their values locally, then lazily
commit the changes all at once (via an explicit commit call). Other
than the call to Commit(), I don't want clients of this class to have
to think about the fact that there's a database behind it all--there are
some interesting and non-obvious dependencies on the particular values
that are committed together, and I want to hide that in the Commit() call.

I'm going to have a number of these, so I wanted to come up with a
solution that's short on glue code and can easily be aggregated into
all the classes that represent record types. This is what I've come
up with:

# Simple currying class, no kwargs
class curry:
def __init__(self, fun, *args):
self.fun = fun
self.pending = args[:]
def __call__(self, *args):
return self.fun(*(self.pending + args))


# Simple table: two columns, "title" and "description"
class SomeRecordType(object):
def __init__(self, title, description):
self.__modifications = {}

# Avoid triggering propset on init
self.__dict__['title'] = title
self.__dict__['description'] = description

def __getValue(name, self):
try:
return self.__modifications[name]
except KeyError:
return self.__dict__[name]

def __setValue(name, self, newValue):
self.modifications[name] = newValue

name = property(curry(__getValue, 'title'),
curry(__setValue, 'title'))
description = property(curry(__getValue, 'description'),
curry(__setValue, 'description'))

def Commit(self):
if self.modifications != {}:
# - snip - Do database commit and clear modifications
self.__dict__.update(self.modifications)
self.modifications = {}

So I can do this:

foo = myDb.LookupTitleRecord('some title')
print foo.description # 'foo'
foo.description = 'bar' # not updated in the DB yet
print foo.description # 'bar'
foo.Commit() # now updated in the DB


Are there any pitfalls to doing this? Am I being dazzled by the shiny
new toy that is currying? Is there another simple solution, or a
refinement of this one, that I'm not seeing?

Thanks.

 

[Jp Calderone]

I'm writing a database helper class that represents records in a SQL
database as objects. I want to be able to instantiate the objects
(from the database), play with their values locally, then lazily
commit the changes all at once (via an explicit commit call). Other
than the call to Commit(), I don't want clients of this class to have
to think about the fact that there's a database behind it all--there are
some interesting and non-obvious dependencies on the particular values
that are committed together, and I want to hide that in the Commit() call.

Sounds familiar...

I'm going to have a number of these, so I wanted to come up with a
solution that's short on glue code and can easily be aggregated into
all the classes that represent record types. This is what I've come
up with:

# Simple currying class, no kwargs
class curry:
def __init__(self, fun, *args):
self.fun = fun
self.pending = args[:]
def __call__(self, *args):
return self.fun(*(self.pending + args))

You don't need to copy args, Python has done that for you already.

# Simple table: two columns, "title" and "description"
class SomeRecordType(object):
def __init__(self, title, description):
self.__modifications = {}

# Avoid triggering propset on init
self.__dict__['title'] = title
self.__dict__['description'] = description

Are you sure? How will the initial values be committed to the database,
if you don't catch them in the modifications dict? I guess you might be
handling initial values somewhere else, in code you didn't post...

def __getValue(name, self):
try:
return self.__modifications[name]
except KeyError:
return self.__dict__[name]

def __setValue(name, self, newValue):
self.modifications[name] = newValue

name = property(curry(__getValue, 'title'),
curry(__setValue, 'title'))
description = property(curry(__getValue, 'description'),
curry(__setValue, 'description'))

Nift. You surely don't use the full capabilities of currying here, but
you are incurring a double function call overhead on every setattr now.
Here's another way to do it:

def makeModificationFunctions(name):
def modget(self, value):
try:
return self.__modifications[name]
except KeyError:
return self.__dict__[name]
def modset(self, value):
self.__modifications[name] = value
return modget, modset
title = property(*makeModificationFunctions('title'))
description = property(*makeModificationFunctions('description'))

Of course, you can go one step better and use a metaclass:

class MetaRecordType(type):
def __new__(metaclass, name, bases, ns):
for k, v in ns.items():
if v is makeModificationFunctions
ns[k] = property(*v(k))
return type.__new__(metaclass, name, bases, ns)

and now in the class definition simply say:

__metaclass__ = MetaRecordType

title = makeModificationFunctions
description = makeModificationFunctions

def Commit(self):
if self.modifications != {}:

This can be just "if self.__modifications:"

# - snip - Do database commit and clear modifications
self.__dict__.update(self.modifications)
self.modifications = {}

So I can do this:

foo = myDb.LookupTitleRecord('some title')
print foo.description # 'foo'
foo.description = 'bar' # not updated in the DB yet
print foo.description # 'bar'
foo.Commit() # now updated in the DB


Are there any pitfalls to doing this? Am I being dazzled by the shiny
new toy that is currying? Is there another simple solution, or a
refinement of this one, that I'm not seeing?

One possible problem is failed commits. If you're doing commits
transactionally, and the transaction fails, your in memory objects will
retain their now-inconsistent values while the database remains unupdated.
If you're not using transactions.. well that's a whole other problem

You may want to look at xsdb and atop. Neither uses SQL, but both have
developed pretty heavily on some of the ideas you're tinkering around with.

Jp

 

[Tim Lesher]

On Mon, Jan 19, 2004 at 08:45:45AM -0800, Tim Lesher wrote:
Are you sure? How will the initial values be committed to the database,
if you don't catch them in the modifications dict? I guess you might be
handling initial values somewhere else, in code you didn't post...

Yes... this class will only be working on rows that already exist in
the database--in other words, it only has to cope with updates and
deletes, not inserts.

Nift. You surely don't use the full capabilities of currying here, but
you are incurring a double function call overhead on every setattr now.
Here's another way to do it:

def makeModificationFunctions(name):
def modget(self, value):
try:
return self.__modifications[name]
except KeyError:
return self.__dict__[name]
def modset(self, value):
self.__modifications[name] = value
return modget, modset
title = property(*makeModificationFunctions('title'))
description = property(*makeModificationFunctions('description'))

Aha. This is what I was really looking for. I tried it with lambdas
at first and got bitten by the expressions-only limitation. Coming
from a C++ background, I keep forgetting about local functions...

Of course, you can go one step better and use a metaclass:

Wow. That's even shinier than the currying.

This can be just "if self.__modifications:"

I know... explicit better than implicit and all.

One possible problem is failed commits. If you're doing commits
transactionally, and the transaction fails, your in memory objects will
retain their now-inconsistent values while the database remains unupdated.
If you're not using transactions.. well that's a whole other problem

Yep, failed commits are handled in the 'snipped' piece of code when
they're handled at all--initially this is running against a
transactionless database, but the place is there to handle
transactions--that's why I don't clear the modifications until after
the update is made.

You may want to look at xsdb and atop. Neither uses SQL, but both have
developed pretty heavily on some of the ideas you're tinkering around with.

Thanks; I will.