IndexedCatalog and ZEO

[Almad]

Hello,

I'm trying to use IndexedCatalog
[http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
[http://www.cherrypy.org] application.
As Zodb support access just from one Python thread, I must either use just
one CherryPy thread (slow), or use ZEO. However, if I understand it good,
when I use
from ZEO import ClientStorage

then ZEO is internally using FileStorage, so no IndexedCatalog is used.

Anyone knews, how to use IndexedCatalog with ZEO, or do I have to "patch"
ZEO for myself (or run this one thread)?

Thanks,
--
Lukas "Almad" Linhart

[:: http://www.almad.net/ ::]
[:: Humans are too complicated to be described with words. ::]
[:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]

 

[Leif K-Brooks]

Hello,

I'm trying to use IndexedCatalog
[http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
[http://www.cherrypy.org] application.
As Zodb support access just from one Python thread, I must either use just
one CherryPy thread (slow), or use ZEO. However, if I understand it good,
when I use
from ZEO import ClientStorage

then ZEO is internally using FileStorage, so no IndexedCatalog is used.

If you're using IndexedCatalog's Shelf class, just pass it a host/port
tuple instead of a path:

from IndexedCatalog.Shelf import Shelf
shelf = Shelf(('localhost', 1234), [Class1, Class2])

 

[Diez B. Roggisch]

I'm trying to use IndexedCatalog

[http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
[http://www.cherrypy.org] application.
As Zodb support access just from one Python thread, I must either use just
one CherryPy thread (slow), or use ZEO. However, if I understand it good,
when I use
from ZEO import ClientStorage

then ZEO is internally using FileStorage, so no IndexedCatalog is used.

Anyone knews, how to use IndexedCatalog with ZEO, or do I have to "patch"
ZEO for myself (or run this one thread)?

I'm not sure that I fully understand your problem - the usage of
IndexedCatalog is IMHO unrelated to threading issues of zodb. And it's not
true that zodb supports only one thread. I use it in a multi-threaded
environment - as does zope.

But it is necessary to use a separate connection object per thread. I've
written myself a transaction service alike to the java transaction api that
creates a connection for every call made to my server. In conjunction with
proxy-objects that keep a unique key for each object, I can happily access
zodb concurrently.

The basic idea is like this:

# open a zodb file storage and store a global reference
db = ....

def get_root():
# the module threadlocal is from the cookbook
conn = threadlocal.get("connection", None)
if conn is None:
conn = db.open()
threadlocal["connection"] = conn
return conn.root()

class Data(PersistenObject):
""" my data-object. It has a property id that identifies it uniquely """
...

class DataProxy(object):
def __init__(self, id):
self.id = id

def _g_data(self):
return get_root()[self.id]

data = property(_g_data)

def __getattr__(self, name):
return getattr(self.data, name)

The magic is in the get_root()-function that fetches a thread-local
connection and returns the root-object in my zodb. In my real application,
the data-object is cached as long as I'm in one transaction, so multiple
attribute accesses are faster.

I have to add that I have no expirience with zeo - so maybe that can also
solve your problems, but that will result in a local loop network access
that also slows down your application.

Regarding the usage of IndexedCatalog: It seems you can use it in
conjunction with zeo as it has no idea of whatever storage is needed - all
it does is indexing zodb objects - which you get from zeo as well. Of
course that means that you have to keep a catalog for every thread/process
that accesses the objects. Alternatively, you maybe can make the
IndexedCatalog a zodb-stored object itself, but I'm not sure about that.

 

[Almad]

from IndexedCatalog.Shelf import Shelf
shelf = Shelf(('localhost', 1234), [Class1, Class2])

Thanks, that is what I was searching for.

--
Lukas "Almad" Linhart

[:: http://www.almad.net/ ::]
[:: Humans are too complicated to be described with words. ::]
[:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]

 

[Almad]

from IndexedCatalog.Shelf import Shelf
shelf = Shelf(('localhost', 1234), [Class1, Class2])

Is it also possible to connect via socket instead of port? I've tried to
pass '/tmp/zeosocket' instead of ('localhost', 1234), but it isn't
successfull. Is is any way to do that?

Thank You,
--
Lukas "Almad" Linhart

[:: http://www.almad.net/ ::]
[:: Humans are too complicated to be described with words. ::]
[:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]