IndexedCatalog and ZEO

Discussion in 'Python' started by Almad, Mar 6, 2005.

  1. Almad

    Almad Guest

    Hello,

    I'm trying to use IndexedCatalog
    [http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
    [http://www.cherrypy.org] application.
    As Zodb support access just from one Python thread, I must either use just
    one CherryPy thread (slow), or use ZEO. However, if I understand it good,
    when I use
    from ZEO import ClientStorage

    then ZEO is internally using FileStorage, so no IndexedCatalog is used.

    Anyone knews, how to use IndexedCatalog with ZEO, or do I have to "patch"
    ZEO for myself (or run this one thread)?

    Thanks,
    --
    Lukas "Almad" Linhart

    [:: http://www.almad.net/ ::]
    [:: Humans are too complicated to be described with words. ::]
    [:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]
    Almad, Mar 6, 2005
    #1
    1. Advertising

  2. Almad wrote:
    > Hello,
    >
    > I'm trying to use IndexedCatalog
    > [http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
    > [http://www.cherrypy.org] application.
    > As Zodb support access just from one Python thread, I must either use just
    > one CherryPy thread (slow), or use ZEO. However, if I understand it good,
    > when I use
    > from ZEO import ClientStorage
    >
    > then ZEO is internally using FileStorage, so no IndexedCatalog is used.


    If you're using IndexedCatalog's Shelf class, just pass it a host/port
    tuple instead of a path:

    from IndexedCatalog.Shelf import Shelf
    shelf = Shelf(('localhost', 1234), [Class1, Class2])
    Leif K-Brooks, Mar 6, 2005
    #2
    1. Advertising

  3. > I'm trying to use IndexedCatalog
    > [http://www.async.com.br/projects/IndexedCatalog/] in my CherryPy
    > [http://www.cherrypy.org] application.
    > As Zodb support access just from one Python thread, I must either use just
    > one CherryPy thread (slow), or use ZEO. However, if I understand it good,
    > when I use
    > from ZEO import ClientStorage
    >
    > then ZEO is internally using FileStorage, so no IndexedCatalog is used.
    >
    > Anyone knews, how to use IndexedCatalog with ZEO, or do I have to "patch"
    > ZEO for myself (or run this one thread)?


    I'm not sure that I fully understand your problem - the usage of
    IndexedCatalog is IMHO unrelated to threading issues of zodb. And it's not
    true that zodb supports only one thread. I use it in a multi-threaded
    environment - as does zope.

    But it is necessary to use a separate connection object per thread. I've
    written myself a transaction service alike to the java transaction api that
    creates a connection for every call made to my server. In conjunction with
    proxy-objects that keep a unique key for each object, I can happily access
    zodb concurrently.

    The basic idea is like this:

    # open a zodb file storage and store a global reference
    db = ....

    def get_root():
    # the module threadlocal is from the cookbook
    conn = threadlocal.get("connection", None)
    if conn is None:
    conn = db.open()
    threadlocal["connection"] = conn
    return conn.root()

    class Data(PersistenObject):
    """ my data-object. It has a property id that identifies it uniquely """
    ...

    class DataProxy(object):
    def __init__(self, id):
    self.id = id

    def _g_data(self):
    return get_root()[self.id]

    data = property(_g_data)

    def __getattr__(self, name):
    return getattr(self.data, name)

    The magic is in the get_root()-function that fetches a thread-local
    connection and returns the root-object in my zodb. In my real application,
    the data-object is cached as long as I'm in one transaction, so multiple
    attribute accesses are faster.

    I have to add that I have no expirience with zeo - so maybe that can also
    solve your problems, but that will result in a local loop network access
    that also slows down your application.

    Regarding the usage of IndexedCatalog: It seems you can use it in
    conjunction with zeo as it has no idea of whatever storage is needed - all
    it does is indexing zodb objects - which you get from zeo as well. Of
    course that means that you have to keep a catalog for every thread/process
    that accesses the objects. Alternatively, you maybe can make the
    IndexedCatalog a zodb-stored object itself, but I'm not sure about that.

    --
    Regards,

    Diez B. Roggisch
    Diez B. Roggisch, Mar 6, 2005
    #3
  4. Almad

    Almad Guest

    Leif K-Brooks wrote:
    > from IndexedCatalog.Shelf import Shelf
    > shelf = Shelf(('localhost', 1234), [Class1, Class2])


    Thanks, that is what I was searching for.

    --
    Lukas "Almad" Linhart

    [:: http://www.almad.net/ ::]
    [:: Humans are too complicated to be described with words. ::]
    [:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]
    Almad, Mar 7, 2005
    #4
  5. Almad

    Almad Guest

    Leif K-Brooks wrote:

    > from IndexedCatalog.Shelf import Shelf
    > shelf = Shelf(('localhost', 1234), [Class1, Class2])


    Is it also possible to connect via socket instead of port? I've tried to
    pass '/tmp/zeosocket' instead of ('localhost', 1234), but it isn't
    successfull. Is is any way to do that?

    Thank You,
    --
    Lukas "Almad" Linhart

    [:: http://www.almad.net/ ::]
    [:: Humans are too complicated to be described with words. ::]
    [:: PGP/GNUPg key: http://www.almad.net/download/pubkey.asc ::]
    Almad, Mar 7, 2005
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. drs

    ZEO and COM

    drs, Jul 11, 2003, in forum: Python
    Replies:
    0
    Views:
    377
  2. drs

    More ZEO/ZODB issues

    drs, Jul 12, 2003, in forum: Python
    Replies:
    0
    Views:
    327
  3. Paul Boots
    Replies:
    0
    Views:
    336
    Paul Boots
    Jun 21, 2005
  4. Paul Boots
    Replies:
    0
    Views:
    279
    Paul Boots
    Jun 21, 2005
  5. Almad
    Replies:
    0
    Views:
    295
    Almad
    Jun 24, 2005
Loading...

Share This Page