ZODB: single database, multiple connections

Discussion in 'Python' started by Petra Chong, Oct 30, 2006.

  1. Petra Chong

    Petra Chong Guest

    Hello all

    I am using Python 2.3 and ZODB (without the rest of Zope) with the
    following pattern:

    * One process which writes stuff to a ZODB instance (call it test.db)
    * Another process which reads stuff from the above ZODB instance
    test.db

    What I find is that when the first process writes, the second doesn't
    pick up the changes. I am sure this must be because I am using ZODB
    wrongly, but I can't find any examples that show how to use ZODB in
    this way, and I can't find any API docs for FileStorage, Connection,
    etc. Reading the source code (from C:\python23\lib\site-packages) has
    not thrown up anything useful.

    Here's my test code:

    A simple database class:

    class Database(object):
    PersistentObject = persistent.Persistent
    PersistentDict = BTrees.OOBTree.OOBTree

    def __init__(self, filename, read_only = False):
    self.storage = FileStorage.FileStorage(filename, read_only =
    read_only)
    self.db = DB(self.storage)
    self.connection = self.db.open()
    self.dbroot = self.connection.root()


    Write:


    db = Database("test.db")
    db.data = db.get_dictionary('data')

    sz = len(db.data.keys())

    class Datum(Persistent):
    def __init__(self, value):
    self.value = value

    if __name__ == '__main__':
    # insert 10 things
    for i in range(0, 10):
    val = i + sz
    d = Datum(val)
    db.data[val] = d
    transaction.commit()

    Read:

    db = Database("test.db", read_only = True)

    data = db.get_dictionary('data')

    If I have a Python shell open and run the above two lines, if I run the
    write process repeatedly, the above "data" object never contains any of
    the newly added items. To pick them up I have to totally recreate the
    "db" object.

    I must be doing something wrongly, but I can't figure out what.

    Any suggestions?

    Thanks,

    PC
     
    Petra Chong, Oct 30, 2006
    #1
    1. Advertising

  2. Petra Chong

    Tim Peters Guest

    [Petra Chong]
    > I am using Python 2.3 and ZODB (without the rest of Zope) with the
    > following pattern:
    >
    > * One process which writes stuff to a ZODB instance (call it test.db)
    > * Another process which reads stuff from the above ZODB instance
    > test.db
    >
    > What I find is that when the first process writes, the second doesn't
    > pick up the changes. I am sure this must be because I am using ZODB
    > wrongly, but I can't find any examples that show how to use ZODB in
    > this way, and I can't find any API docs for FileStorage, Connection,
    > etc. Reading the source code (from C:\python23\lib\site-packages) has
    > not thrown up anything useful.
    >
    > Here's my test code:
    >
    > A simple database class:
    >
    > ...
    >
    > Write:
    >
    > ...
    >
    > Read:
    >
    > db = Database("test.db", read_only = True)
    >
    > data = db.get_dictionary('data')
    >
    > If I have a Python shell open and run the above two lines, if I run the
    > write process repeatedly, the above "data" object never contains any of
    > the newly added items. To pick them up I have to totally recreate the
    > "db" object.


    You say that like it's hard to do ;-)

    It's a decent way to proceed. ZODB is a database, and has
    transactional semantics: you never see new object state on the read
    side because you're /in/ a transaction, and a transaction guarantees
    to give you a consistent view of the data. The view would be
    inconsistent if it showed you state committed by different
    transactions on the write side while you're still in the same
    transaction on the read side.

    > I must be doing something wrongly, but I can't figure out what.


    Seems to be a conceptual problem more than anything else.

    > Any suggestions?


    You already know that tossing your connection and opening a new
    connection will give you a newer view of the database, and it's
    unclear why you don't think that's good enough. Other approaches
    amount to telling ZODB (on the read side) that you're done with the
    current transaction. For example, try doing

    transaction.abort()

    on the read side when you're ready to see newer object state.

    BTW, a better place to ask about ZODB is the zodb-dev list:

    http://mail.zope.org/mailman/listinfo/zodb-dev

    It's not just for developers /of/ ZODB. Note that you need to
    subscribe to it in order to post to it (that's a heavyweight anti-spam
    gimmick common to all Zope lists).
     
    Tim Peters, Oct 30, 2006
    #2
    1. Advertising

  3. Petra Chong

    Petra Chong Guest


    > > If I have a Python shell open and run the above two lines, if I run the
    > > write process repeatedly, the above "data" object never contains any of
    > > the newly added items. To pick them up I have to totally recreate the
    > > "db" object.

    >
    > You say that like it's hard to do ;-)
    >


    It isn't, but this was the problem:

    It took 15 seconds to open the database, so I thought that I shouldn't
    be recreating the database and should be refreshing it. There is
    nothing in the docs that says that the right behaviour is to recreate
    it.

    However, I then put some debug statements around it (I know, I know, I
    could have used profiling) to find out exactly what was taking 15
    seconds. Turned out there was some rubbish in the db that shouldn't
    have been there. I got rid of it, and now it doesn't take 15 seconds to
    recreate the database.

    So- I am now recreating the db, and my problem is solved.

    >
    > BTW, a better place to ask about ZODB is the zodb-dev list:
    >
    > http://mail.zope.org/mailman/listinfo/zodb-dev
    >
    > It's not just for developers /of/ ZODB. Note that you need to
    > subscribe to it in order to post to it (that's a heavyweight anti-spam
    > gimmick common to all Zope lists).


    Thanks for that- it's been hard to track down information on ZODB.

    Regards,

    PC
     
    Petra Chong, Oct 30, 2006
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    4
    Views:
    2,550
  2. Replies:
    3
    Views:
    565
    Roedy Green
    Jul 31, 2007
  3. jobs
    Replies:
    2
    Views:
    909
  4. James Lopes

    Multiple database connections

    James Lopes, May 1, 2009, in forum: Ruby
    Replies:
    4
    Views:
    107
    Rick DeNatale
    May 3, 2009
  5. ela
    Replies:
    12
    Views:
    373
    Uri Guttman
    Apr 6, 2009
Loading...

Share This Page