Python choice of database

P

Philippe C. Martin

Hi,

I am looking for a stand-alone (not client/server) database solution for
Python.

1) speed is not an issue
2) I wish to store less than 5000 records
3) each record should not be larger than 16K


As I start with Python objects, I thought of using shelve, but looking at
the restrictions (record size + potential collisions) I feel I should study
my options a bit further before I get started.


Regards,

Philippe
 
E

Erik Max Francis

Philippe said:
I am looking for a stand-alone (not client/server) database solution for
Python.

1) speed is not an issue
2) I wish to store less than 5000 records
3) each record should not be larger than 16K


As I start with Python objects, I thought of using shelve, but looking at
the restrictions (record size + potential collisions) I feel I should study
my options a bit further before I get started.

Why not just use native Python data structures and pickle them?
 
P

Peter Hansen

Philippe said:
I am looking for a stand-alone (not client/server) database solution for
Python.

1) speed is not an issue
2) I wish to store less than 5000 records
3) each record should not be larger than 16K

As I start with Python objects, I thought of using shelve, but looking at
the restrictions (record size + potential collisions) I feel I should study
my options a bit further before I get started.

You don't say whether you want *pure* Python solutions, so I'll suggest
pysqlite which wraps the SQLite embedded database in a pretty much
totally transparent fashion and is highly effective, fast, compact,
reliable (so far, in my experience), and clean.

You also don't say whether you want a SQL database, so if you are free
to try anything, you might look at ZODB or Durus (think of it as a
lighter-weight ZODB). I believe Durus is pure Python, but it might have
some C code for performance (like ZODB). It's not SQL, and should
perhaps be thought of (as it describes itself) as an object persistence
solution, rather than a "database".

-Peter
 
P

Peter Hansen

John said:
Gadfly
PySQLite ( requires SQLite library )

I want to clarify this parenthetical comment, for the record. When I
first downloaded PySQLite I had already gone and installed SQLite,
thinking it was a prerequisite in that sense.

In fact, the PySQLite install includes a .pyd which contains a
statically linked version of the complete SQLite library. No additional
installation is required, making it an even simpler solution than I
thought at first.

-Peter
 
P

Philippe C. Martin

Well that would be shelve I guess ... with the restrictions I mentioned.

Regards,

Philippe
 
P

Philippe C. Martin

Thank you all for your answers.

A pure Python would have beenmy first choice. yet I now feel I should spend
some time looking at PySQLite (I like the fact it's pre-compiled for
Windows).

Thanks.

Philippe
 
E

Erik Max Francis

Philippe said:
You mean pickling a dictionnary of 5000/16K objects ?

Yes. You said speed was not an issue; pickling only 5000 objects, each
no more than 16 kB, is easily handled by any remotely modern machine
(and even plenty which are not very modern).
 
P

Philippe C. Martin

OK, I'll try that too.

Regards,

Philippe


Yes. You said speed was not an issue; pickling only 5000 objects, each
no more than 16 kB, is easily handled by any remotely modern machine
(and even plenty which are not very modern).
 
J

John Abel

Philippe said:
Thank you all for your answers.

A pure Python would have beenmy first choice. yet I now feel I should spend
some time looking at PySQLite (I like the fact it's pre-compiled for
Windows).

Thanks.

Philippe



Philippe C. Martin wrote:
Out of the suggestions SnakeSQL and KirbyBase are pure python. Gadfly
is sorta pure, in that it will work without the compiled kjbuckets lib.

J
 
W

William Park

Philippe C. Martin said:
Hi,

I am looking for a stand-alone (not client/server) database solution
for Python.

1) speed is not an issue
2) I wish to store less than 5000 records
3) each record should not be larger than 16K

As I start with Python objects, I thought of using shelve, but looking
at the restrictions (record size + potential collisions) I feel I
should study my options a bit further before I get started.

Possible approach might be:
1. 5000 files -- my personal favourite.
2. GDBM
3. SQLite

--
William Park <[email protected]>, Toronto, Canada
ThinFlash: Linux thin-client on USB key (flash) drive
http://home.eol.ca/~parkw/thinflash.html
BashDiff: Full featured Bash shell
http://freshmeat.net/projects/bashdiff/
 
P

Philippe C. Martin

Thanks, I'm looking at KirbyBase also but wonder if it can handle bitmaps (I
could always pickle it first I guess).

Regards,

Philippe
 
P

Paul Rubin

Philippe C. Martin said:
1) speed is not an issue
2) I wish to store less than 5000 records
3) each record should not be larger than 16K

You don't mention whether multiple running programs need to use it
concurrently. That's usually done with client/server db's but it can
be done standalone.
 
P

Philippe C. Martin

Correct, that's not a constraint right now.


Paul said:
You don't mention whether multiple running programs need to use it
concurrently. That's usually done with client/server db's but it can
be done standalone.
 
O

Oren Tirosh

Philippe said:
Hi,

I am looking for a stand-alone (not client/server) database solution for
Python.

1) speed is not an issue
2) I wish to store less than 5000 records
3) each record should not be larger than 16K

How about using the filesystem as a database? For the number of records
you describe it may work surprisingly well. A bonus is that the
database is easy to manage manually. One tricky point is updating: you
probably want to create a temporary file and then use os.rename to
replace a record in one atomic operation.

For very short keys and record (e.g. email addresses) you can use
symbolic links instead of files. The advantage is that you have a
single system call (readlink) to retrieve the contents of a link. No
need to open, read and close.

This works only on posix systems, of course. The actual performance
depends on your filesystem but on linux and BSDs I find that
performance easily rivals that of berkeleydb and initialization time is
much faster. This "database" also supports reliable concurrent access
by multiple threads or processes.

See http://www.tothink.com/python/linkdb

Oren
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,901
Latest member
Noble71S45

Latest Threads

Top