C database :: compared to a relational database engine?

[sonos]

Hi,
I am working on a program to archive data to disk.

At what point is it best to use a relational database like MySQL as
the backend instead of a C program alone?

Thanks to any and all replies.

 

[Ian Collins]

Hi,
I am working on a program to archive data to disk.

At what point is it best to use a relational database like MySQL as
the backend instead of a C program alone?

That's your call. You can either start out with a database, or start
out with your own implementation, depending on the complexity of the
data set, the target platform, performance requirements and a host of
other variables.

There isn't a simple answer.

 

[sonos]

That's your call. You can either start out with a database, or start
out with your own implementation, depending on the complexity of the
data set, the target platform, performance requirements and a host of
other variables.

There isn't a simple answer.

OK, thanks. Can you give an example of a simple C data set, a
moderately complex data set, and a data set clearly meant for a
relational database engine?

 

[Ian Collins]

*Please don't quote signatures*

OK, thanks. Can you give an example of a simple C data set, a
moderately complex data set, and a data set clearly meant for a
relational database engine?

No, because of the other variables I mentioned. I have implemented very
complex database engines in C for embedded platforms where we couldn't
find a database engine that would work on that platform. I have also
used a database (because it was there) to perform simple archiving on
hosted platforms.

You have to weigh up the situation and go for what's best for you. On a
hosted platform, I'd use a database for all but the most basic
applications so long as performance wasn't a issue, just to save the
trouble of writing and testing my own.

 

[user923005]

Hi,
I am working on a program to archive data to disk.

At what point is it best to use a relational database like MySQL as
the backend instead of a C program alone?

Only when you care about your data.

 

[Carramba]

sonos skrev:

Hi,
I am working on a program to archive data to disk.

At what point is it best to use a relational database like MySQL as
the backend instead of a C program alone?

Thanks to any and all replies.

I would sugest using PostgreSQL and it's c library...
http://www.postgresql.org/docs/8.1/static/libpq.html

 

[Richard Heathfield]

Carramba said:

sonos skrev:

I would sugest using PostgreSQL and it's c library...

It might be better to move comparative databasology discussions to a
database programming newsgroup. FTR, I don't see any reason for the OP
to avoid MySQL; a database programming newsgroup might, so why not ask
them?

 

[Peter J. Holzer]

sonos skrev:

* If you need or want to manipulate your data with other tools than
your program.

* If your data fits well into the relational model.

* If you want to access your data from other hosts

* If you need transactions, constraints, or other services
ordinarily provided by an RDBMS.

I would sugest using PostgreSQL and it's c library...
http://www.postgresql.org/docs/8.1/static/libpq.html

No. PostgreSQL is not a C library. It comes with a C library which
simplifies sending queries to the server. The database engine itself is
running as a separate process.

There are SQL engines implemented as libraries: SQLite, the MS Jet
engine (used by MS Access), embedded MySQL, etc.

hp

 

[Peter J. Holzer]

* If you need or want to manipulate your data with other tools than
your program.

* If your data fits well into the relational model.

* If you want to access your data from other hosts

* If you need transactions, constraints, or other services
ordinarily provided by an RDBMS.

Forgot the most important reason (and the one least off-topic here):

* The queries you want to make are easier to express in SQL than in C.

hp

 

[sonos]

* If you need or want to manipulate your data with other tools than
your program.

* If your data fits well into the relational model.

* If you want to access your data from other hosts

* If you need transactions, constraints, or other services
ordinarily provided by an RDBMS.

It just seems to me that w/ the current uC computational power,
performance return from a relational database engine is not really
much better than C. Or is it that a SQL query is faster to code than
to hard code the C program? As to remote access, that does not really
pose a problem w. POSIX. Or, were you thinking about concurrent use of
a file? i.e. is contention easier to manage w. a RDB engine compared
to root file systems? And I guess that poses another perspective; C
programs must have a root filesystem (and less portability) whereas
a .db file can be easily ported to another OS?

 

[Ian Collins]

It just seems to me that w/ the current uC computational power,
performance return from a relational database engine is not really
much better than C. Or is it that a SQL query is faster to code than
to hard code the C program?

It's not a matter of performance, it's more a case of not reinventing
the wheel. If you are targeting a platform or platforms where a
database engine is available, just use it for what it does and spend
your time on your application, not worrying about the consistency of you
data.

As to remote access, that does not really
pose a problem w. POSIX. Or, were you thinking about concurrent use of
a file? i.e. is contention easier to manage w. a RDB engine compared
to root file systems? And I guess that poses another perspective; C
programs must have a root filesystem (and less portability) whereas
a .db file can be easily ported to another OS?

Remote access may not be a problem to implement, but with a database the
implementation is provide for you.

One aspect of database storage I've come to like is the ability to
manipulate and view data from the command line or scripts.

 

[sonos]

It's not a matter of performance, it's more a case of not reinventing
the wheel. If you are targeting a platform or platforms where a
database engine is available, just use it for what it does and spend
your time on your application, not worrying about the consistency of you
data.


Remote access may not be a problem to implement, but with a database the
implementation is provide for you.

One aspect of database storage I've come to like is the ability to
manipulate and view data from the command line or scripts.

Ian, thanks for your comments. You commented earlier that your
embedded system was in C, and you also seem to know about the RDB too.

The current problem is that the system data is mainly housed on a
linux server, yet there are several embedded systems downloading just
a chunk of data, not the entire database. The embedded system then
updates a few variables before uploading information to the server. In
the meantime, another user may have added information too but on the
serever, so there is a contention problem. The data really isn't a
prime candidate for relational queries, and it seems that a RDB engine
is overkill in that respect. It's the interchange of data streams that
is the real problem. So maybe it is a matter of server data streams to
the headless system by way of scripting in C. I don't know how to
script for SQL engines, and didn't know if it was worth the effort to
learn for this system, or to recode the server in C.

 

[Ian Collins]

Please don't quote signatures.

Ian, thanks for your comments. You commented earlier that your
embedded system was in C, and you also seem to know about the RDB too.

The current problem is that the system data is mainly housed on a
linux server, yet there are several embedded systems downloading just
a chunk of data, not the entire database. The embedded system then
updates a few variables before uploading information to the server. In
the meantime, another user may have added information too but on the
serever, so there is a contention problem. The data really isn't a
prime candidate for relational queries, and it seems that a RDB engine
is overkill in that respect. It's the interchange of data streams that
is the real problem. So maybe it is a matter of server data streams to
the headless system by way of scripting in C. I don't know how to
script for SQL engines, and didn't know if it was worth the effort to
learn for this system, or to recode the server in C.

Wandering a bit OT here, but if you embedded system supports sockets
(which it sounds like it does to access a remote sever), you should be
able to compile the interface library for one of the popular opensource
databases and use database connections to access the data. The
interface is very simple to use, the only SQL you have to understand is
"select" and "insert".

Whether the data is a prime candidate for relational queries or not, it
sounds like you would benefit from the data integrity offered by
transactions.

A database engine is a good solutions to the requirement for multiple
readers and writers to a pool of data.

 

[sonos]

A database engine is a good solutions to the requirement for multiple

readers and writers to a pool of data.

So it's possible then to capture data input from more than one source
on the same data set, even when several are not 'online'? I understood
the RDB engines must lock a record and not allow edits until it is
released.

 

[Ian Collins]

So it's possible then to capture data input from more than one source
on the same data set, even when several are not 'online'? I understood
the RDB engines must lock a record and not allow edits until it is
released.

That's what transactions are for.

Do some background research and then ask on a database group (no, I
don't know which), this has gone beyond the scope of this group.

 

[Richard Heathfield]

Peter J. Holzer said:

No. PostgreSQL is not a C library. It comes with a C library

He meant "using PostgreSQL and its C library".

Careless apostrophes cost meaning.

 

[Carramba]

Richard Heathfield skrev:

Peter J. Holzer said:


He meant "using PostgreSQL and its C library".

Careless apostrophes cost meaning.

Thank you!

 

[Peter J. Holzer]

Peter J. Holzer said:

He meant "using PostgreSQL and its C library".

Makes sense. Sorry to Carramba.

Careless apostrophes cost meaning.

The interesting thing is that I mentally (mis-)corrected "and it's c library"
to "and it's a c library" without even noticing it, even though I used
to write "it's" instead of "its" a lot, too. One would assume that one
tends to reverse their own errors, but that doesn't seem to be the case.

hp

 

[Peter J. Holzer]

[...]
It just seems to me that w/ the current uC computational power,
performance return from a relational database engine is not really
much better than C.

I didn't realize from your initial posting that you were working on
embedded systems. That adds two additional criteria:

* Is there a client library available for your target system at all?
Even if your system is POSIX-compatible it may have restrictions which
the authors (which were developing for servers and workstations)
simply didn't think of.

* Does it fit on your system? The MySQL and PostgreSQL client libraries
seem to be rather small (a few 100 kB), but for some RDBMSs the client
libs (plus support files: charset conversion tables, error message
catalogs, etc.) are huge (tens of MB).

hp

 

[user923005]

sonos skrev:
I am working on a program to archive data to disk.
At what point is it best to use a relational database like MySQL as
the backend instead of a C program alone?

[...]
It just seems to me that w/ the current uC computational power,
performance return from a relational database engine is not really
much better than C.

I didn't realize from your initial posting that you were working on
embedded systems. That adds two additional criteria:

* Is there a client library available for your target system at all?
Even if your system is POSIX-compatible it may have restrictions which
the authors (which were developing for servers and workstations)
simply didn't think of.

* Does it fit on your system? The MySQL and PostgreSQL client libraries
seem to be rather small (a few 100 kB), but for some RDBMSs the client
libs (plus support files: charset conversion tables, error message
catalogs, etc.) are huge (tens of MB).

hp

MySQL and PostgreSQL have huge footprints and require a server
installation.

Perhaps SQLite or FastDB would be more appropriate for embedded work.
http://www.sqlite.org/
http://www.fastdb.org/fastdb.html

Just a guess.