store data files in database versus filesystem

J

jobs239

I have a web application and each time the app is used I want to save
the output as a compressed html file in my database. Is this a good
idea to save file in database? as opposed to filesystem and just have
path in the database table.
From a java program is it possible to copy file to a certain location
on filesystem if I use the second option?

I need to get this tar file and untar it and display it later. Is
tar/untar possible through java programs?
 
R

Rhino

I have a web application and each time the app is used I want to save
the output as a compressed html file in my database. Is this a good
idea to save file in database? as opposed to filesystem and just have
path in the database table.
This is more of a database question than a Java question. But, since I am a
database guy, I will try to answer you.

The answer partially depends on what database you are using. If you are
talking about a serious, professional database, like DB2 (I suppose MySQL is
also mature enough to qualify as serious and professional by now), you
almost certainly have the option of storing large files in the database or
in the filesystem. If you are talking about less serious databases, like
Microsoft Access or FileMaker, I'm not sure if they give you _either_
option! (I've dabbled with Access a tiny bit but don't pretend to be current
on the features.)

Certainly, DB2 and MySQL, the only relational databases I know well, let you
store files directly in the database. They also give you the alternative of
storing something like a URL in the database and storing the file itself in
the filesystem. Both approaches have pros and cons. If the file is stored in
the database, it is obviously more convenient to access it; you don't have
to find its location in the database and then do a second lookup to get the
file. But if the file is stored in the database, it makes the data rows much
bigger and this can have an impact on recovery, logging, performance and
perhaps locking. DB2 gives you the option of not logging updates to the
blobs (datatype used for things like pictures) and clobs (datatype used for
large text objects) which effectively makes you responsible for recovering
them yourself; this is often desireable to avoid the overhead of having DB2
log updates to these objects, which can be a performance hit. I think MySQL
has many of these same features but I've never had much contact with blobs
and clobs in MySQL. In fact, I really haven't done much work with blobs and
clobs in DB2 either! I just now a bit of the theory. If you want links to
reference information on this topic for either database, let me know and I
can try to dig that up.
on filesystem if I use the second option?
Yes, this is usually possible as I recall; after all if you can't get at the
data after you store it, it isn't much good, is it? In fact, I know that DB2
gives you locators so that you can find just one section of a file and grab
just that one portion in a program, then do with it whatever you like.

I expect that locators will be difficult to use if the file is tarred. You
might find it easier (or just plain necessary) to leave the data untarred so
that you can search it. But if you want to work with the entire file and
just copy it somewhere, that should be possible.
I need to get this tar file and untar it and display it later. Is
tar/untar possible through java programs?
I saw a post on this newsgroup the other day that said there were some third
party Java classes that did tar/untar. I don't remember the exact answer but
if you do a Google newsgroups search on 'tar' within this newsgroup, I'm
sure you'll find some useful details. I'm pretty sure that the post I'm
thinking of was within the last week.
 
L

lewmania942

Hi,

Rhino already answered your question... I'm just adding
a minor nitpick :)

I have a web application and each time the app is used I want to save
the output as a compressed html file in my database. Is this a good
idea to save file in database? as opposed to filesystem and just have
path in the database table.

So each time an html page is served, you create one "compressed
html file" on the filesystem right? So you're not including other
resources that your server may be serving (eg pictures linked in
the page) ?
I need to get this tar file and untar it and display it later. Is
tar/untar possible through java programs?

tar doesn't compress. tar simply archives. It doesn't make much
sense to use tar for a single file.

Hope it helps,
 
J

Jon Martin Solaas

tar doesn't compress. tar simply archives. It doesn't make much
sense to use tar for a single file.

Hope it helps,

Zip compression would make sense, though, and I suppose thats natively
supported in Java/JDK, after all it's the native format of the various
java archive types.
 
J

Jon Martin Solaas

I have a web application and each time the app is used I want to save
the output as a compressed html file in my database. Is this a good
idea to save file in database? as opposed to filesystem and just have
path in the database table.

on filesystem if I use the second option?

I need to get this tar file and untar it and display it later. Is
tar/untar possible through java programs?

If you want to treat the compressed files as any other resource stored
in the database, it may be a good idea to store them there too. If you
backup your database your files will be backed up as well, and they're
accsessible like any other resource in the database, both when it comes
to security and general access in a networked environment. Also you will
avoid synchronization issues between what is stored in the database and
what actually is stored on the filesystem, suppose you succeed in
storing the path in the database, but fails writing to the filesystem?
If everything is stored within a database transaction you avoid that
problem.

On the other hand, if you store the files on the filesystem your
database won't grow that large, files will be easily accessible for all
kinds of programs and users, independent of access to the database
itself, etc. etc.

I suppose it all depends on how you want to use and administer the
compressed files after they've been stored.
 
T

tom fredriksen

I have a web application and each time the app is used I want to save
the output as a compressed html file in my database. Is this a good
idea to save file in database? as opposed to filesystem and just have
path in the database table.

This is mostly a question of two issues, performance and data sharing.
- if the need is to share the data between servers then a db or a
fileserver should be used.
- if the need is performance (for reading and processing it, perhaps
several times) then one should go for the file solution as network
connections are slow compared to local access. Further on, processing
blobs with stored procedures in a db is not a really good solution or
even possible.

/tom
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,755
Messages
2,569,536
Members
45,014
Latest member
BiancaFix3

Latest Threads

Top