best practice in organising folders

[WindAndWaves]

Hi Gurus

I am developing a site with several thousand (database generated) pages and
I am having some problems in organising my folders

This was my plan:
http://www.myurl.com/old/... = all the old files

http://www.myurl.com/2005/... = all the new files with ....

http://www.myurl.com/2005/ = the basic files, such as search.html and
index.html

http://www.myurl.com/2005/i/ = image folder

http://www.myurl.com/2005/i/L/ = large images ~ 1000

http://www.myurl.com/2005/i/M/ = medium images ~ 1000

http://www.myurl.com/2005/i/S/ = small images ~ 1000

http://www.myurl.com/2005/s/ = stylesheet folder ~ 3

http://www.myurl.com/2005/j/ = scripts ~ 4

http://www.myurl.com/2005/o/ = buttons and other gifs ~ 100

http://www.myurl.com/2005/H/ = individual pages ~ 500

http://www.myurl.com/2005/A/ = admin page for each individual page ~ 500

html pages can be in the following directories

/2005/
/2005/H/
/2005/A/

but they all link to the same stylesheet (2005/s/), javascripts (2005/j/),
images and buttons

now my question is how I can, using the least amount of code, make sure that
html pages in all three directories can hook up with their stylesheets and
the like.

I tried the <BASE HREF> thing, but that means that I have to make all my
#-links absolute.

I thought about adding an absolute path for all images, stylesheets,
scripts, etc.., but that also is a lot of code

besides, I am not very confident with using the ../ and ./ syntax.

I usually put all my html documents in one bunch and then just have a couple
of child directories with pics, stylesheets and the like. That way, any
links between html documents do not need any slashes or dots at all.

Any hints greatly appreciated


- Nicolaas

 

[Els]

Hi Gurus

I am developing a site with several thousand (database
generated) pages and I am having some problems in
organising my folders

This was my plan:
http://www.myurl.com/old/... = all the old files [...]
http://www.myurl.com/2005/s/ = stylesheet folder ~ 3 [...]
html pages can be in the following directories [...]
but they all link to the same stylesheet (2005/s/),
javascripts (2005/j/), images and buttons

now my question is how I can, using the least amount of
code, make sure that html pages in all three directories
can hook up with their stylesheets and the like.

I tried the <BASE HREF> thing, but that means that I have
to make all my #-links absolute.

I thought about adding an absolute path for all images,
stylesheets, scripts, etc.., but that also is a lot of code

besides, I am not very confident with using the ../ and ./
syntax.

I usually put all my html documents in one bunch and then
just have a couple of child directories with pics,
stylesheets and the like. That way, any links between html
documents do not need any slashes or dots at all.

Dots aren't needed, but you will need a couple of slashes
obviously..

Any hints greatly appreciated

Refer to them like this:
@import url(/2005/s/yourstylesheet.css);

 

[Sid Ismail]

: besides, I am not very confident with using the ../ and ./ syntax.
:
: I usually put all my html documents in one bunch and then just have a couple
: of child directories with pics, stylesheets and the like. That way, any
: links between html documents do not need any slashes or dots at all.

Much easier to learn relative addressing. Also, the Aranaea editor can
automatically insert those for you (so does Homesite 5, as well as others)
when you insert a hyperlink.

Sid

 

[Jan Faerber]

I am developing a site with several thousand (database generated) pages
and I am having some problems in organising my folders

Then improve the database generation for those html files and you won't have
any problems in 2006, 2007, ...

This was my plan:
http://www.myurl.com/old/... = all the old files

http://www.myurl.com/2005/... = all the new files with ....

http://www.myurl.com/2005/ = the basic files, such as search.html and
index.html

http://www.myurl.com/2005/i/ = image folder

http://www.myurl.com/2005/i/L/ = large images ~ 1000

http://www.myurl.com/2005/i/M/ = medium images ~ 1000

http://www.myurl.com/2005/i/S/ = small images ~ 1000

http://www.myurl.com/2005/s/ = stylesheet folder ~ 3

http://www.myurl.com/2005/j/ = scripts ~ 4

http://www.myurl.com/2005/o/ = buttons and other gifs ~ 100

http://www.myurl.com/2005/H/ = individual pages ~ 500

http://www.myurl.com/2005/A/ = admin page for each individual page ~ 500

html pages can be in the following directories

/2005/
/2005/H/
/2005/A/

For stylesheets you can do this:

<link rel="stylesheet" type="text/css" href="s/style.css">
<link rel="stylesheet" type="text/css" href="../s/style.css">
<link rel="stylesheet" type="text/css" href="../s/s/style.css">

There won't be an error in the browser if one or two of those three files
are missing.

I was looking for a way to include a stylesheet file into another one
but at the moment I can only find such a thing with XML:

<!-- Category: top-level-element -->
<xsl:include
href = uri-reference />

<xsl:import
href = uri-reference />

.... don't ask me any details.

but they all link to the same stylesheet (2005/s/), javascripts (2005/j/),
images and buttons

now my question is how I can, using the least amount of code, make sure
that html pages in all three directories can hook up with their
stylesheets and the like.

I tried the <BASE HREF> thing, but that means that I have to make all my
#-links absolute.
lol

I thought about adding an absolute path for all images, stylesheets,
scripts, etc.., but that also is a lot of code

besides, I am not very confident with using the ../ and ./ syntax.

I usually put all my html documents in one bunch and then just have a
couple
of child directories with pics, stylesheets and the like. That way, any
links between html documents do not need any slashes or dots at all.

Any hints greatly appreciated

Do you use Windows or Linux? I remember on the Suse ftp server and aswell on
the local file system a folder which is a link to another folder. So you
can have a main folder c:\webdir with the sub directories \i and \H and \a
and so on and then you can easiely do linked directories within the
subfolders containing html pages ... on linux like this:

# cd /webdir/H
# ln -s /webdir/images/
# ls -l
lrwxrwxrwx 1 root root ..... images -> /web/images/

That is a symbolic link. When you are in the subdirectory \webdir\H with the
linked folder \images which is the same like \webdir\H\images but pointing
to \webdir\images you can have a file in \webdir\H with <img
src="images/car.gif"> and the browser will find this gif by the location
\webdir\images\car.gif - it is similar with windows - just use the context
menu of your mouse in the explorer window.

best wishes
yours

 

[WindAndWaves]

[.............]

Refer to them like this:
@import url(/2005/s/yourstylesheet.css);

Hoi Els

Do you mean

<STYLE>

@import url(/2005/s/yourstylesheet.css);

</STYLE>

if so -what is that better than the standard
<LINK HREF="s/s.css" REL="stylesheet" TYPE="text/css">

TIA

- Nicolaas

 

[Els]

[.............]

Refer to them like this:
@import url(/2005/s/yourstylesheet.css);

Hoi Els

Do you mean

<STYLE>

@import url(/2005/s/yourstylesheet.css);

</STYLE>

if so -what is that better than the standard
<LINK HREF="s/s.css" REL="stylesheet" TYPE="text/css">

Two differences:
- /2005/s/s.css takes it from the root, where s/s.css takes
it from the folder you're in.
- @import isn't understood by for instance NN4, so that
browser can't screw up your styles.

Of course it depends on your stylesheet and on what you want
whether you want NN4 to read your styles or not.

Personally, I put all my styles in @import, and have a
separate stylesheet (simpler!) for NN4 in the <link> element.

Have to be careful though, as that stylesheet is also read by
better browsers. Anything in the linked stylesheet that is
wrong for modern browsers, should be overridden in the
imported stylesheet. (and of course have the <link> earlier in
the page than the @import)

 

[WindAndWaves]

[.............]

Refer to them like this:
@import url(/2005/s/yourstylesheet.css);

Hoi Els

Do you mean

<STYLE>

@import url(/2005/s/yourstylesheet.css);

</STYLE>

if so -what is that better than the standard
<LINK HREF="s/s.css" REL="stylesheet" TYPE="text/css">

Two differences:
- /2005/s/s.css takes it from the root, where s/s.css takes
it from the folder you're in.
- @import isn't understood by for instance NN4, so that
browser can't screw up your styles.

Of course it depends on your stylesheet and on what you want
whether you want NN4 to read your styles or not.

Personally, I put all my styles in @import, and have a
separate stylesheet (simpler!) for NN4 in the <link> element.

Have to be careful though, as that stylesheet is also read by
better browsers. Anything in the linked stylesheet that is
wrong for modern browsers, should be overridden in the
imported stylesheet. (and of course have the <link> earlier in
the page than the @import)

Hi Els

Thank you for your clear reply. Personally, call me an idiot, I think that
people with NN4 are probably so used to bad pages that it is hardly
worthwhile to cater for them.

Thank you again for your reply.

- Nicolaas

 

[Toby Inkster]

http://www.myurl.com/2005/i/ = image folder
http://www.myurl.com/2005/i/L/ = large images ~ 1000
http://www.myurl.com/2005/i/M/ = medium images ~ 1000
http://www.myurl.com/2005/i/S/ = small images ~ 1000
http://www.myurl.com/2005/s/ = stylesheet folder ~ 3
http://www.myurl.com/2005/j/ = scripts ~ 4
http://www.myurl.com/2005/o/ = buttons and other gifs ~ 100
http://www.myurl.com/2005/H/ = individual pages ~ 500
http://www.myurl.com/2005/A/ = admin page for each individual page ~ 500

Sorry, but sounds a bit OTT to me.

http://www.myurl.com/2005/ = pages made in 2005
http://www.myurl.com/2005/images/ = any images specific to 2005
http://www.myurl.com/images/ = other images
http://www.myurl.com/style/ = style sheets
http://www.myurl.com/scripts/ = scripts
http://www.myurl.com/admin/ = admin pages

Rationale? Are you likely to have different style sheets for pages made in
2005 and pages made in 2006? No.

Are you likely to use different Javascript on pages made in 2005 and pages
made in 2006? No.

Admin interface -- if I want to edit a document on my website, should I
have to look up when it was created first? No.

Many images (navigation buttons, logos, etc) are likely to also be shared
over the years. Other images will be specific to one particular article,
so won't need to be shared, hence the "/2005/images/" directory.

Then give the pages meaningful URLs like:

http://www.myurl.com/2005/shoes
http://www.myurl.com/2005/ships
http://www.myurl.com/2005/sealing_wax
http://www.myurl.com/2005/cabbages
http://www.myurl.com/2005/kings
http://www.myurl.com/2005/why_is_the_sea_boiling_hot
http://www.myurl.com/2005/do_pigs_have_wings

If you were dealing with thousands of articles per year, then I'd also
recommend splitting up by month, and perhaps even by day, but with about
500, by year ought to be enough.

 

[Jan Faerber]

Sorry, but sounds a bit OTT to me.

That can have something to do with 'Feng Shui HTML' which means
'WindAndWaves'.

 

[WindAndWaves]

That can have something to do with 'Feng Shui HTML' which means
'WindAndWaves'.




--

You guys are rather funny.

Well, 2005 actually stands for 2005 onwards. In a few months time it will
actually move to the root directory and the pre-2005 stuff will be deleted.

I guess another question that I have is (and I have many.... thank you for
taking the time to read them):

how many documents can a folder handle.

With that I am mainly curious at what number of documents it will slow down.
You see, I am big on lean and fast coding (that is why I use letters).
While most people put their images in the 'images' folder, i just put them
in the 'i' folder and supply a little directory document with the thing.

TIA

- Nicolaas

 

[email]

If you want to learn more about Feng Shui (in Chinese and English) I'd
recommend:

Feng Shui Chinese
http://www.fengshuichinese.com/

 

[Jan Faerber]

how many documents can a folder handle.

That is an interesting question.


http://httpd.apache.org/docs-2.0/howto/auth.html.en#gettingitworking

Here it says:

<q>The AuthUserFile directive sets the path to the password file that we
just created with htpasswd. If you have a large number of users, it can be
quite slow to search through a plain text file to authenticate the user on
each request.
</q>

.... and the next sentence is ...

<q>
Apache also has the ability to store user information in fast database
files.
</q>


So in terms of the above settings: Use database generation on the fly when
the user clicks a link instead of database generating thousands of html
files into one folder.

 

[WindAndWaves]

That is an interesting question.


http://httpd.apache.org/docs-2.0/howto/auth.html.en#gettingitworking

Here it says:

<q>The AuthUserFile directive sets the path to the password file that we
just created with htpasswd. If you have a large number of users, it can be
quite slow to search through a plain text file to authenticate the user on
each request.
</q>

... and the next sentence is ...

<q>
Apache also has the ability to store user information in fast database
files.
</q>


So in terms of the above settings: Use database generation on the fly when
the user clicks a link instead of database generating thousands of html
files into one folder.

Sorry, but you have lost me, what does apache password settings got to do
with it?

My question is, how many files can a folder handle.

Say, for example, if there are more than 500 files in a folder, will this
slow down the site?

Thanks

- Nicolaas

 

[Toby Inkster]

Well, 2005 actually stands for 2005 onwards. In a few months time it will
actually move to the root directory and the pre-2005 stuff will be deleted.

So you will move the stuff from, say "/2005/blah" to "/blah"? If so, make
sure you set up redirects and don't break links!

how many documents can a folder handle.

This depends on the operating system and filesystem, but is generally in
the tens of thousands. If you hit against it you can always store them in
several directories and then use mod_rewrite to make them appear as one
virtual directory.

 

[WindAndWaves]

[....]

This depends on the operating system and filesystem, but is generally in
the tens of thousands. If you hit against it you can always store them in
several directories and then use mod_rewrite to make them appear as one
virtual directory.

Physically, a folder can probably handle millions of files, but I am
wondering at what point you will start to see the performance of the server
being reduced, because it has to spend time searching for files longer.
I.e. I request to see index.html when I type http://www.yourUrl.com , but
because there are so many files it takes the server extra time to locate the
actual file.

Has anyone ever tested any of this stuff.

My alternative is to turn it all into php and make only one file that
generates all files on the fly, but this also takes time as the syntax for
creating the listing file is rather complex (100s of if then else
statements, etc...).

What do you think?

 

[Toby Inkster]

Physically, a folder can probably handle millions of files, but I am
wondering at what point you will start to see the performance of the server
being reduced, because it has to spend time searching for files longer.

When it comes to the file system, don't worry about optimisation. You can
be certain that the people who wrote your OS did all that optimisation
already, and they're a lot better at it than you.

 

[WindAndWaves]

When it comes to the file system, don't worry about optimisation. You can
be certain that the people who wrote your OS did all that optimisation
already, and they're a lot better at it than you.

SO you reckon I can just slam a folder with files until is squeaks? Nice.

 

[Toby Inkster]

SO you reckon I can just slam a folder with files until is squeaks?

Yup.

 

[Jan Faerber]

Sorry, but you have lost me, what does apache password settings got to do
with it?

Apache is open source and they work with tricks.

My question is, how many files can a folder handle.

Say, for example, if there are more than 500 files in a folder, will this
slow down the site?

I never had so many documents in one folder on one of my experimental newbee
bullshit shame disgusts. - Or, once I was doing my practical training, yes
there were many pages, but not in one single folder. We can say, that those
free webspaces are horrible slow - even with one file in one folder. I have
got now a single server account with 700 kb upload - that is much faster.
But I wouldn't say 'NO - 500 files in one folder won't slow anything down.'
I just can tell, that when I open a file on the floppy disk everything is
much slower and it is even more slow when opening a folder with many files;
and the other thing: When I open a folder in the filemanager on linux
everything is waiting for the last file beeing loaded, so if there are many
files you have to wait longer until you can start using your mouse in the
GUI. On the other hand side the command line on Windows or the bash is
quite fast looking for a file in a folder. So somehow your question is:
"Will very, very long URLs slow down the site?" I mean it will be the same
if you look for a file named aa_nxblagusfutsch.html in 100 cascaded
subdirectories where the second is a subdirectory of the first, the third a
subdirectory of the second and so on... and you ask "How long will that
take?"

But you can work on your question:

Does the speed depend on the number of users searching for files in one
folder simultaniously and does a high number of requests in just one single
folder at the same time decrease the speed more than it does when the
requests look up in many different folders? ... if yes, how many users
visit your page in one second / minute?

Does the speed of a site search depend on the number of pages in one folder
targeted by a specific search range? ... if yes, is there a search form on
your page?

Does the speed depend more on the speed of your processor or on upload speed
than on nearly unrecognizable time intervals caused by big folder sizes?
.... if yes, do you have an old or a newer and faster processor? ... if it
is old will then the time intervals become bigger with bigger folders with
many files?

Does the speed of updating your whole site project depend on a clearly
structured folder system with subfolders, so you can do an update within a
few seconds or a few minutes - do you look for the documents by number or
location? ... Are there daily updates on your page you have to do manually
and how important is it that your visitors can get as fast as possible the
latest brandest newest news? How do you work?

Do you want to grant access to your folders to all visitors so they can view
their content? ... if yes, do you want the visitors to wait until one page
displays the whole content of a huge folder? e. g. on
http://java.sun.com/j2se/1.5.0/docs/api/index.html you can click on the
letters of the alphabet and you have to wait until the browser shows all
words in the right frame - uh - I mentioned a bad word. If you use frames
it will look nice to have an index.html file in every subfolder to have
different framesets by default. Yes, so nice. But also take a brief look at
a description of the javadoc tool and find out how this html generating
idea works.

Do you want to edit the robots.txt for specific files and how many files
will a search engine download at one visit? ... if this is a point worth of
consideration, how can you set the priority which files are found for sure
in a huge folder in contrast to a folder system with restrictions and open
paths?

Do you want to provide a zip file or a taped archive for your visitors to
download and does this version of your project has to have exactly the same
structure like your online version so you can not use any server side
parsing or including throw the affiliated gimmicks of the webserver?


I understand that this question is still relating to your top posting.