Maximum number of files on a disk ?

A

Axel Etzold

Dear all,

is there a way to read out the maximum number of files that can be created on a hard disk/memory
stick etc., from Ruby ?
I'd need to know this for ext3, NTFS and fat32 systems, and from a search about ext3, there doesn't
seem to be a general limit, but a number fixed at the creation of the file system. How could i get that ?

Thank you very much,

Best regards,

Axel
 
W

Wayne Vucenic

the maximum number of files that can be created on a hard disk...
for ... NTFS and fat32 ...

For NTFS, the maximum number of files that can be created is 2**32 - 1.

I'd expect that the number for fat32 is similarly large.

Hope this helps,

Wayne
 
A

Axel Etzold

-------- Original-Nachricht --------
Datum: Fri, 28 Nov 2008 08:10:34 +0900
Von: "Wayne Vucenic" <[email protected]>
An: (e-mail address removed)
Betreff: Re: Maximum number of files on a disk ?
For NTFS, the maximum number of files that can be created is 2**32 - 1.

I'd expect that the number for fat32 is similarly large.

Hope this helps,

Wayne

---
Wayne Vucenic
No Bugs Software
Agile Ruby (but preferably not Rails) Contract Programming in Silicon
Valley since 2001

Dear Wayne,

thank you for responding.
I am writing to an NTFS disk, where I am splitting up a large XML file into smaller
bits, which are saved as individual files ( and closed immediately after writing to them).
The process runs smoothly until I get the following last correct output

49'000'000 # <- number of lines traversed in the XML file so far (print every 500'000)
<current file title>
1483042 # <- smaller files written to disk so far

and this error:

lbwiki2.rb:114:in `initialize': Operation not supported - /media/Philips External Hard Disk/en_wiki/Warnock's\ Dilemma (Errno::EOPNOTSUPP)
from lbwiki2.rb:114:in `new'
from lbwiki2.rb:114
from lbwiki2.rb:85:in `each'
from lbwiki2.rb:85

In this line 114, there's a File.new for the next small file to be created. The spaces in the file name are no problem.
I am using Ubuntu 8.10 (32 bit) + ruby-1.8.7-p72.

I cannot create any files on the disk anymore, neither with Ruby nor on the command line. This works
again if I delete some file/folder.

So there must be some maximum amount of files that this NTFS disk can handle, which is way below 2**32-1 ...
What can I do here ?


Thank you very much!

Best regards,

Axel
 
T

Tim Hunter

Axel said:
-------- Original-Nachricht --------


Dear Wayne,

thank you for responding.
I am writing to an NTFS disk, where I am splitting up a large XML file into smaller
bits, which are saved as individual files ( and closed immediately after writing to them).
The process runs smoothly until I get the following last correct output

49'000'000 # <- number of lines traversed in the XML file so far (print every 500'000)
<current file title>
1483042 # <- smaller files written to disk so far

and this error:

lbwiki2.rb:114:in `initialize': Operation not supported - /media/Philips External Hard Disk/en_wiki/Warnock's\ Dilemma (Errno::EOPNOTSUPP)
from lbwiki2.rb:114:in `new'
from lbwiki2.rb:114
from lbwiki2.rb:85:in `each'
from lbwiki2.rb:85

In this line 114, there's a File.new for the next small file to be created. The spaces in the file name are no problem.
I am using Ubuntu 8.10 (32 bit) + ruby-1.8.7-p72.

I cannot create any files on the disk anymore, neither with Ruby nor on the command line. This works
again if I delete some file/folder.

So there must be some maximum amount of files that this NTFS disk can handle, which is way below 2**32-1 ...
What can I do here ?

I don't know anything about NTFS. I'm just thinking out loud. I wonder
if you've hit some kind of limit on the number of files that can be in
the same directory? Is there some way to organize the files into
multiple directories?
 
B

Brian Candler

Axel said:
is there a way to read out the maximum number of files that can be
created on a hard disk/memory
stick etc., from Ruby ?

Under Linux: df -i

$ df -i
Filesystem Inodes IUsed IFree IUse% Mounted on
/dev/sda3 786432 246411 540021 32% /

Each file (and directory) takes one inode, so when you've run out of
inodes, you can't create any more files.
I'd need to know this for ext3, NTFS and fat32 systems

I don't know what limits NTFS and fat32 have. ISTR that the DOS
filesystem had a limit of ~200 files in the root directory, but this
limit didn't apply for subdirectories.

B.
 
W

Wayne Vucenic

The spaces in the file name are no problem.

Agreed spaces are no problem, but your file name is somewhat atypical.
Have you tried with a totally plain vanilla file name?
I cannot create any files ...This works again if I delete some file/folde=
r.

Sorry to ask the obvious, but are you out of space on the disk (or
anywhere close to being out of space)?

Also, Tim makes a good point about files in folders. Although NTFS has
no limit on this, on older versions of NTFS, things got very slow if there
were more than a few thousand files in a folder. That's been fixed in
newer versions, but it's still a good idea not to put lots of files in the
same folder.

Good luck,

Wayne

---
Wayne Vucenic
No Bugs Software
Agile Ruby (but preferably not Rails) Contract Programming in Silicon
Valley since 2001
-------- Original-Nachricht --------


Dear Wayne,

thank you for responding.
I am writing to an NTFS disk, where I am splitting up a large XML file in= to smaller
bits, which are saved as individual files ( and closed immediately after = writing to them).
The process runs smoothly until I get the following last correct output

49'000'000 # <- number of lines traversed in the XML file so far (pri= nt every 500'000)
<current file title>
1483042 # <- smaller files written to disk so far

and this error:

lbwiki2.rb:114:in `initialize': Operation not supported - /media/Philips =
External Hard Disk/en_wiki/Warnock's\ Dilemma (Errno::EOPNOTSUPP)
from lbwiki2.rb:114:in `new'
from lbwiki2.rb:114
from lbwiki2.rb:85:in `each'
from lbwiki2.rb:85

In this line 114, there's a File.new for the next small file to be create=
d. The spaces in the file name are no problem.
I am using Ubuntu 8.10 (32 bit) + ruby-1.8.7-p72.

I cannot create any files on the disk anymore, neither with Ruby nor on t= he command line. This works
again if I delete some file/folder.

So there must be some maximum amount of files that this NTFS disk can han=
dle, which is way below 2**32-1 ...
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top