Reproducing du/ls in Java

[Oreo]

Hi there,

I am writing an application that needs to compute the exact size of a
file. I need for this function to work in Windows and UNIX.

One of my issues is that the length() method provided by the File
class does not return the exact size for all files.

The essence of my problem is that I cannot properly compute the size
of a directory or symbolic links in UNIX properly.

File.class() returns 0 for the size of directories regardless of OS.
This appears to be correct on Windows (right-click -> properties on an
empty directory in Windows). However this seems to be incorrect
according to Linux.

When compared to the output of ls, the File.length() method will not
return the proper length of directories (whether on a FAT or reiserfs
file system) or a symbolic link in UNIX. The length of a UNIX
directory is not zero, and the size of symbolic link is the length of
the filename that it points to.

When the windows partition is mounted in Linux, the size of the
directories on the windows partition are reported as being 4096 by
file system utilities ls and du. This contrasts the value of zero
reported by the Microsoft 98 Windows Explorer.

Detailed examples of my problem can be found at the following thread:
http://forum.java.sun.com/thread.jspa?threadID=574755

How can I compute a function that will be able to compute the size of
any file under any operating system?

I would like to avoid his path:
1. Determine if the class is being run in Windows or UNIX
2. If running in UNIX,
a. If a directory of symbolic link: execute : du -sb [file]
| awk '{print $1}'
or ls -al [file ] | awk ' { print $5 }'
b. Otherwise: File.length()
3. If running Windows, simply class File.length() (?)


By taking this route I need to perform an if() statement every time
the size of any file is being computed, and also execute an expensive
command for some special cases (when the file is a directory of link).
The expensive commands will be common, and I would like to avoid
having to do that.

- Oreo

 

[Sudsy]

Oreo wrote:

When compared to the output of ls, the File.length() method will not
return the proper length of directories (whether on a FAT or reiserfs
file system) or a symbolic link in UNIX. The length of a UNIX
directory is not zero, and the size of symbolic link is the length of
the filename that it points to.

WRONG! Take a look at this output from a Linux system:
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Oct 26 13:43 /bin/sh -> bash

That's showing a file size of 4 bytes. It's most empatically NOT
the size of /bin/bash.
As for the correct size of directories, I tested out the following
very simple code and it returned the same value as what ls returned
(not surprising, as it's just mapping to a stat(2) system call
under the covers), namely 5120 bytes.

import java.io.*;

public class DirUsage {
public static void main( String args[] ) {
File f = new File( "." );
System.out.println( "Size = " + f.length() );
}
}

 

[Gordon Beaton]

WRONG! Take a look at this output from a Linux system:
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Oct 26 13:43 /bin/sh -> bash

That's showing a file size of 4 bytes. It's most empatically NOT
the size of /bin/bash.

Well, he did say the length of the filename, not the length of the file...

It might also be worth pointing out that some linux filesystems can
store symlinks in the inode itself without requiring a separate data
block. These are called "fast symlinks" and ls will show you their
size just as if a data block had been allocated to hold the filename.

To the OP: most file operations provided by the operating system act
on the link target, not the link itself, and that's what Java
provides. However ls uses lstat() to get the size of the symlink
itself, but will show the size of the link target if you specify -L.

/gordon

 

[Ann]

Hi there,

I am writing an application that needs to compute the exact size of a
file. I need for this function to work in Windows and UNIX.

One of my issues is that the length() method provided by the File
class does not return the exact size for all files.

The essence of my problem is that I cannot properly compute the size
of a directory or symbolic links in UNIX properly.

File.class() returns 0 for the size of directories regardless of OS.
This appears to be correct on Windows (right-click -> properties on an
empty directory in Windows). However this seems to be incorrect
according to Linux.

When compared to the output of ls, the File.length() method will not
return the proper length of directories (whether on a FAT or reiserfs
file system) or a symbolic link in UNIX. The length of a UNIX
directory is not zero, and the size of symbolic link is the length of
the filename that it points to.

When the windows partition is mounted in Linux, the size of the
directories on the windows partition are reported as being 4096 by
file system utilities ls and du. This contrasts the value of zero
reported by the Microsoft 98 Windows Explorer.

Detailed examples of my problem can be found at the following thread:
http://forum.java.sun.com/thread.jspa?threadID=574755

How can I compute a function that will be able to compute the size of
any file under any operating system?

I would like to avoid his path:
1. Determine if the class is being run in Windows or UNIX
2. If running in UNIX,
a. If a directory of symbolic link: execute : du -sb [file]
| awk '{print $1}'
or ls -al [file ] | awk ' { print $5 }'
b. Otherwise: File.length()
3. If running Windows, simply class File.length() (?)


By taking this route I need to perform an if() statement every time
the size of any file is being computed, and also execute an expensive
command for some special cases (when the file is a directory of link).
The expensive commands will be common, and I would like to avoid
having to do that.

- Oreo

What about the "master file table" ?

And, since there is extra overhead in a directory, do you want
to assign a fraction of that overhead to your filesize too?

 

[Oreo]

Oreo wrote:

When compared to the output of ls, the File.length() method will not
return the proper length of directories (whether on a FAT or reiserfs
file system) or a symbolic link in UNIX. The length of a UNIX
directory is not zero, and the size of symbolic link is the length of
the filename that it points to.

WRONG! Take a look at this output from a Linux system:
$ ls -l /bin/sh
lrwxrwxrwx 1 root root 4 Oct 26 13:43 /bin/sh -> bash

That's showing a file size of 4 bytes. It's most empatically NOT
the size of /bin/bash.
As for the correct size of directories, I tested out the following
very simple code and it returned the same value as what ls returned
(not surprising, as it's just mapping to a stat(2) system call
under the covers), namely 5120 bytes.

import java.io.*;

public class DirUsage {
public static void main( String args[] ) {
File f = new File( "." );
System.out.println( "Size = " + f.length() );
}
}

I am computing the size of the symbolic link itself, not the file it points to.

 

[Oreo]

Hi there,

I am writing an application that needs to compute the exact size of a
file. I need for this function to work in Windows and UNIX.

One of my issues is that the length() method provided by the File
class does not return the exact size for all files.

The essence of my problem is that I cannot properly compute the size
of a directory or symbolic links in UNIX properly.

File.class() returns 0 for the size of directories regardless of OS.
This appears to be correct on Windows (right-click -> properties on an
empty directory in Windows). However this seems to be incorrect
according to Linux.

When compared to the output of ls, the File.length() method will not
return the proper length of directories (whether on a FAT or reiserfs
file system) or a symbolic link in UNIX. The length of a UNIX
directory is not zero, and the size of symbolic link is the length of
the filename that it points to.

When the windows partition is mounted in Linux, the size of the
directories on the windows partition are reported as being 4096 by
file system utilities ls and du. This contrasts the value of zero
reported by the Microsoft 98 Windows Explorer.

Detailed examples of my problem can be found at the following thread:
http://forum.java.sun.com/thread.jspa?threadID=574755

How can I compute a function that will be able to compute the size of
any file under any operating system?

I would like to avoid his path:
1. Determine if the class is being run in Windows or UNIX
2. If running in UNIX,
a. If a directory of symbolic link: execute : du -sb [file]
| awk '{print $1}'
or ls -al [file ] | awk ' { print $5 }'
b. Otherwise: File.length()
3. If running Windows, simply class File.length() (?)


By taking this route I need to perform an if() statement every time
the size of any file is being computed, and also execute an expensive
command for some special cases (when the file is a directory of link).
The expensive commands will be common, and I would like to avoid
having to do that.

- Oreo

What about the "master file table" ?

And, since there is extra overhead in a directory, do you want
to assign a fraction of that overhead to your filesize too?

How can I access the master file table? A search did not produce any
results, other than some recovery software.