Deleting files modified before a specified number of days using Java.

[Sameer]

Dear All,
We have one server (FTP). One drive of which stores scanned documents.
Users usually upload documents (.JPG files). Currently that drive is
of about 27 GB containing more that 1,50,000 files.
We have to keep data of 45 days only and have to remove other files
regularly. Currently we use the AWK like utility for the same. It has
some limitations and it do not accept command line parameters and
therefore can not be scheduled as a program.

Now i have written one Java program for the same.
The java file as a Google document:

http://docs.google.com/View?docid=dfvj6k7q_3x4pr28gf

This program wont help. I am running the program from the server.
But it just got hanged after

ile = new File(dirPath);
fileArray = file.listFiles(this);

Not even able to create the fileArray object....

Do increasing memory of server will help or how to change the program?
I don't think multi-threading like concepts will help as the file
number is big.

What should i do to make it work or any other suggstions to carry out
the server files deletion task.

Please help.

Thanks in advance.

-Sameer Shinde

 

[Gordon Beaton]

We have to keep data of 45 days only and have to remove other files
regularly. Currently we use the AWK like utility for the same. It
has some limitations and it do not accept command line parameters
and therefore can not be scheduled as a program.

What should i do to make it work or any other suggstions to carry
out the server files deletion task.

Use find:

find . -mtime +45 -exec /bin/rm {} \;

or

find . -mtime +45 -print0 | xargs -0 /bin/rm

/gordon

--

 

[Lew]

We have one server (FTP). One drive of which stores scanned documents.
Users usually upload documents (.JPG files). Currently that drive is
of about 27 GB containing more that 1,50,000 files.
We have to keep data of 45 days only and have to remove other files
regularly. Currently we use the AWK like utility for the same. It has
some limitations and it do not accept command line parameters and
therefore can not be scheduled as a program.

Now i have written one Java program for the same.
The java file as a Google document:

http://docs.google.com/View?docid=dfvj6k7q_3x4pr28gf

This program wont help. I am running the program from the server.
But it just got hanged after

ile = new File(dirPath);
fileArray = file.listFiles(this);

Not even able to create the fileArray object....

Do increasing memory of server will help or how to change the program?

It's not a memory problem, it's a bug in the way you wrote the program.

I don't think multi-threading like concepts will help as the file
number is big.

Plus, multi-threaded programs are harder to get correct. Get the
single-threaded one right first.

What should i do to make it work or any other suggstions to carry out
the server files deletion task.

Start with the little things - class names really should start with an
upper-case letter, by widespread convention. You really should indent your
code appropriately. Violation of those two conventions made it very difficult
to read the listing.

A good way to name a class is in two parts representing a noun phrase, e.g.,
FileDeleter.

Now the reason for your bug. The class is doing too much work in the
constructor - the constructor should only construct, nothing more. You have
it placing an incomplete object as the FileFilter for the listFiles() call.
No wonder you have trouble.

Build the FileFilter in the constructor, but use it from another method; I
suggest deleteFiles() (which you could rename to, say, run()). That would let
you take the variable 'fileArray' and 'dirPath' out of the member attributes
and make them local variables only. ('dirPath' would be local to the
constructor, and 'fileArray' to the method.)

Speaking of deleteFiles(), variables should not have underscores in their
names, "no_of_files". Use camelCase, where each word part except the first is
capitalized, "noOfFiles".

public class FileDeleter implements Runnable
{
// members should be private!
private final File file;
private final int daysKeep;

// notice the 'public' visibility of the constructor
public FileDeleter( String path, int days )
{
file = new File( path );
if ( file == null || ! file.isDirectory() )
{
String err = dirPath +" not a directory.";
System.err.println( err );
throw new IllegalArgumentException( err );
}
daysKeep = days;
}

@Override public void run()
{
File [] files = file.listFiles( new FileFilter()
{
@Override public boolean accept( File p )
{
return ( ! f.isDirectory() && daysSince( f ) > daysKeep );
}
} );
if ( files != null && files.length > 0 )
{
delete( files );
}
}

private void delete( File [] files )
{
for ( File f : files )
{
f.delete();
}
}

private static final int PERDIEM = 86400000;

private int daysSince( File f )
{
return (int) ( (new Date().getTime() - f.lastModified()) / PERDIEM);
}
}

Throw in some imports and a test client and Bob's your uncle.

 

[Lew]

Use find:

find . -mtime +45 -exec /bin/rm {} \;

or

find . -mtime +45 -print0 | xargs -0 /bin/rm

find . -mtime +45 -delete

 

[Lew]

Lew wrote:

Corrections:

String err = dirPath +" not a directory.";

String err = path +" not a directory.";

@Override public boolean accept( File p )

@Override public boolean accept( File f )

 

[Lew]

find . -mtime +45 -delete

find . -mtime +45 -type f -delete

 

[Lew]

public class FileDeleter implements Runnable
{
// members should be private!
private final File file;
private final int daysKeep;

// notice the 'public' visibility of the constructor
public FileDeleter( String path, int days )
{
file = new File( path );
if ( file == null || ! file.isDirectory() )
{
String err = path +" not a directory.";
System.err.println( err );
throw new IllegalArgumentException( err );
}
daysKeep = days;
}

@Override public void run()
{
File [] files = file.listFiles( new FileFilter()
{
@Override public boolean accept( File f )
{
return ( ! f.isDirectory() && daysSince( f ) > daysKeep );
}

private static final int PERDIEM = 86400000;
private final long now = new Date().getTime();

private int daysSince( File f )
{
return (int) ((now - f.lastModified()) / PERDIEM);
}
} );
if ( files != null && files.length > 0 )
{
delete( files );
}
}

private void delete( File [] files )
{
for ( File f : files )
{
f.delete();
}
}
}