Finding Duplicate File Objects

L

Luc The Perverse

I'm rewriting my duplicate file checker program so it doesn't require
recompilation every time I want to use it. (I'm basically starting over
from scratch.)

The problem is, I have no way to prevent a user from adding a directory
twice, or adding a directory inside of a directory.

What is the best way to get rid of duplicate files (right now they are going
into a linked list, but I can change that.)

Here is the code - it doesn't really do much yet (but if you know I'm
looking for duplicate File objects in a linked list I don't think you should
need it.)

import java.util.*;
import java.io.*;
import java.security.*;

class FileFilter{
String[] extInclude = {};
String[] extExclude = {};
long minFileSize = 0; // 0
long maxFileSize = 2147483647; // 2 Gb
boolean matches(File cur){
long curL= cur.length();
if(curL<minFileSize || curL>maxFileSize)
return false;
int i;
String v = cur.getAbsolutePath();
if(extExclude != null)
for(i = 0;i<extExclude.length;i++)
if(extExclude.compareToIgnoreCase(v.substring(v.length()-extExclude.length()))==0)
return false;
if(extInclude == null)
return true;
for(i = 0;i<extInclude.length;i++)
if(extInclude.compareToIgnoreCase(v.substring(v.length()-extInclude.length()))==0)
return true;
return false||(extInclude.length==0);
}
}

public class FileUtil{
public static boolean DoDir(LinkedList<File> fileList, String dirName,
FileFilter filter){
File theDir = new File(dirName);
if(!theDir.exists())
return false;
LinkedList<File> directoryList = new LinkedList<File>();

directoryList.add(theDir);
while(!directoryList.isEmpty()){
File cur = directoryList.removeFirst();
String[] list = cur.list();
if(list==null)
continue;
for (int i=0; i<list.length; i++){
File looking = new File(cur, list);
if(looking.isDirectory())
directoryList.add(looking);
else{
if(looking.canRead()&&(filter==null || filter.matches(looking)))
fileList.add(looking);
}
}
}
return true;
}

public static void main(String[] h){
int i;
FileFilter myFilter = new FileFilter();
LinkedList<File> FileList = new LinkedList<File>();
//--
if(h.length==0){
System.out.println("\"p=PATH\""); //param 0
System.out.println("e=.EXT (include)"); //param 1
System.out.println("E=.EXT (exclude)"); //param 2
System.out.println("f=MINFILESIZE"); //param 3
System.out.println("F=MAXFILESIZE"); //param 4
return;
}
int[] parameterCount = new int[20];
for(i=0;i<h.length;i++){
String curString = h;
if(curString.length()<3||curString.charAt(1)!='='||!(curString.charAt(0)>='a'&&curString.charAt(0)<='z'||curString.charAt(0)>='A'&&curString.charAt(0)<='Z')){
System.out.println("Improperly formatted parameter - " + curString);
}
switch(curString.charAt(0)){
case 'p': parameterCount[0]++; break;
case 'e': parameterCount[1]++; break;
case 'E': parameterCount[2]++; break;
case 'f': parameterCount[3]++; break;
case 'F': parameterCount[4]++; break;
default: System.out.println("Unrecognized parameter - " +
curString);
}
}
if(parameterCount[1]>0&&parameterCount[2]>0){
System.out.println("I'm sorry - until wildcards are implemented,
include and exclude extensions are mutually exclusive.");
return;
}
String[] extensions = null;
if(parameterCount[1]>0||parameterCount[2]>0){
extensions = new String[parameterCount[1]+parameterCount[2]];
int extCount = 0;
for(i=0;i<h.length;i++){
String curString2 = h;
if(curString2.charAt(0)=='e'||curString2.charAt(0)=='E')
extensions[extCount++] = curString2.substring(2);
}
if(parameterCount[1]>0)
myFilter.extInclude = extensions;
else
myFilter.extExclude = extensions;
}
if(parameterCount[3]>1){
System.out.println("Warning: Redundant min file size
declarations. First one will be used.");
}
if(parameterCount[3]>0)
for(i=0;i<h.length;i++){
String curString2 = h;
if(curString2.charAt(0)=='f'){
myFilter.minFileSize=Integer.decode(curString2.substring(2))
;
break;
}
}
if(parameterCount[4]>1){
System.out.println("Warning: Redundant max file size
declarations. First one will be used.");
}
if(parameterCount[4]>0)
for(i=0;i<h.length;i++){
String curString2 = h;
if(curString2.charAt(0)=='F'){
myFilter.maxFileSize=Integer.decode(curString2.substring(2))
;
break;
}
}
if(myFilter.maxFileSize < myFilter.minFileSize){
System.out.println("Max file size is set lower than min file
size!");

}

for(i=0;i<h.length;i++){
String curString = h;
if(curString.length()<3)
continue;
if(curString.charAt(0)=='p'&&curString.charAt(1)=='=')
DoDir(FileList, curString.substring(2), myFilter);
}
System.out.println(FileList.size()); //display report of how many files
for debug purposes
}
}
 
R

Roedy Green

The problem is, I have no way to prevent a user from adding a directory
twice, or adding a directory inside of a directory.
you could look at the source code for the Replicator to see how I
handle that. Basically you have a HashMap of filenames in compact
canonical form.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top