My PERL program slows down over time.

M

Marc

I wrote a PERL program to creat thousands of files. The program zips
along at first but then begins to slow down.

Does any one have any ideas on what might be causing this?

-M
 
G

gnari

Marc said:
I wrote a PERL program to creat thousands of files. The program zips
along at first but then begins to slow down.

Does any one have any ideas on what might be causing this?

yes. you are doing at leas 3 things wrong:

a) you are createing all these files in the same directory, and your filing
system
slows down as the number increases

b) you use a fancy method to make sure the file does not exist already.
for my $fileno (1....10000) {
$subnum=0;
$subnum++ while (-f "file_$subnum");
# create file
}
you have to do more and more work for each file
if you had called your file "file_$fileno"."_$subnum" instead,
you would not have had this problem

c) the logic error on line 42 makes you create file number n
n times.

gnari
 
S

Sam Holden

I wrote a PERL program to creat thousands of files. The program zips
along at first but then begins to slow down.

Does any one have any ideas on what might be causing this?

Most file systems use a linear search through directory entries. Hence file
system operations (opening a file, deleting a file, creating a file, and so
on) slow down as the number of directory entries increase.

Use a file system with a better than linear data structure/algorithm or
create subdirectories based on sum property of the filename (a hash, or
the first letter are commonly used).
 
J

Jürgen Exner

Marc said:
I wrote a PERL program to creat thousands of files. The program zips
along at first but then begins to slow down.

Does any one have any ideas on what might be causing this?

Your while loop on line 42 causes a memory leak.

jue
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,482
Members
44,901
Latest member
Noble71S45

Latest Threads

Top