File Monitoring modules

N

nagandla

Hi All,

For monitoring files in directories i am using SGI::FAM module...
this module is monitoring perfectlt but taking more 90% CPU usage
while monitoring ...
so any one can suggest me the better module to monitor the files in
directoties....here one more thing i want to monitor more than a
single directory at a time...

Regards ,
Lakshmi Nagandla
 
B

Ben Morrow

Quoth (e-mail address removed):
For monitoring files in directories i am using SGI::FAM module...
this module is monitoring perfectlt but taking more 90% CPU usage
while monitoring ...

It is polling, rather than using a kernel-based notification mechanism.
What OS are you on?
so any one can suggest me the better module to monitor the files in
directoties....

There are OS-specific interfaces to most of the file-change services I
know of (kqueue, inotify, Win32's ChangeNotify). If you know what your
OS supports, you can search for a direct interface to that.
here one more thing i want to monitor more than a
single directory at a time...

All of the above will do that.

Ben
 
N

nagandla

Quoth (e-mail address removed):




It is polling, rather than using a kernel-based notification mechanism.
What OS are you on?


There are OS-specific interfaces to most of the file-change services I
know of (kqueue, inotify, Win32's ChangeNotify). If you know what your
OS supports, you can search for a direct interface to that.


All of the above will do that.

Ben


My OS is Linux,,Dapper Drake Ubuntu System
 
B

Ben Morrow

Quoth (e-mail address removed):
My OS is Linux,,Dapper Drake Ubuntu System

Huh. fam(3) ought to work perfectly well under Linux... If you write a
minimal C program which watches the same directories, does it have the
same behaviour? If so, this is a problem with fam on your system, and
you need to take it up with the developers, if there are any. If there
are, they'll probably tell you to switch to gamin; making SGI::FAM work
with gamin has been on my todo list for a while now... :)

Ben
 
N

nagandla

Quoth (e-mail address removed):



Huh. fam(3) ought to work perfectly well under Linux... If you write a
minimal C program which watches the same directories, does it have the
same behaviour? If so, this is a problem with fam on your system, and
you need to take it up with the developers, if there are any. If there
are, they'll probably tell you to switch to gamin; making SGI::FAM work
with gamin has been on my todo list for a while now... :)

Ben


Ya thank U Ben,

Just now i found one more module for monitoring is File::Monitor....
It is also all most all same as fam...and also taking equivalent time
to moniotr when a file is newly created or deleted....
and also it is taking very less CPU usage (below 20%) while monitoring
directories...
Have a look on that....and suggest me that whether can i use that...
 
T

Ted Zlatanov

On Mon, 21 Jan 2008 23:29:20 -0800 (PST) (e-mail address removed) wrote:

n> For monitoring files in directories i am using SGI::FAM module...
n> this module is monitoring perfectlt but taking more 90% CPU usage
n> while monitoring ... so any one can suggest me the better module to
n> monitor the files in directoties....here one more thing i want to
n> monitor more than a single directory at a time...

It could be a bug in your code. Can you show the code?

Use `top' or something similar to see what's actually using the CPU. Is
it the FAM daemon or your code? You could be in a busy loop...

Ted
 
N

nagandla

On Mon, 21 Jan 2008 23:29:20 -0800 (PST) (e-mail address removed) wrote:

n> For monitoring files in directories i am using SGI::FAM module...
n> this module is monitoring perfectlt but taking more 90% CPU usage
n> while monitoring ... so any one can suggest me the better module to
n> monitor the files in directoties....here one more thing i want to
n> monitor more than a single directory at a time...

It could be a bug in your code. Can you show the code?

Use `top' or something similar to see what's actually using the CPU. Is
it the FAM daemon or your code? You could be in a busy loop...

Ted



Here is the code to monitor directories using Fam....
use strict;
use warnings;

use SGI::FAM;
my $fam;
my $terminate = 0;
my $path = '/data';
my %interested_dirs = ();
sub today_datetime
{
my($ss,$mi,$hh,$day,$mon,$year) = localtime();
$mon++;
$year+=1900;
my $now = "$year-$mon-$day:$hh:$mi:$ss";
return($now);
}

opendir(DIR,$path);
my @file_s = readdir(DIR);
foreach(@file_s)
{
if ($_ !~ /\.$/)
{
if(-d "$path/$_")
{
# print "$_ <-> $path/$_ \n ";
$interested_dirs{$_} = $path."/".$_ ;
}
}
}
closedir(DIR);

until ($terminate)
{
unless (defined($fam))
{
eval { $fam = new SGI::FAM; };
if ($@)
{
print "Failed to connect to FAMd - starting forcibly...\n\n\n";
system("/usr/local/sbin/famd");
next;
}


die "FAM Failed: $!\n" unless (defined($fam));
foreach (values %interested_dirs)
{
$fam->monitor($_);
#print "$_\n";
}
}

# check for FAM events
if ($fam->pending)
{
my $event;
eval { $event = $fam->next_event; };

if ($@)
{
print "FAM error: rebuilding cache...\n\n\n";
$fam = undef;
next;
}
#print "Received: " . $event->type . " for " . $event->filename .
"\n";
&process_fam_event($event);
}

}
sub process_fam_event
{
my $event = shift;
my $filename = $event->filename;
my $dir = $fam->which($event);
my $now = &today_datetime;
if($event->type eq 'create')
{
if($filename ne $dir)
{
my $subdir = "$dir/$filename";
while(`/usr/sbin/lsof +d $dir | grep "$filename"`)
{
next;
}

if(-d $subdir)
{
print "Creating $subdir \n";
eval { $fam->monitor($subdir); };
if($@)
{
print "Fam Error : Failed to monitor $subdir : $@ \n";
}
}
}

print "Createed--------NewFile : $filename , Dir : $dir $now\n";

}elsif($event->type eq 'delete')
{
if($filename ne $dir)
{
my $subdir = "$dir/$filename";
while(`/usr/sbin/lsof +d $dir | grep "$filename"`)
{
next;
}

if(-d $subdir)
{
print "Deleting $subdir \n";
eval { $fam->cancel($subdir); };
if($@)
{
print "Fam Error : Failed to monitor $subdir : $@ \n";
}
}
}

print "Delete : $filename , Dir : $dir $now\n";
}

}


Nagandla
 
T

Ted Zlatanov

On Thu, 24 Jan 2008 01:22:57 -0800 (PST) (e-mail address removed) wrote:

n> until ($terminate)
n> {
....
n> # check for FAM events
n> if ($fam->pending)
n> {
....
n> &process_fam_event($event);
n> }
n> }


Looks like you are just running this in a busy loop, meaning you never
pause to give the rest of the system a chance to catch up. Your process
will be preempted by the kernel if necessary, but otherwise it will take
up all the CPU (which is what you're seeing). Try a sleep(1) for a
1-second pause at the end of the until() loop and see if your CPU
utilization drops; it should.

Ted
 
X

xhoster

# check for FAM events
if ($fam->pending)
{
my $event;
eval { $event = $fam->next_event; };

if ($@)
{
print "FAM error: rebuilding cache...\n\n\n";
$fam = undef;
next;
}


pending does not block, it returns immediately, and hence forms a tight
loop. Since that isn't what you want to do, then don't do that. Skip
the pending and just use the next_event. That is, replace the above with:

# check for FAM events
my $event = eval { $fam->next_event; };
if ($@) {
print "FAM error: rebuilding cache...\n\n\n";
$fam = undef;
next;
};


Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
N

nagandla

pending does not block, it returns immediately, and hence forms a tight
loop. Since that isn't what you want to do, then don't do that. Skip
the pending and just use the next_event. That is, replace the above with:

# check for FAM events
my $event = eval { $fam->next_event; };
if ($@) {
print "FAM error: rebuilding cache...\n\n\n";
$fam = undef;
next;
};

Xho

--
--------------------http://NewsReader.Com/--------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.


Ya,,Thank U..
As u said pending event is taking so much CPU usage more than(90%)...
I removed the pending event so that the CPU usage came more or less
equal to (5%).....

Thank u

Regards
Nagandla
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,048
Latest member
verona

Latest Threads

Top