Need help in comparing permissions/ownerships on different systems

[snoopy_]

Hello,
I am looking for help or suggestion for a way to compare
filesystems between two different systems. I have two development
environments, one is stable the other is not. I like to get a report
of how similar they are, and if the permissions are similar.

I tried rsync, but it doesn't tell you what is wrong with the file
or why it needs to be synced.

The other method is to run a find on one server, capture that as a
source csv file, then write a script to run through all those files and
compare them against the second hosts. I tried to use Perl's
File::stat for this, but was unsuccessful. I resorted to the
following:

find /d00 -print | xargs ls -ld | awk '{print $1","$3","$4","$9}' | tee
/tmp/find.out.csv

It results in a csv file contining perms on one system, then I could
use this to compare:

-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/ss.runtime.tar
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.modules.dat
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.runtime.tar

Not how to interpret "-rwx" fields. Thought of perl with expression
matching, or using the File:stat modules. Doesn't seem pratical.

Any other ideas?

Thanks,

Snoopy_

 

[Ayaz Ahmed Khan]

"(e-mail address removed)" typed:

[Follow-up set to comp.lang.perl.misc]

It results in a csv file contining perms on one system, then I could
use this to compare:

-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/ss.runtime.tar
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.modules.dat
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.runtime.tar

Not how to interpret "-rwx" fields. Thought of perl with expression
matching, or using the File:stat modules. Doesn't seem pratical.

Any other ideas?

I once had a need to do something similar. I never completed the
application. The need either thined away into insignificance, or the
damage done wasn't so serious as had been presumed initially (the lead web
developer had accidentaly run `chmod -R 777 /` on one production system I
administrate). Nonetheless, the following snippet from the application
might help you. You might want to use a different module to traverse the
filesystem, as File::Glob doesn't quite work as I wanted it to.

sub read_filesystem
{
my $startpath = shift;

# Limitation: Doesn't do recursive globbing.
use File::Glob ':glob';
my @files = glob($startpath);

use File::Stat::Ls qwall);

foreach my $file (@files) {
my @filestat = stat($file);

my ($mode, $uid, $gid) = ($filestat[2], $filestat[4], $filestat[5]);

my $file_obj = File::Stat::Ls->new;

my $perm = $file_obj->format_mode($mode);
my $octal = &convert_to_octal($perm);

# Quick hack to workaround the fact that directory entries read from
# MANIFEST have a trailing "/" character attached to them. Globbing
# does not behave the same way with respect to directory entries.
$file .= "/" if -d $file;
$fs_live{$file} = [$uid, $gid, $perm, $octal];
}
}


###
### Given a string like "-rwx-r-xr-x", return equivalent octal representation.
###
sub convert_to_octal
{
my $perm = shift;
my $octal;

my ($special, $user, $group, $other) = (0, 0, 0, 0);

my ($x, $ur, $uw, $ux, $gr, $gw, $gx, $or, $ow, $ox) = split //, $perm;

$user += 4 if $ur eq 'r';
$user += 2 if $uw eq 'w';
$user += 1 if $ux eq 'x';
if ($ux eq 's') { $user += 1; $special += 4; }

$group += 4 if $gr eq 'r';
$group += 2 if $gw eq 'w';
$group += 1 if $gx eq 'x';
if ($gx eq 's') { $group += 1; $special += 2; }

$other += 4 if $or eq 'r';
$other += 2 if $ow eq 'w';
$other += 1 if $ox eq 'x';
if ($ox eq 't') { $other += 1; $special += 1; }

$octal = $special . $user . $group . $other;

return $octal;
}

I suggest reading through a few tutorials explaining Unix/Linux file modes
and permissions.

 

[snoopy_]

Ayaz,
Thanks.

I have started down both paths that you recommend only to choke.
For one, the octal conversion for pattern matching might be a high
overhead for a program; every file it would have to figure out the
octal? I think I have decided to just match the whole expression.
I.E. If ( "-rwxrw-r--" -eq "-rw-r--r--)....

The other issue is for me, GID and UID do not seem to be reported
properly via File::stat. They report gid and uid as "0"

My file: mister

ls -al mister
-rw-r----- 1 root root 25 Dec 14 22:35 mister


############## my prog: stats.pl #####################
#!/usr/bin/perl

use File::stat;
$filename="mister";
$st = stat($filename);
$dev = $st->dev;
$ino = $st->ino;
$mode = $st->mode;
$nlink = $st->nlink;
$uid = $st->uid;
$gid= $st->gid;
$rdev = $st->rdev;
$size = $st->size;
$atime = $st->atime;
$mtime = $st->mtime;
$ctime = $st->ctime;
$blksize = $st->blksize;
$blocks = $st->blocks;

print "dev: $dev\n";
print "ino: $ino\n";
print "mode: $mode\n";
print "nlink: $nlink\n";
print "uid: $uid\n";
print "gid: $gid\n";
print "rdev: $rdev\n";
print "size: $size\n";
print "atime: $atime\n";
print "mtime: $mtime\n";
print "ctime: $ctime\n";
print "blksize: $blksize\n";
print "blocks: $blocks\n";
###########################################

Output from stats.pl

mmontone@sl39g03 stats.pl
dev: 71565312
ino: 428306
mode: 33184
nlink: 1
uid: 0
gid: 0
rdev: 0
size: 25
atime: 1166136325
mtime: 1166135724
ctime: 1166203392
blksize: 8192
blocks: 2

##########################################

"(e-mail address removed)" typed:

[Follow-up set to comp.lang.perl.misc]

It results in a csv file contining perms on one system, then I could
use this to compare:

-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/ss.runtime.tar
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.modules.dat
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.runtime.tar

Not how to interpret "-rwx" fields. Thought of perl with expression
matching, or using the File:stat modules. Doesn't seem pratical.

Any other ideas?

I once had a need to do something similar. I never completed the
application. The need either thined away into insignificance, or the
damage done wasn't so serious as had been presumed initially (the lead web
developer had accidentaly run `chmod -R 777 /` on one production system I
administrate). Nonetheless, the following snippet from the application
might help you. You might want to use a different module to traverse the
filesystem, as File::Glob doesn't quite work as I wanted it to.

sub read_filesystem
{
my $startpath = shift;

# Limitation: Doesn't do recursive globbing.
use File::Glob ':glob';
my @files = glob($startpath);

use File::Stat::Ls qwall);

foreach my $file (@files) {
my @filestat = stat($file);

my ($mode, $uid, $gid) = ($filestat[2], $filestat[4], $filestat[5]);

my $file_obj = File::Stat::Ls->new;

my $perm = $file_obj->format_mode($mode);
my $octal = &convert_to_octal($perm);

# Quick hack to workaround the fact that directory entries read from
# MANIFEST have a trailing "/" character attached to them. Globbing
# does not behave the same way with respect to directory entries.
$file .= "/" if -d $file;
$fs_live{$file} = [$uid, $gid, $perm, $octal];
}
}


###
### Given a string like "-rwx-r-xr-x", return equivalent octal representation.
###
sub convert_to_octal
{
my $perm = shift;
my $octal;

my ($special, $user, $group, $other) = (0, 0, 0, 0);

my ($x, $ur, $uw, $ux, $gr, $gw, $gx, $or, $ow, $ox) = split //, $perm;

$user += 4 if $ur eq 'r';
$user += 2 if $uw eq 'w';
$user += 1 if $ux eq 'x';
if ($ux eq 's') { $user += 1; $special += 4; }

$group += 4 if $gr eq 'r';
$group += 2 if $gw eq 'w';
$group += 1 if $gx eq 'x';
if ($gx eq 's') { $group += 1; $special += 2; }

$other += 4 if $or eq 'r';
$other += 2 if $ow eq 'w';
$other += 1 if $ox eq 'x';
if ($ox eq 't') { $other += 1; $special += 1; }

$octal = $special . $user . $group . $other;

return $octal;
}

I suggest reading through a few tutorials explaining Unix/Linux file modes
and permissions.

--
Ayaz Ahmed Khan

It is impossible to defend perfectly against the attack of those who
want to die.

 

[snoopy_]

Never mind, I kust realized that root was both owner and group, that
is why stats.pl reported "0" Change owner and stats.pl reports
correctly now. Now I need to fine tune it.

"(e-mail address removed)" typed:

[Follow-up set to comp.lang.perl.misc]

It results in a csv file contining perms on one system, then I could
use this to compare:

-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/ss.runtime.tar
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.modules.dat
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.runtime.tar

Not how to interpret "-rwx" fields. Thought of perl with expression
matching, or using the File:stat modules. Doesn't seem pratical.

Any other ideas?

I once had a need to do something similar. I never completed the
application. The need either thined away into insignificance, or the
damage done wasn't so serious as had been presumed initially (the lead web
developer had accidentaly run `chmod -R 777 /` on one production system I
administrate). Nonetheless, the following snippet from the application
might help you. You might want to use a different module to traverse the
filesystem, as File::Glob doesn't quite work as I wanted it to.

sub read_filesystem
{
my $startpath = shift;

# Limitation: Doesn't do recursive globbing.
use File::Glob ':glob';
my @files = glob($startpath);

use File::Stat::Ls qwall);

foreach my $file (@files) {
my @filestat = stat($file);

my ($mode, $uid, $gid) = ($filestat[2], $filestat[4], $filestat[5]);

my $file_obj = File::Stat::Ls->new;

my $perm = $file_obj->format_mode($mode);
my $octal = &convert_to_octal($perm);

# Quick hack to workaround the fact that directory entries read from
# MANIFEST have a trailing "/" character attached to them. Globbing
# does not behave the same way with respect to directory entries.
$file .= "/" if -d $file;
$fs_live{$file} = [$uid, $gid, $perm, $octal];
}
}


###
### Given a string like "-rwx-r-xr-x", return equivalent octal representation.
###
sub convert_to_octal
{
my $perm = shift;
my $octal;

my ($special, $user, $group, $other) = (0, 0, 0, 0);

my ($x, $ur, $uw, $ux, $gr, $gw, $gx, $or, $ow, $ox) = split //, $perm;

$user += 4 if $ur eq 'r';
$user += 2 if $uw eq 'w';
$user += 1 if $ux eq 'x';
if ($ux eq 's') { $user += 1; $special += 4; }

$group += 4 if $gr eq 'r';
$group += 2 if $gw eq 'w';
$group += 1 if $gx eq 'x';
if ($gx eq 's') { $group += 1; $special += 2; }

$other += 4 if $or eq 'r';
$other += 2 if $ow eq 'w';
$other += 1 if $ox eq 'x';
if ($ox eq 't') { $other += 1; $special += 1; }

$octal = $special . $user . $group . $other;

return $octal;
}

I suggest reading through a few tutorials explaining Unix/Linux file modes
and permissions.

--
Ayaz Ahmed Khan

It is impossible to defend perfectly against the attack of those who
want to die.

 

[Ben Morrow]

[don't top-post]

I have started down both paths that you recommend only to choke.
For one, the octal conversion for pattern matching might be a high
overhead for a program; every file it would have to figure out the
octal? I think I have decided to just match the whole expression.
I.E. If ( "-rwxrw-r--" -eq "-rw-r--r--)....

Depending on exactly what you are trying to match, it may be easier if
you can get the modes as numbers. Perl's oct function will convert an
octal representation of a number into a number, if it's easier to get
them in octal, or the conversion from text is pretty simple, something
like

my $modestring = '-rwxrw-r--';
$modestring =~ s/^.//;
my $mode = 0;
$mode *= 2, $mode += ($1 ne '-') while $modestring =~ s/^(.)//;

plus a bit more fiddling to handle the setid bits; or, alternatively, an
easier option may be to get raw 'ls -lR' listing and use File::Listing
to parse it. Anyway, once you've got a number, you can compare them with
the bitmask operators, so if for instance you don't care about the
owner-write bit you use someting like

my $mode1 = 0644;
my $mode2 = 0444;

print 'equivalent' if ($mode1 & 0577) == ($mode2 & 0577);

I think you perhaps need to leran a little more about how your operating
system works: I would recommend you find a decent beginners' book on
Unix.

The other issue is for me, GID and UID do not seem to be reported
properly via File::stat. They report gid and uid as "0"

My file: mister

ls -al mister
-rw-r----- 1 root root 25 Dec 14 22:35 mister

This file has a uid and gid of 0. File owners and groups are numerical;
you can convert them to user- and group-names with User:went and
User::grent (since you, like me, seem to prefer the by-name interface to
Perl's native lists):

use User:went;
use User::grent;

my $stat = stat("mister");
my $user = getpw($stat->uid)->name;
my $group = getpw($stat->gid)->name;

############## my prog: stats.pl #####################
#!/usr/bin/perl

Where are

use strict;
use warnings;

? Have you read the Posting Guidelines?

use File::stat;
$filename="mister";
$st = stat($filename);
$dev = $st->dev;
$ino = $st->ino;

print "dev: $dev\n";
print "ino: $ino\n";

You *know* there's got to be an easier way than that, right? I don't
usually use printf, but this seems to be a perfect occasion for it:

my $filename = "mister";
my $st = stat($filename);

printf "%-7s: %s\n", $_, $st->$_ for qw/dev ino mode etc./;

Ben

 

[Hemal Pandya]

Hello,
I am looking for help or suggestion for a way to compare
filesystems between two different systems. I have two development
environments, one is stable the other is not. I like to get a report
of how similar they are, and if the permissions are similar.

I tried rsync, but it doesn't tell you what is wrong with the file
or why it needs to be synced.

Actually it does. Try the --dry-run option with -vv.

The other method is to run a find on one server, capture that as a
source csv file, then write a script to run through all those files and
compare them against the second hosts. I tried to use Perl's
File::stat for this, but was unsuccessful. I resorted to the
following:

find /d00 -print | xargs ls -ld | awk '{print $1","$3","$4","$9}' | tee
/tmp/find.out.csv

It results in a csv file contining perms on one system, then I could
use this to compare:

-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/ss.runtime.tar
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.modules.dat
-rw-rw----,egatereg,egateg,/d00/app/home/egatereg/schemas/trn.runtime.tar

Not how to interpret "-rwx" fields. Thought of perl with expression
matching, or using the File:stat modules. Doesn't seem pratical.

Are you looking to separate the three "rwx"? Look at cut.

Even without that, comparing the output on the dev and production boxes
using diff will give you the differences. If the production and dev
boxes are rooted in different directories,

Any other ideas?

What does this have to do with java?

 

[Ayaz Ahmed Khan]

"(e-mail address removed)" typed:

I have started down both paths that you recommend only to choke.
For one, the octal conversion for pattern matching might be a high
overhead for a program; every file it would have to figure out the
octal? I think I have decided to just match the whole expression.
I.E. If ( "-rwxrw-r--" -eq "-rw-r--r--)....

Yes, you can do without converting permission strings into equivalent
octal representations. I needed that in my script because the missing rest
part of that script, which I will work on in future, deals with
*resetting* the permissions of files whose permissions look warped.