search and replace in a file tree, only for files that match

[Michael Friendly]

I'm wondering if there is a perl script or a simpler way to accomplish
the task of doing global search/replace in all files, possibly of
specified file types in a file tree, where *only* the files that match
the search string will be modified (because I need to preserve time
stamps so that only truly modified files will be rsync'd with a server).

Here's what I do now, using modified version of tcgrep (supporting a -E
flag to limit search to specified file extensions) to generate the list
of files to which the perl -pi~ command will apply:

% perl -pi~ -e
's|http://www.math.yorku.ca/SCS/Online|http://www.datavis.ca/online|g'
`tcgrep -lE html '/SCS/Online' .`

That is, I want to do the global search/replace, only among those files
whose names are returned by the tcgrep command.

I'm doing this a lot in moving stuff from one web server to another,
so it would be nice to simplify this someway.

-Michael

--
Michael Friendly Email: (e-mail address removed)
Professor, Psychology Dept.
York University Voice: 416 736-5115 x66249 Fax: 416 736-5814
4700 Keele Street http://www.math.yorku.ca/SCS/friendly.html
Toronto, ONT M3J 1P3 CANADA

 

[jl_post]

Here's what I do now, using modified version of tcgrep (supporting a -E
flag to limit search to specified file extensions) to generate the list
of files to which the perl -pi~ command will apply:

% perl -pi~ -e
's|http://www.math.yorku.ca/SCS/Online|http://www.datavis.ca/online|g'
`tcgrep -lE html '/SCS/Online' .`

That is, I want to do the global search/replace, only among those files
whose names are returned by the tcgrep command.

Dear Michael,

I confess that I'm not familiar with the tcgrep command. However,
I think you can do what you want by just setting the @ARGV array to
whatever files you want inside a BEGIN block.

So you might use something like this:

perl -pi~ -e 'BEGIN{@ARGV=`command`; chomp(@ARGV)} s/old/new/'

For your purposes, "command" would probably be:

tcgrep -lE html '/SCS/Online' .

I hope this helps, Michael.

-- Jean-Luc

 

[Jim Gibson]

I'm wondering if there is a perl script or a simpler way to accomplish
the task of doing global search/replace in all files, possibly of
specified file types in a file tree, where *only* the files that match
the search string will be modified (because I need to preserve time
stamps so that only truly modified files will be rsync'd with a server).

Here's what I do now, using modified version of tcgrep (supporting a -E
flag to limit search to specified file extensions) to generate the list
of files to which the perl -pi~ command will apply:

% perl -pi~ -e
's|http://www.math.yorku.ca/SCS/Online|http://www.datavis.ca/online|g'
`tcgrep -lE html '/SCS/Online' .`

That is, I want to do the global search/replace, only among those files
whose names are returned by the tcgrep command.

I'm doing this a lot in moving stuff from one web server to another,
so it would be nice to simplify this someway.

You can certainly implement this procedure with a Perl program using
standard Perl features and modules. I would:

1. Find all of the files in a directory (or directory tree) using
opendir, readdir (just one directory) or File::Find (directory tree).
2. Ignore files without the proper suffix.
3. Read a candidate file and see if it contains the string.
4. If it does, then seek to the beginning of the file, re-read it,
copy it to a temporary, change the lines as needed, rename the file as
a backup, and rename the temporary file to the original name.

If the files are short, you can avoid re-reading the file by reading
the lines into an array, searching the array for the string,
substituting if necessary, and writing out the modified file if any
lines have been modified.

You could also use the Tie::File module to avoid coding the reading and
writing steps.

 

[Uri Guttman]

JG> 1. Find all of the files in a directory (or directory tree) using
JG> opendir, readdir (just one directory) or File::Find (directory tree).
JG> 2. Ignore files without the proper suffix.
JG> 3. Read a candidate file and see if it contains the string.
JG> 4. If it does, then seek to the beginning of the file, re-read it,
JG> copy it to a temporary, change the lines as needed, rename the file as
JG> a backup, and rename the temporary file to the original name.

JG> If the files are short, you can avoid re-reading the file by reading
JG> the lines into an array, searching the array for the string,
JG> substituting if necessary, and writing out the modified file if any
JG> lines have been modified.

or even simpler, use file::slurp. why do seeking and such which is a
waste?

untested pseudo code. run this on each file. the s/// needs to be done
with the proper args.

use File::Slurp ;

my $text = read_file( $file_name ) ;

if ( $text =~ s/$old/$new/g ) {

write_file( $file_name, $text ) ;
}


you can add the atomic option to write_file if you want to make sure you
always have a clean file present. this is not difficult to do. the
harder part is finding all the files you want first with file::find. you
still need to scan them to see if they are needing the change. whether
that is done externally with a grep or internally with perl's s/// is a
minor issue. my version does the scan and replace in one step which may
be faster in some cases (where most of the files needed
replacements). in either case, the code is very simple.

uri

 

[Ilya Zakharevich]

pfind

should be on ilyaz.org/software/tmp

JG> 1. Find all of the files in a directory (or directory tree) using
JG> opendir, readdir (just one directory) or File::Find (directory tree).
JG> 2. Ignore files without the proper suffix.
JG> 3. Read a candidate file and see if it contains the string.
JG> 4. If it does, then seek to the beginning of the file, re-read it,
JG> copy it to a temporary, change the lines as needed, rename the file as
JG> a backup, and rename the temporary file to the original name.

pfind . -f '/\.suff$/' '=~ /pattern/' '=~ s/pattern2/replace/g'

This would not keep backup, but you can easily add this via File::Copy::copy.

Hope this helps,
Ilya

 

[Jürgen Exner]

I'm wondering if there is a perl script or a simpler way to accomplish
the task of doing global search/replace in all files, possibly of
specified file types in a file tree, where *only* the files that match
the search string will be modified (because I need to preserve time
stamps so that only truly modified files will be rsync'd with a server).

File::Find is your friend. In the wanted() function you can specify
whatever search criteria comes to your mind.

jue