pass cmd line file glob to grep for readdir

[qanda]

Hi all

I want to process lots of files (too many for the shell to expand on
one line) so I use readdir instead of glob. If I know the file glob
to use such as file*.ext I can put this directly into the grep ...
local @ARGV = grep /file.*\.ext/, readir CURDIR;
My question is how can I take a file glob on the command line and
transform it into an expression for grep. I suppose this also implies
we must prevent the shell from expanding by quoting the expression
such as myscript *file*.ext"

Thanks.

 

[Sam Holden]

My question is how can I take a file glob on the command line and
transform it into an expression for grep. I suppose this also implies
we must prevent the shell from expanding by quoting the expression
such as myscript *file*.ext"

Why not use perl's glob function (perldoc -f glob)?

 

[Eric J. Roode]

(e-mail address removed) (qanda) wrote in @posting.google.com:

Hi all

I want to process lots of files (too many for the shell to expand on

Have you considered using the xargs shell command?

 

[qanda]

Why not use perl's glob function (perldoc -f glob)?

Because it won't work, as perldoc says, it returns the filename
expansions from the shell, the underlying problem is because the shell
has a limit in the number of characters allowed in one line (it may be
big, maybe 1MB) but with thousands of files, each with 30 or 40
character filenames that soon runs out. I don't want the user to try
using find and/or xargs, I just want to call myscript file*.ext or
quoted as in myscript "file*.ext" if need be.

Thanks.

 

[Tad McClellan]

Why not use perl's glob function (perldoc -f glob)?

This is a good idea if you have a perl version >= 5.6.0.

(otherwise the glob will fail because of the shell's limitations)

 

[Tad McClellan]

(e-mail address removed) (qanda) wrote in @posting.google.com:


Have you considered using the xargs shell command?

This is a good idea if you have perl version < 5.6.0.

 

[Sam Holden]

Because it won't work, as perldoc says, it returns the filename
expansions from the shell, the underlying problem is because the shell
has a limit in the number of characters allowed in one line (it may be
big, maybe 1MB) but with thousands of files, each with 30 or 40
character filenames that soon runs out. I don't want the user to try
using find and/or xargs, I just want to call myscript file*.ext or
quoted as in myscript "file*.ext" if need be.

Maybe you should use a perl from this century, since the text:

Beginning with v5.6.0, this operator is implemented using the
standard "File::Glob" extension. See File::Glob for details.

in the documentation seems to disagree with your claim.

It might also be worthwhile finding a non-broken shell.

; cd /dev
; echo * * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
| wc
1 128984 783012
; perl -le 'print join " ", <* * * * * * * * * * * * * * * * * * * * * * *'\
' * * * * * * * * * * * * * * * * * * * * * * *'\
' * * * * * * * * * * * * * * * * * * * * * * *'\
' * * * * * * * * * * * * * * * * * * * * * * *>' | wc
0 128984 783012
; /bin/echo * * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
* * * * * * * * * * * * * * * * * * * * * * * \
| wc
bash: /bin/echo: Argument list too long

The limit is with the operating system, not the shell, so why should perl's
glob have any more difficulty than the shell's?

Have you actually tried it out? My 10 second test above seems to disagree
with your "it won't work" statement.

 

[Tad McClellan]

Because it won't work, as perldoc says, it returns the filename
expansions from the shell,

Only with old perls.

What perl version do you have?

 

[qanda]

Thanks guys and girls, I wasn't aware it had been improved in the new
versions; I'm limited (by customer) to using 5.005.

As for the argument limit, yes the underlying issue is with the OS, as
I understand it the limit is set by ARG_MAX in limits.h.

Would anyone be able to answer the (original) question for me,
certainly if using a newer version of Perl glob would be fine; however
I can't do that for this particular problem and the only overhead I
want for the user is to possibly quote the file glob so we avoid find
and/or xargs, etc.

Thanks again.

 

[qanda]

Sorry for posting on top of my last reply, but I really do need help
with this one. I will be forever (well quite some time at least) in
your debt if you can help