pass cmd line file glob to grep for readdir

Discussion in 'Perl Misc' started by qanda, Sep 9, 2003.

  1. qanda

    qanda Guest

    Hi all

    I want to process lots of files (too many for the shell to expand on
    one line) so I use readdir instead of glob. If I know the file glob
    to use such as file*.ext I can put this directly into the grep ...
    local @ARGV = grep /file.*\.ext/, readir CURDIR;
    My question is how can I take a file glob on the command line and
    transform it into an expression for grep. I suppose this also implies
    we must prevent the shell from expanding by quoting the expression
    such as myscript *file*.ext"

    Thanks.
    qanda, Sep 9, 2003
    #1
    1. Advertising

  2. qanda

    Sam Holden Guest

    On 8 Sep 2003 22:57:15 -0700, qanda <> wrote:
    >
    > My question is how can I take a file glob on the command line and
    > transform it into an expression for grep. I suppose this also implies
    > we must prevent the shell from expanding by quoting the expression
    > such as myscript *file*.ext"


    Why not use perl's glob function (perldoc -f glob)?

    --
    Sam Holden
    Sam Holden, Sep 9, 2003
    #2
    1. Advertising

  3. (qanda) wrote in news:62b4710f.0309082157.4c16ee32
    @posting.google.com:

    > Hi all
    >
    > I want to process lots of files (too many for the shell to expand on


    Have you considered using the xargs shell command?

    --
    Eric
    $_ = reverse sort $ /. r , qw p ekca lre uJ reh
    ts p , map $ _. $ " , qw e p h tona e and print
    Eric J. Roode, Sep 9, 2003
    #3
  4. qanda

    qanda Guest

    > Why not use perl's glob function (perldoc -f glob)?

    Because it won't work, as perldoc says, it returns the filename
    expansions from the shell, the underlying problem is because the shell
    has a limit in the number of characters allowed in one line (it may be
    big, maybe 1MB) but with thousands of files, each with 30 or 40
    character filenames that soon runs out. I don't want the user to try
    using find and/or xargs, I just want to call myscript file*.ext or
    quoted as in myscript "file*.ext" if need be.

    Thanks.
    qanda, Sep 9, 2003
    #4
  5. Sam Holden <> wrote:
    > On 8 Sep 2003 22:57:15 -0700, qanda <> wrote:
    >>
    >> My question is how can I take a file glob on the command line and
    >> transform it into an expression for grep. I suppose this also implies
    >> we must prevent the shell from expanding by quoting the expression
    >> such as myscript *file*.ext"

    >
    > Why not use perl's glob function (perldoc -f glob)?



    This is a good idea if you have a perl version >= 5.6.0.

    (otherwise the glob will fail because of the shell's limitations)


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Sep 9, 2003
    #5
  6. Eric J. Roode <> wrote:
    > (qanda) wrote in news:62b4710f.0309082157.4c16ee32
    > @posting.google.com:
    >
    >> Hi all
    >>
    >> I want to process lots of files (too many for the shell to expand on

    >
    > Have you considered using the xargs shell command?



    This is a good idea if you have perl version < 5.6.0.


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Sep 9, 2003
    #6
  7. qanda

    Sam Holden Guest

    On 9 Sep 2003 05:53:29 -0700, qanda <> wrote:
    >> Why not use perl's glob function (perldoc -f glob)?

    >
    > Because it won't work, as perldoc says, it returns the filename
    > expansions from the shell, the underlying problem is because the shell
    > has a limit in the number of characters allowed in one line (it may be
    > big, maybe 1MB) but with thousands of files, each with 30 or 40
    > character filenames that soon runs out. I don't want the user to try
    > using find and/or xargs, I just want to call myscript file*.ext or
    > quoted as in myscript "file*.ext" if need be.


    Maybe you should use a perl from this century, since the text:

    Beginning with v5.6.0, this operator is implemented using the
    standard "File::Glob" extension. See File::Glob for details.

    in the documentation seems to disagree with your claim.

    It might also be worthwhile finding a non-broken shell.

    ; cd /dev
    ; echo * * * * * * * * * * * * * * * * * * * * * * * \
    * * * * * * * * * * * * * * * * * * * * * * * \
    * * * * * * * * * * * * * * * * * * * * * * * \
    * * * * * * * * * * * * * * * * * * * * * * * \
    | wc
    1 128984 783012
    ; perl -le 'print join " ", <* * * * * * * * * * * * * * * * * * * * * * *'\
    ' * * * * * * * * * * * * * * * * * * * * * * *'\
    ' * * * * * * * * * * * * * * * * * * * * * * *'\
    ' * * * * * * * * * * * * * * * * * * * * * * *>' | wc
    0 128984 783012
    ; /bin/echo * * * * * * * * * * * * * * * * * * * * * * * \
    * * * * * * * * * * * * * * * * * * * * * * * \
    * * * * * * * * * * * * * * * * * * * * * * * \
    * * * * * * * * * * * * * * * * * * * * * * * \
    | wc
    bash: /bin/echo: Argument list too long

    The limit is with the operating system, not the shell, so why should perl's
    glob have any more difficulty than the shell's?

    Have you actually tried it out? My 10 second test above seems to disagree
    with your "it won't work" statement.

    --
    Sam Holden
    Sam Holden, Sep 9, 2003
    #7
  8. qanda <> wrote:
    >> Why not use perl's glob function (perldoc -f glob)?

    >
    > Because it won't work, as perldoc says, it returns the filename
    > expansions from the shell,



    Only with old perls.

    What perl version do you have?


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Sep 9, 2003
    #8
  9. qanda

    qanda Guest

    Thanks guys and girls, I wasn't aware it had been improved in the new
    versions; I'm limited (by customer) to using 5.005.

    As for the argument limit, yes the underlying issue is with the OS, as
    I understand it the limit is set by ARG_MAX in limits.h.

    Would anyone be able to answer the (original) question for me,
    certainly if using a newer version of Perl glob would be fine; however
    I can't do that for this particular problem and the only overhead I
    want for the user is to possibly quote the file glob so we avoid find
    and/or xargs, etc.

    Thanks again.
    qanda, Sep 10, 2003
    #9
  10. qanda

    qanda Guest

    Sorry for posting on top of my last reply, but I really do need help
    with this one. I will be forever (well quite some time at least) in
    your debt if you can help :)
    qanda, Sep 11, 2003
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Georgy Pruss
    Replies:
    15
    Views:
    716
    Tim Roberts
    Dec 1, 2003
  2. Tim Peters
    Replies:
    1
    Views:
    350
    Duncan Booth
    Dec 1, 2003
  3. Sean Berry

    Question about glob.glob <--newbie

    Sean Berry, May 4, 2004, in forum: Python
    Replies:
    3
    Views:
    342
    David M. Cooke
    May 4, 2004
  4. qanda
    Replies:
    1
    Views:
    107
    Tassilo v. Parseval
    Sep 15, 2003
  5. it_says_BALLS_on_your forehead

    opendir and readdir vs glob iteration

    it_says_BALLS_on_your forehead, Dec 1, 2005, in forum: Perl Misc
    Replies:
    9
    Views:
    441
    robic0
    Dec 3, 2005
Loading...

Share This Page