Concatenating selective lines from a bunch of files into a singlefile

Discussion in 'Perl Misc' started by Rider, Jul 25, 2009.

  1. Rider

    Rider Guest

    Hi experts,

    I have a unix dir, test and it has 100 files. I need to extract all
    the lines that contain a word "keyword" key word" (case insensitive)
    from each of those files and put all those lines into another file,
    concatenate.txt. Similarly, I want all the lines that don't contain
    that key word (keyword or key word) into another file,
    non_concatenate.txt.

    I know a little bit of perl and I am wondering how can I do it here.

    Thanks,
    J
    Rider, Jul 25, 2009
    #1
    1. Advertising

  2. Rider

    Rider Guest

    On Jul 25, 9:10 am, Rider <> wrote:
    > Hi experts,
    >
    > I have a unix dir, test and it has 100 files. I need to extract all
    > the lines that contain a word "keyword" key word" (case insensitive)
    > from each of those files and put all those lines into another file,
    > concatenate.txt. Similarly, I want all the lines that don't contain
    > that key word (keyword or key word) into another file,
    > non_concatenate.txt.
    >
    > I know a little bit of perl and I am wondering how can I do it here.
    >
    > Thanks,
    > J


    What is the easiest way? To use File::Find module and then use grep,
    join commands?

    Thanks in advance,
    J
    Rider, Jul 25, 2009
    #2
    1. Advertising

  3. Rider

    Jim Gibson Guest

    Re: Concatenating selective lines from a bunch of files into a single file

    In article
    <>,
    Rider <> wrote:

    > On Jul 25, 9:10 am, Rider <> wrote:
    > > Hi experts,
    > >
    > > I have a unix dir, test and it has 100 files. I need to extract all
    > > the lines that contain a word "keyword" key word" (case insensitive)
    > > from each of those files and put all those lines into another file,
    > > concatenate.txt. Similarly, I want all the lines that don't contain
    > > that key word (keyword or key word) into another file,
    > > non_concatenate.txt.
    > >
    > > I know a little bit of perl and I am wondering how can I do it here.
    > >
    > > Thanks,
    > > J

    >
    > What is the easiest way? To use File::Find module and then use grep,
    > join commands?


    The easiest way would be call your Perl program with all of the file
    names on the command-line:

    program.pl /test/*

    Then use the construct

    while(<>) {
    ...
    }

    to read all of the lines in all of the files. Open two files before you
    enter the while loop:

    open(my $yes, '>', 'concatenate.txt') or die(...);
    open(my $no, '>', 'non_concatenate.txt') or die(...);

    Then in the body of the while loop, use the match operator with a
    regular expression (case insensitive):

    if( /keyword/i ) {
    print $yes;
    }else{
    print $no;
    }

    If your shell complains about the length of the command-line, you can
    use opendir and readdir to fetch the file names in your directory, then
    open each file for reading individually.

    --
    Jim Gibson
    Jim Gibson, Jul 25, 2009
    #3
  4. Rider

    Guest

    Re: Concatenating selective lines from a bunch of files into a single file

    On Sat, 25 Jul 2009 09:10:08 -0700 (PDT), Rider <> wrote:

    >Hi experts,
    >
    >I have a unix dir, test and it has 100 files. I need to extract all
    >the lines that contain a word "keyword" key word" (case insensitive)
    >from each of those files and put all those lines into another file,
    >concatenate.txt. Similarly, I want all the lines that don't contain
    >that key word (keyword or key word) into another file,
    >non_concatenate.txt.
    >
    >I know a little bit of perl and I am wondering how can I do it here.
    >
    >Thanks,
    >J


    I am curious as to what end this de-interlace achieves?
    There seems to be no pragmatic function to it.
    Its fine if its just an exercise. Usually though, a real world
    effort will reap more rewards to you personally.

    Good luck!
    -sln
    , Jul 25, 2009
    #4
  5. Re: Concatenating selective lines from a bunch of files into a single file

    Rider <> wrote:
    >I have a unix dir, test and it has 100 files.


    opendir() and readdir() or glob().

    >I need to extract all
    >the lines that contain a word "keyword" key word" (case insensitive)
    >from each of those files


    loop through all those files who's names you gathered in step 1,
    open() each, read it line by line and depening upon if the line matches
    (m//), then

    >and put all those lines into another file,
    >concatenate.txt.


    print() that line to the target file, which you open()ed earlier

    > Similarly, I want all the lines that don't contain
    >that key word (keyword or key word) into another file,
    >non_concatenate.txt.


    or the other target file, which you also open()ed earlier.

    jue
    Jürgen Exner, Jul 26, 2009
    #5
  6. Re: Concatenating selective lines from a bunch of files into asinglefile

    On Sat, 25 Jul 2009 09:10:08 -0700, Rider wrote:

    > Hi experts,
    >
    > I have a unix dir, test and it has 100 files. I need to extract all the
    > lines that contain a word "keyword" key word" (case insensitive) from
    > each of those files and put all those lines into another file,
    > concatenate.txt. Similarly, I want all the lines that don't contain that
    > key word (keyword or key word) into another file, non_concatenate.txt.
    >
    > I know a little bit of perl and I am wondering how can I do it here.


    I would not do this in perl, use the right tool for the job!

    #!/bin/sh
    grep -i "keyword" /path/to/test/* >/path/concatenate.txt
    grep -iv "keyword" /path/to/test/* >/path/non_concatenate.txt

    HTH,
    M4
    Martijn Lievaart, Jul 26, 2009
    #6
  7. Re: Concatenating selective lines from a bunch of files into a single file

    >>>>> "R" == Rider <> writes:

    R> Hi experts, I have a unix dir, test and it has 100 files. I need
    R> to extract all the lines that contain a word "keyword" key word"
    R> (case insensitive) from each of those files and put all those
    R> lines into another file, concatenate.txt. Similarly, I want all
    R> the lines that don't contain that key word (keyword or key word)
    R> into another file, non_concatenate.txt.

    R> I know a little bit of perl and I am wondering how can I do it
    R> here.

    The simplest thing would be to forgo Perl altogether and use the Unix
    grep utility.

    Charlton



    --
    Charlton Wilbur
    Charlton Wilbur, Jul 26, 2009
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Gary Morrison
    Replies:
    5
    Views:
    2,557
    Gary Morrison
    Mar 28, 2006
  2. Replies:
    2
    Views:
    311
  3. sasuke
    Replies:
    5
    Views:
    1,974
    Tom Anderson
    Jul 6, 2008
  4. Gilles Ganault
    Replies:
    2
    Views:
    252
    Gilles Ganault
    Apr 8, 2011
  5. Rapier
    Replies:
    11
    Views:
    168
    Anno Siegel
    Oct 14, 2004
Loading...

Share This Page