search.pl

Discussion in 'Perl Misc' started by Robin, Apr 30, 2004.

  1. Robin

    Robin Guest

    I have some questions on this , why isn't sub parse and sub search not
    performing the way I want them too, they are supposed to go through all of
    the directories and then open the files in them and then if the contents of
    the file match param ('query') print the results. It's been driving me
    crazy, if someone could point me in the right direction it would be great.
    This is very incomplete and doesn't use all of it's data yet, so I would
    appreciate comments that don't point that out, for example, what if it opens
    binary files? I dunno..thanks in advance, by the way, I know that it's
    somewhere in the methods search or parse, but other than that I don't have
    the foggiest.

    -Robin


    #!/usr/bin/perl

    use strict;
    use warnings;

    use Fcntl qw :)flock);

    use CGI qw:)all);

    $CGI::pOST_MAX=1024 * 100; # max 100K posts
    $CGI::DISABLE_UPLOADS = 1; # no uploads

    $" = '';

    $ENV{'PATH'} = '/bin:/usr/bin:/usr/local/bin';

    my @directories = ("./", "../", "../design"); #change this to the
    directories you want to have searched.
    my $action = url_param ('action');
    my $rootfile = url (relative=>1);
    my $headerfile = "searchheader.txt";
    my $footerfile = "searchfooter.txt";
    my $errorfile = "ERR.txt";
    my @head = getheader ($headerfile);
    my @foot = getfooter ($footerfile);
    my $date = getdate ();
    my @errors;

    if ($action eq "search")
    {
    search ();
    }

    else
    {
    newsearch ();
    }

    sub search
    {
    print header;
    print (@head);
    print <<END;
    <p><strong><em>Infused Search</em></strong>
    <br>
    <br>
    Search Results:
    </p>
    END
    #code for parsing results
    foreach my $dir (@directories)
    {
    opendir (DIR, $dir);
    my @files_from_dir = readdir (DIR);
    closedir (DIR);
    foreach my $file (@files_from_dir)
    {
    if (! -d $file)
    {
    parse ($file, $dir);
    }
    }
    }
    print <<END;
    <hr size="1">
    </body>
    </html>
    END
    }

    sub parse
    {
    my ($filetoparse, $dirtoparse) = @_;
    my @results;
    open (FILE, $filetoparse) or push (@errors, "A file open operation
    failed.");
    flock (FILE, LOCK_SH) or push (@errors, "A file lock operation
    failed.");
    my @filecontents = <FILE>;
    close (FILE);
    chomp (@filecontents);
    my $result;
    my $filecontents = join ('', @filecontents);
    if (param ('query') =~ /$filecontents/m and param ('query'))
    {
    $result = $filetoparse;
    print "<a href=\"$dirtoparse" . "$filetoparse\">$result</a><br>";
    }
    }

    sub newsearch
    {
    print header;
    print (@head);
    print <<END;
    <strong><em>Infused Search</em></strong>
    <br>
    <br>
    <hr size="1">
    <form name="form1" method="post" action="search.pl?action=search">
    <input type="text" name="query">
    <input type="submit" name="Submit" value="Submit">
    </form>
    <hr size="1">
    END
    print (@foot);
    }

    sub checkerrors
    {
    if (@errors)
    {
    print header;
    print "<html><body><center>";
    print "There were errors while trying to execute Infused Search. They
    are listed as follows.<br><br>\n";
    foreach my $error (@errors)
    {
    print ($error, "<br>\n");
    }
    my $errflag = 0;
    if (! open (ERRORF, ">>$errorfile") and flock (ERRORF, LOCK_EX))
    {
    print "There was an error logging the errors: file cannot be locked or
    opened.<br>";
    $errflag = 1;
    }
    else
    {
    print ERRORF ("Current date: $date", "\n");
    foreach my $error2 (@errors)
    {
    print ERRORF $error2, "\n";
    }
    }
    close (ERRORF);
    if (! $errflag)
    {
    print "<br>", "Errors have been logged in $errorfile.";
    }
    print "</body></html>";
    exit (0);
    }
    else
    {
    return;
    }
    }

    sub getheader
    {
    my $header_sub = shift;
    my (@headertoret);
    if (-e $header_sub)
    {
    open (HEADERF, $header_sub);
    flock (HEADERF, LOCK_SH);
    @headertoret = <HEADERF>;
    close (HEADERF);
    }
    else
    {
    open (HEADERF, ">$header_sub");
    flock (HEADERF, LOCK_EX);
    @headertoret = <<END;
    <!DOCTYPE HTML PUBLIC "-//W3C//DTD HTML 4.01 Transitional//EN"
    "http://www.w3.org/TR/html4/loose.dtd">
    <html>
    <head>
    <title>Untitled Document</title>
    <meta http-equiv="Content-Type" content="text/html; charset=iso-8859-1">
    <style type="text/css">
    <!--
    body,td,th {
    color: #FFA4A4;
    }
    body {
    background-color: #000000;
    }
    a:link {
    color: #66FFFF;
    }
    a:visited {
    color: #66FFFF;
    }
    a:hover {
    color: #FF9933;
    }
    a:active {
    color: #66FFFF;
    }
    -->
    </style></head>

    <body>
    END
    print HEADERF @headertoret;
    close (HEADERF);
    }
    return (@headertoret);
    }

    sub getfooter
    {
    my $footer_sub = shift;
    my (@footertoret);
    if (-e $footer_sub)
    {
    open (HEADERF, $footer_sub);
    flock (HEADERF, LOCK_SH);
    @footertoret = <HEADERF>;
    close (HEADERF);
    }
    else
    {
    open (HEADERF, ">$footer_sub");
    flock (HEADERF, LOCK_EX);
    @footertoret = <<END;
    </body></html>
    END
    print HEADERF @footertoret;
    close (HEADERF);
    }
    return (@footertoret);
    }

    sub getdate
    {
    my ($day, $mon, $year)=(localtime)[3,4,5];
    $mon++;
    $year +=1900;
    my $date = $mon . "/" . $day . "/" . $year;
    return $date;
    }
     
    Robin, Apr 30, 2004
    #1
    1. Advertising

  2. Robin wrote:

    > I have some questions on this , why isn't sub parse and sub search not
    > performing the way I want them too, they are supposed to go through all of
    > the directories and then open the files in them and then if the contents
    > of the file match param ('query') print the results. It's been driving me
    > crazy, if someone could point me in the right direction it would be great.


    Your script passes the directory to parse(), but it doesn't look like
    parse() is using it. So any file that isn't in the current directory ('./')
    won't get parsed.

    > open (FILE, $filetoparse) or push (@errors, "A file open operation
    > failed.");


    You should be using more helpful error messages. The above tells you nothing
    more than that a file failed to open - it would be better if it also told
    you what file and why. Including the file name and $! would make this bug
    easier to find.

    sherm--

    --
    Cocoa programming in Perl: http://camelbones.sourceforge.net
    Hire me! My resume: http://www.dot-app.org
     
    Sherm Pendley, Apr 30, 2004
    #2
    1. Advertising

  3. Robin

    GreenLight Guest

    "Robin" <webmaster @ infusedlight . net> wrote in message news:<c6st0v$3de$>...
    > #code for parsing results
    > foreach my $dir (@directories)
    > {
    > opendir (DIR, $dir);
    > my @files_from_dir = readdir (DIR);
    > closedir (DIR);
    > foreach my $file (@files_from_dir)
    > {
    > if (! -d $file)
    > {
    > parse ($file, $dir);
    > }
    > }
    > }


    Do you really indent your code like that, or is this just an artifact of
    whatever utility that you use to post to clpm? That is so painful to look at
    that I cannot even consider analyzing it to help you with your problems...

    --
    my real address is perl - at - milbaugh - dot - com
     
    GreenLight, Apr 30, 2004
    #3
  4. GreenLight wrote:

    > "Robin" <webmaster @ infusedlight . net> wrote in message news:<c6st0v$3de$>...

    <snip>
    > Do you really indent your code like that, or is this just an artifact of
    > whatever utility that you use to post to clpm? That is so painful to look at
    > that I cannot even consider analyzing it to help you with your problems...


    Robin: you could do with looking at templating solutions, eg
    HTML::Template or Template-Toolkit (though there are a fair number out
    there and people tend to have strong opinions about choice of templating
    package). Embedding HTML ie presentation and content directly within a
    script makes it hard to read and hard to maintain. It also makes it very
    difficult to work with web designers who may not have any knowledge of
    Perl.

    As far as the indenting goes, check out perltidy.

    Mark
     
    Mark Clements, Apr 30, 2004
    #4
  5. Robin <webmaster @ infusedlight . net> wrote:

    > I have some questions on this , why isn't sub parse and sub search
    > not performing the way I want them too, they are supposed to go
    > through all of the directories and then open the files in them and
    > then if the contents of the file match param ('query') print the
    > results. It's been driving me crazy, if someone could point me in
    > the right direction it would be great.


    use File::Find;

    Writing your own code to recurse through directories is a useful
    exercise, but if you just want to get the program written and use it,
    File::Find is faster and easier to use, and almost certainly more
    robust.

    I'd also suggest not printing the results immediately, but save them
    somewhere (possibly in an array of some sort, maybe of hashes) and
    only print them out when the search has finished.

    And please, *please*, format your code in a readable way. perlstyle
    has suggestions. You don't have to follow them exactly, but at least
    pick a style and use it *consistently*. If I didn't have perltidy
    available to reformat your code I would not have even bothered to
    skim over it. If you don't have an editor that assists with
    indenting, then get one.


    > sub parse
    > {
    > my ($filetoparse, $dirtoparse) = @_;
    > my @results;
    > open (FILE, $filetoparse) or push (@errors, "A file open
    > operation failed.");


    So if the open fails, save a non-informative message in an array
    (which file? why did it fail?) ...

    > flock (FILE, LOCK_SH) or push (@errors, "A file lock operation
    > failed.");
    > my @filecontents = <FILE>;


    ....and then try to read it anyway.

    > close (FILE);
    > chomp (@filecontents);


    ....and then do other stuff with an empty array.


    Hmmm...

    [I got tired at this point and stopped reading]
     
    David K. Wall, Apr 30, 2004
    #5
  6. On Thu, 29 Apr 2004 22:50:38 -0800, "Robin" <webmaster @ infusedlight
    .. net> wrote:

    >I have some questions on this , why isn't sub parse and sub search not
    >performing the way I want them too, they are supposed to go through all of
    >the directories and then open the files in them and then if the contents of
    >the file match param ('query') print the results. It's been driving me


    I have not read your code, but I seemed to notice some readdir()s so
    from what is described above I feel like pointing out that the
    standard answer -and indeed generally a good one!- to this kind of
    questions is 'use File::Find'. Maybe you knew, maybe you don't: if you
    didn't, it's there for your ease!


    Michele
    --
    you'll see that it shouldn't be so. AND, the writting as usuall is
    fantastic incompetent. To illustrate, i quote:
    - Xah Lee trolling on clpmisc,
    "perl bug File::Basename and Perl's nature"
     
    Michele Dondi, Apr 30, 2004
    #6
  7. Robin

    Joe Smith Guest

    Robin wrote:

    > opendir (DIR, $dir);
    > my @files_from_dir = readdir (DIR);
    > closedir (DIR);
    > foreach my $file (@files_from_dir)
    > {
    > if (! -d $file)
    > {
    > parse ($file, $dir);
    > }
    > }
    > }


    1) You really messed up on the indentation there.
    2) Your main logic error is in the argument to -d() and parse(); it's
    incomplete. Anytime $dir is not the same as ".", your code will fail.

    foreach my $fil (@files_from_dir) {
    my $file = "$dir/$fil";
    parse($file,$dir) if -f $file;
    }

    -Joe
     
    Joe Smith, Apr 30, 2004
    #7
  8. Also sprach Michele Dondi:

    > On Thu, 29 Apr 2004 22:50:38 -0800, "Robin" <webmaster @ infusedlight
    > . net> wrote:
    >
    >>I have some questions on this , why isn't sub parse and sub search not
    >>performing the way I want them too, they are supposed to go through all of
    >>the directories and then open the files in them and then if the contents of
    >>the file match param ('query') print the results. It's been driving me

    >
    > I have not read your code, but I seemed to notice some readdir()s so
    > from what is described above I feel like pointing out that the
    > standard answer -and indeed generally a good one!- to this kind of
    > questions is 'use File::Find'. Maybe you knew, maybe you don't: if you
    > didn't, it's there for your ease!


    As far as I could see, he is not recursing further into subdirectories.
    For ordinary directory listings, I'd use readdir() alright.

    Tassilo
    --
    $_=q#",}])!JAPH!qq(tsuJ[{@"tnirp}3..0}_$;//::niam/s~=)]3[))_$-3(rellac(=_$({
    pam{rekcahbus})(rekcah{lrePbus})(lreP{rehtonabus})!JAPH!qq(rehtona{tsuJbus#;
    $_=reverse,s+(?<=sub).+q#q!'"qq.\t$&."'!#+sexisexiixesixeseg;y~\n~~dddd;eval
     
    Tassilo v. Parseval, Apr 30, 2004
    #8
  9. Tassilo v. Parseval <> wrote:

    > Also sprach Michele Dondi:
    >
    >> On Thu, 29 Apr 2004 22:50:38 -0800, "Robin" <webmaster @
    >> infusedlight . net> wrote:
    >>
    >>>I have some questions on this , why isn't sub parse and sub
    >>>search not performing the way I want them too, they are supposed
    >>>to go through all of the directories and then open the files in
    >>>them and then if the contents of the file match param ('query')
    >>>print the results. It's been driving me

    >>
    >> I have not read your code, but I seemed to notice some readdir()s
    >> so from what is described above I feel like pointing out that the
    >> standard answer -and indeed generally a good one!- to this kind
    >> of questions is 'use File::Find'. Maybe you knew, maybe you
    >> don't: if you didn't, it's there for your ease!

    >
    > As far as I could see, he is not recursing further into
    > subdirectories. For ordinary directory listings, I'd use readdir()
    > alright.


    Yeah, you're right. I didn't look closely enough either.
     
    David K. Wall, Apr 30, 2004
    #9
  10. Robin

    Robin Guest

    "Sherm Pendley" <> wrote in message
    news:...
    > Robin wrote:
    >
    > > I have some questions on this , why isn't sub parse and sub search not
    > > performing the way I want them too, they are supposed to go through all

    of
    > > the directories and then open the files in them and then if the contents
    > > of the file match param ('query') print the results. It's been driving

    me
    > > crazy, if someone could point me in the right direction it would be

    great.
    >
    > Your script passes the directory to parse(), but it doesn't look like
    > parse() is using it. So any file that isn't in the current directory

    ('./')
    > won't get parsed.


    parse uses the directory when the link is printed out.

    > > open (FILE, $filetoparse) or push (@errors, "A file open operation
    > > failed.");

    >
    > You should be using more helpful error messages. The above tells you

    nothing
    > more than that a file failed to open - it would be better if it also told
    > you what file and why. Including the file name and $! would make this bug
    > easier to find.


    yes, this is my first draft and I haven't really worked in the error
    checking routine, but I will....

    -Robin
     
    Robin, Apr 30, 2004
    #10
  11. Robin

    Robin Guest

    "GreenLight" <> wrote in message
    news:...
    > "Robin" <webmaster @ infusedlight . net> wrote in message

    news:<c6st0v$3de$>...
    > > #code for parsing results
    > > foreach my $dir (@directories)
    > > {
    > > opendir (DIR, $dir);
    > > my @files_from_dir = readdir (DIR);
    > > closedir (DIR);
    > > foreach my $file (@files_from_dir)
    > > {
    > > if (! -d $file)
    > > {
    > > parse ($file, $dir);
    > > }
    > > }
    > > }

    >
    > Do you really indent your code like that, or is this just an artifact of
    > whatever utility that you use to post to clpm? That is so painful to look

    at
    > that I cannot even consider analyzing it to help you with your problems...


    The code has screwed up indendation because of my newsreader which does some
    weird stuff with tabs and spaces, but it looks fine when I edit it, and also
    when I open the file with a text editor, it's just when I paste it into
    something, sorry for posting code that's hard to read.

    > my real address is perl - at - milbaugh - dot - com
     
    Robin, Apr 30, 2004
    #11
  12. Robin

    Robin Guest

    "Mark Clements" <> wrote in message
    news:40924e1c$...
    > GreenLight wrote:
    >
    > > "Robin" <webmaster @ infusedlight . net> wrote in message

    news:<c6st0v$3de$>...
    > <snip>
    > > Do you really indent your code like that, or is this just an artifact of
    > > whatever utility that you use to post to clpm? That is so painful to

    look at
    > > that I cannot even consider analyzing it to help you with your

    problems...
    >
    > Robin: you could do with looking at templating solutions, eg
    > HTML::Template or Template-Toolkit (though there are a fair number out
    > there and people tend to have strong opinions about choice of templating
    > package). Embedding HTML ie presentation and content directly within a
    > script makes it hard to read and hard to maintain. It also makes it very
    > difficult to work with web designers who may not have any knowledge of
    > Perl.


    I'll keep it in mind, I've never used either one, I tend to work with
    minimal modules because it's sometimes a compatibility issue with some
    servers using scripts that don't have the modules that I'm using...but
    thanks.

    > As far as the indenting goes, check out perltidy.


    I will check it out. See above.

    > Mark
     
    Robin, Apr 30, 2004
    #12
  13. Robin

    Robin Guest

    "David K. Wall" <> wrote in message
    news:Xns94DB6B155A0B2dkwwashere@216.168.3.30...
    > Robin <webmaster @ infusedlight . net> wrote:
    >
    > > I have some questions on this , why isn't sub parse and sub search
    > > not performing the way I want them too, they are supposed to go
    > > through all of the directories and then open the files in them and
    > > then if the contents of the file match param ('query') print the
    > > results. It's been driving me crazy, if someone could point me in
    > > the right direction it would be great.

    >
    > use File::Find;
    >
    > Writing your own code to recurse through directories is a useful
    > exercise, but if you just want to get the program written and use it,
    > File::Find is faster and easier to use, and almost certainly more
    > robust.
    >
    > I'd also suggest not printing the results immediately, but save them
    > somewhere (possibly in an array of some sort, maybe of hashes) and
    > only print them out when the search has finished.


    yeah, I was thinking that exact same thing. I'll check out file::find.

    >
    > And please, *please*, format your code in a readable way. perlstyle
    > has suggestions. You don't have to follow them exactly, but at least
    > pick a style and use it *consistently*. If I didn't have perltidy
    > available to reformat your code I would not have even bothered to
    > skim over it. If you don't have an editor that assists with
    > indenting, then get one.


    yeah, next time I post code I will make sure it's formatted good.

    -Robin
     
    Robin, Apr 30, 2004
    #13
  14. Robin

    Robin Guest

    "Michele Dondi" <> wrote in message
    news:...
    > On Thu, 29 Apr 2004 22:50:38 -0800, "Robin" <webmaster @ infusedlight
    > . net> wrote:
    >
    > >I have some questions on this , why isn't sub parse and sub search not
    > >performing the way I want them too, they are supposed to go through all

    of
    > >the directories and then open the files in them and then if the contents

    of
    > >the file match param ('query') print the results. It's been driving me

    >
    > I have not read your code, but I seemed to notice some readdir()s so
    > from what is described above I feel like pointing out that the
    > standard answer -and indeed generally a good one!- to this kind of
    > questions is 'use File::Find'. Maybe you knew, maybe you don't: if you
    > didn't, it's there for your ease!



    I have to check out the docs for that! Thanks.
    -Robin
     
    Robin, Apr 30, 2004
    #14
  15. Robin

    Robin Guest

    "Joe Smith" <> wrote in message
    news:_aukc.3350$0H1.362719@attbi_s54...
    > Robin wrote:
    >
    > > opendir (DIR, $dir);
    > > my @files_from_dir = readdir (DIR);
    > > closedir (DIR);
    > > foreach my $file (@files_from_dir)
    > > {
    > > if (! -d $file)
    > > {
    > > parse ($file, $dir);
    > > }
    > > }
    > > }

    >
    > 1) You really messed up on the indentation there.
    > 2) Your main logic error is in the argument to -d() and parse(); it's
    > incomplete. Anytime $dir is not the same as ".", your code will fail.
    >
    > foreach my $fil (@files_from_dir) {
    > my $file = "$dir/$fil";
    > parse($file,$dir) if -f $file;
    > }
    >


    Thanks! I see what you mean.
    -Robin
     
    Robin, Apr 30, 2004
    #15
  16. Robin wrote:
    > Sherm Pendley wrote:
    >> You should be using more helpful error messages. The above tells
    >> you nothing more than that a file failed to open - it would be
    >> better if it also told you what file and why. Including the file
    >> name and $! would make this bug easier to find.

    >
    > yes, this is my first draft and I haven't really worked in the
    > error checking routine, but I will....


    It's rude IMO to post a "first draft" of script and ask for criticism,
    since it unnecessarily takes up people's time.

    Do your *best* first, and (possibly) ask for criticism only after that.

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
     
    Gunnar Hjalmarsson, Apr 30, 2004
    #16
  17. Robin wrote:

    > "Sherm Pendley" <> wrote in message
    > news:...
    >>
    >> Your script passes the directory to parse(), but it doesn't look like
    >> parse() is using it. So any file that isn't in the current directory

    > ('./')
    >> won't get parsed.

    >
    > parse uses the directory when the link is printed out.


    .... but not when the file is opened. That's the problem.

    >> you what file and why. Including the file name and $! would make this bug
    >> easier to find.

    >
    > yes, this is my first draft and I haven't really worked in the error
    > checking routine, but I will....


    The error checking isn't something to pretty up later in the process - it's
    there to help you find your bugs. This bug would have been obvious to you,
    if the error message had included the full file name and $!.

    sherm--

    --
    Cocoa programming in Perl: http://camelbones.sourceforge.net
    Hire me! My resume: http://www.dot-app.org
     
    Sherm Pendley, Apr 30, 2004
    #17
  18. Robin

    Robin Guest

    "Gunnar Hjalmarsson" <> wrote in message
    news:c6u86d$g0gqj$-berlin.de...
    > Robin wrote:
    > > Sherm Pendley wrote:
    > >> You should be using more helpful error messages. The above tells
    > >> you nothing more than that a file failed to open - it would be
    > >> better if it also told you what file and why. Including the file
    > >> name and $! would make this bug easier to find.

    > >
    > > yes, this is my first draft and I haven't really worked in the
    > > error checking routine, but I will....

    >
    > It's rude IMO to post a "first draft" of script and ask for criticism,
    > since it unnecessarily takes up people's time.


    ok, point taken.
     
    Robin, Apr 30, 2004
    #18
  19. Robin wrote:
    > Gunnar Hjalmarsson wrote:
    >> It's rude IMO to post a "first draft" of script and ask for
    >> criticism, since it unnecessarily takes up people's time.

    >
    > ok, point taken.


    Really? And 17 minutes later you post a new script with shortcomings
    that you say you are well aware of...

    Stop it!!

    --
    Gunnar Hjalmarsson
    Email: http://www.gunnar.cc/cgi-bin/contact.pl
     
    Gunnar Hjalmarsson, Apr 30, 2004
    #19
  20. Robin wrote:


    > I'll keep it in mind, I've never used either one, I tend to work with
    > minimal modules because it's sometimes a compatibility issue with some
    > servers using scripts that don't have the modules that I'm using...but
    > thanks.

    you could always install them... You don't need to have root access to
    install and use CPAN modules: they can be installed into eg ~/lib and
    used from there.

    >>As far as the indenting goes, check out perltidy.

    > I will check it out. See above.

    ditto.

    Mark
     
    Mark Clements, May 1, 2004
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. =?Utf-8?B?TGFrc2htaSBOYXJheWFuYW4uUg==?=

    Google search result like site search!! How?

    =?Utf-8?B?TGFrc2htaSBOYXJheWFuYW4uUg==?=, May 5, 2005, in forum: ASP .Net
    Replies:
    3
    Views:
    699
    Lucas Tam
    May 6, 2005
  2. Andy
    Replies:
    1
    Views:
    367
    Jack Klein
    Nov 25, 2003
  3. Anand Pillai

    String search vs regexp search

    Anand Pillai, Oct 12, 2003, in forum: Python
    Replies:
    10
    Views:
    607
    Anand Pillai
    Oct 15, 2003
  4. mason66
    Replies:
    0
    Views:
    430
    mason66
    Jul 27, 2006
  5. Abby Lee
    Replies:
    5
    Views:
    443
    Abby Lee
    Aug 2, 2004
Loading...

Share This Page