Upload meter for a CGI perl script

Discussion in 'Perl Misc' started by Sean Berry, Nov 7, 2004.

  1. Sean Berry

    Sean Berry Guest

    I was toying with the idea of making an upload progress meter for a perl
    script I have that usually takes somewhere in the neigborhood of 20M files.
    I figured I could have a popup that takes as an argument the four digit
    sequence number CGI.pm creates when uploading a file, then check the file
    size and report it. Then, this window could update periodically, say every
    2 seconds.

    But, how can I get access to the $sequence variable used in creating files
    in the tmp dir like /tmp/CGItemp3323 ????

    I don't want to go screwing around with CGI.pm, but see no other way of
    getting this number.

    Any help here is appreciated.
     
    Sean Berry, Nov 7, 2004
    #1
    1. Advertising

  2. Sean Berry

    Stuart Moore Guest

    Sean Berry wrote:
    > I was toying with the idea of making an upload progress meter for a perl
    > script I have that usually takes somewhere in the neigborhood of 20M files.
    > I figured I could have a popup that takes as an argument the four digit
    > sequence number CGI.pm creates when uploading a file, then check the file
    > size and report it. Then, this window could update periodically, say every
    > 2 seconds.
    >
    > But, how can I get access to the $sequence variable used in creating files
    > in the tmp dir like /tmp/CGItemp3323 ????
    >
    > I don't want to go screwing around with CGI.pm, but see no other way of
    > getting this number.
    >
    > Any help here is appreciated.
    >


    I am not an expert, but I was under the impression that the webserver
    (apache or whatever) waits until the transfer has been completed before
    then running the CGI script. A way to test this would be to be to record
    when the script was started.
     
    Stuart Moore, Nov 7, 2004
    #2
    1. Advertising

  3. > I am not an expert, but I was under the impression that the webserver
    > (apache or whatever) waits until the transfer has been completed before
    > then running the CGI script. A way to test this would be to be to record
    > when the script was started.


    You may be right. I do not know the ins and outs of it all. However, when
    I start an upload of a 10 meg file through a perl script using CGI.pm, I can
    see a file named CGItemp$sequence - where $sequence is a four digit number -
    being written to in /usr/tmp. As I do ls -lrt's I can see the file growing
    in size. I also have a little program that, given the four digit number,
    displays the file size every two seconds.

    I just need to be able to get this four digit number. I can see in CGI.pm
    where it is being created, but it uses a random during generation so it
    would not be possible to get the same number by doing a similar algorithm.
    What I need is the ability to get the number of $seqno CGI.pm.

    Help is greatly appreciated.
     
    news.west.cox.net, Nov 8, 2004
    #3
  4. Sean Berry

    Stuart Moore Guest

    news.west.cox.net wrote:

    >>I am not an expert, but I was under the impression that the webserver
    >>(apache or whatever) waits until the transfer has been completed before
    >>then running the CGI script. A way to test this would be to be to record
    >>when the script was started.

    >
    >
    > You may be right. I do not know the ins and outs of it all. However, when
    > I start an upload of a 10 meg file through a perl script using CGI.pm, I can
    > see a file named CGItemp$sequence - where $sequence is a four digit number -
    > being written to in /usr/tmp. As I do ls -lrt's I can see the file growing
    > in size. I also have a little program that, given the four digit number,
    > displays the file size every two seconds.
    >
    > I just need to be able to get this four digit number. I can see in CGI.pm
    > where it is being created, but it uses a random during generation so it
    > would not be possible to get the same number by doing a similar algorithm.
    > What I need is the ability to get the number of $seqno CGI.pm.
    >
    > Help is greatly appreciated.
    >


    I don't suppose you're able to guess it from the time of the file
    creation? Depends how often this is used, I guess...

    If not, I suggest you investigate whether the Perl script is actually
    running at the time of the upload. If it isn't, you'd probably be better
    off asking in an apache(/or other webserver) newsgroup.
     
    Stuart Moore, Nov 8, 2004
    #4
  5. "Stuart Moore" <> wrote in message
    news:cmmlth$mua$...
    > news.west.cox.net wrote:
    >
    >>>I am not an expert, but I was under the impression that the webserver
    >>>(apache or whatever) waits until the transfer has been completed before
    >>>then running the CGI script. A way to test this would be to be to record
    >>>when the script was started.

    >>
    >>
    >> You may be right. I do not know the ins and outs of it all. However,
    >> when I start an upload of a 10 meg file through a perl script using
    >> CGI.pm, I can see a file named CGItemp$sequence - where $sequence is a
    >> four digit number - being written to in /usr/tmp. As I do ls -lrt's I
    >> can see the file growing in size. I also have a little program that,
    >> given the four digit number, displays the file size every two seconds.
    >>
    >> I just need to be able to get this four digit number. I can see in
    >> CGI.pm where it is being created, but it uses a random during generation
    >> so it would not be possible to get the same number by doing a similar
    >> algorithm. What I need is the ability to get the number of $seqno CGI.pm.
    >>
    >> Help is greatly appreciated.
    >>

    >
    > I don't suppose you're able to guess it from the time of the file
    > creation? Depends how often this is used, I guess...
    >
    > If not, I suggest you investigate whether the Perl script is actually
    > running at the time of the upload. If it isn't, you'd probably be better
    > off asking in an apache(/or other webserver) newsgroup.


    I am fairly certain it is running during the upload. In CGI.pm I can see
    ....

    # choose a relatively unpredictable tmpfile sequence number
    my $seqno = unpack("%16C*",join('',localtime,values %ENV));
    for (my $cnt=10;$cnt>0;$cnt--) {
    next unless $tmpfile = new CGITempFile($seqno);
    $tmp = $tmpfile->as_string;
    last if defined($filehandle =
    Fh->new($filename,$tmp,$PRIVATE_TEMPFILES));
    $seqno += int rand(100);
    }

    Then, later on I see...

    last if ! -f ($filename =
    sprintf("${TMPDIRECTORY}${SL}CGItemp%d",$sequence++));

    ".../CGItemo%d", $sequence++

    Then, in my /usr/tmp dir I see files with names like CGItemp1234,
    CGItemp5678, etc.
    So, based on that, it seems like CGI.pm is running during the upload.

    I just need to get the $sequence var.
     
    news.west.cox.net, Nov 8, 2004
    #5
  6. Sean Berry

    David Filmer Guest

    I don't think it's possible to do a "meter" (ie, a graphical slider) because
    CGI.pm does not know the expected size of the file in advance. It is
    possible, however, to do a progress indicator where the client sees the
    total number of bytes (or KB or MB or whatever you like) received so far
    (and, at least this gives some sense of progress during a very large upload,
    and this seems more specifically what the OP was looking for). It requires a
    little bit of JavaScript as well, though. You could do this in the main
    browser window or a pop-up window (but pop-ups can be problematic with
    modern browsers that might be set to disallow them).

    Here are some code fragments from an ancient CGI that I wrote years ago. The
    use of subroutine prototypes isn't really correct, and it uses
    "intermediate" variables that I've since come to avoid, and there are some
    other questionable programming practices, but hey, it works....

    This is assuming the CGI inherits a multipart form that includes a parameter
    such as this (for uploading up to eight files at once; this occurs within a
    table (gads, these days I would use CSS)):

    for (1..8) {
    print Tr({-align => 'center'}, td({-colspan => '3'}, filefield(-name =>
    'file', -size => '60')))
    }

    Now you can handle that upload such as this:

    use Number::Format qw/number_format/; #or use sprintf, but I'm too lazy to
    remember the syntax

    #first, we need a named textfield that JavaScript will use to provide
    message updates...
    #... this is a form, BUT one that will be "filled out" by the program, not
    by the client...

    print
    start_form ({name=>'status'}),
    center(textfield ({name=>'info', -size => 80})), br(),
    end_form();

    #now upload the file - standard CGI.pm stuff, except we add a call to a
    subroutine to update the message field...
    foreach $file(param('file')) { #allows for more than one file uploaded at
    a time
    $file =~ s/^\s+//; $file =~ s/\s+$//;
    next unless $file;
    $_ = $file;
    s!\\!/!g; #Convert DOS \ to UNIX /
    s!/+$!!; #Strip off trailing / if any
    s/\.zip$/.zip/i; #make sure it's lowercase .zip (if applicable), for
    various reasons
    s!^.*/!!; #Strip off path, if any
    s/\.z$/\.Z/; #change .z extension (if any) to uppercase, for various
    reasons

    $localfile = &UniqID.".$_"; #supply your own method for uniqueness -
    maybe an IO::All call?
    $localfile =~ s/[^\w\.\-]/_/g; #Change funny characters to underscore

    open (OUTFILE, ">$localfile") || die "Unable to open for write:
    $localfile\n";
    $count = 0;

    while ($bytesread=read($file, $buffer, 1024)) { #1K increments
    $count += $bytesread; $count2 = format_number (($count / 2**20), 2,
    2); #megabytes
    print OUTFILE $buffer;
    &Progress("Receiving $file ($count2 MegaBytes received so far)");
    #this is where the magic happens!
    }
    $size = (stat($localfile))[7];
    &Progress("File $file Received - $size bytes");
    }

    sub Progress() {
    #This simply updates the named form-field (using JavaScript) with
    whatever message we wish...
    my $message = shift;
    print "<script language='JavaScript'>",
    "document.status.info.value=' $message';",
    "</script>";
    }


    Cheers!

    "Sean Berry" <> wrote in message
    news:bwxjd.120416$hj.27962@fed1read07...
    >I was toying with the idea of making an upload progress meter for a perl
    >script I have that usually takes somewhere in the neigborhood of 20M files.
    >I figured I could have a popup that takes as an argument the four digit
    >sequence number CGI.pm creates when uploading a file, then check the file
    >size and report it. Then, this window could update periodically, say every
    >2 seconds.
    >
    > But, how can I get access to the $sequence variable used in creating files
    > in the tmp dir like /tmp/CGItemp3323 ????
    >
    > I don't want to go screwing around with CGI.pm, but see no other way of
    > getting this number.
    >
    > Any help here is appreciated.
    >
    >
    >
     
    David Filmer, Nov 8, 2004
    #6
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Brian Birtle
    Replies:
    3
    Views:
    2,437
  2. praba kar

    Python-cgi or Perl-cgi script doubt

    praba kar, Jul 30, 2005, in forum: Python
    Replies:
    1
    Views:
    685
    Michael Sparks
    Jul 30, 2005
  3. Avin Patel
    Replies:
    4
    Views:
    959
    Brian McCauley
    Nov 5, 2004
  4. Replies:
    20
    Views:
    750
    Gunnar Hjalmarsson
    Jan 18, 2005
  5. kath
    Replies:
    4
    Views:
    670
    J. Gleixner
    Apr 9, 2007
Loading...

Share This Page