Problems flushing my buffer! (perl)

N

Nigel

Hi there,

I hope you can help - I'm writing a perl program and I want it report
back to the browser
on its progress. I read the perl faq and saw that by setting $| true
the buffer would be
flushed each time I printed instead of waiting until the program
terminates. But I can't
get it to work.

I'm expecting the web page to say that it's on the job, then every
second (for 10 seconds) to say Please Wait and then at the end to say
it's finished. But what happens is I wait ten seconds for a response
and the whole lot appears at once. Any advice would be VERY welcome.

Here is my VERY basic program...

#!/usr/bin/perl -w
use strict;
use CGI::Carp qw(fatalsToBrowser);
use CGI qw:)standard escape escapeHTML);

# Force the buffer to flush
$|++;

my $i;

print << "EOF";

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://
www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">
<head>
<title>Test Buffer</title>
</head>
<body>
Working on it!<br />
EOF

for ($i=0;$i<10;$i++) {
print "Please wait!<br />\n";
sleep 1;
};

print << "EOF";
All Done!
</body>
</html>

EOF

exit (0);
 
J

Jürgen Exner

Nigel said:
Hi there,

I hope you can help - I'm writing a perl program and I want it report
back to the browser
on its progress. I read the perl faq and saw that by setting $| true
the buffer would be
flushed each time I printed instead of waiting until the program
terminates. But I can't
get it to work.

I'm expecting the web page to say that it's on the job, then every
second (for 10 seconds) to say Please Wait and then at the end to say
it's finished. But what happens is I wait ten seconds for a response
and the whole lot appears at once. Any advice would be VERY welcome.

The whole lot appears at once _WHERE_?

Run your sample program from the command line and you will notice,that
there is the desired one second pause between each "Please wait!<br
/>".

Of course your web server typically will not transmit the HTTP response
in such small chuncks but wait until it recieves EOF. And a web browser
may not display any HTML-page until it has been downloaded completely.
But neither of these has _ANYTHING_ to do with Perl but would happen
exactly the same way no matter in which programming language the CGI
program was written in.

jue
 
N

Nigel

The whole lot appears at once _WHERE_?

Run your sample program from the command line and you will notice,that
there is the desired one second pause between each  "Please wait!<br
/>".

Of course your web server typically will not transmit the HTTP response
in such small chuncks but wait until it recieves EOF. And a web browser
may not display any HTML-page until it has been downloaded completely.
But neither of these has _ANYTHING_ to do with Perl but would happen
exactly the same way no matter in which programming language the CGI
program was written in.

jue

To clarify - it all appears at once in my web browser.

So it seems I must rephrase my question and perhaps address it to a
different Group as apparently the problem isn't with my Perl as
such...but I am writing this in Perl so I guess I still have a Perl
related question: Is it possible to achieve what I want i.e. to
request my perl program from my browser and have the program tell me
the progress it is making from time to time and have that appear in my
browser in real time? My impression from what I read in the faq (and
elsewhere) was that such a thing IS possible, but perhaps I've
misunderstood. In the worked examples I saw, they talked about
'Forking' - which I understand to mean that one part of the program
responds to the browser, whilst another part carries on with the
processing. I have to say I couldn't really get my head around the
examples I saw which is why I was trying to start off simple! Thanks
for the response anyway.
 
L

Larry

Nigel said:
To clarify - it all appears at once in my web browser.

I would go something like that:

#!/perl

use strict;
use warnings;

select( (select ( STDOUT ), $|=1)[0] );

syswrite STDOUT "Content-type: text/plain\n\n";

syswrite STDOUT, "hello world!\n";

sleep 10;
syswrite STDOUT, "hello world!\n";

sleep 10;
syswrite STDOUT, "hello world!\n";

exit;

__END__;

hopefully your web server will support "chunked" content-lenght as you
do not know how long the response size is going to be...
 
X

xhoster

Nigel said:
I'm expecting the web page to say that it's on the job, then every
second (for 10 seconds) to say Please Wait and then at the end to say
it's finished. But what happens is I wait ten seconds for a response
and the whole lot appears at once. Any advice would be VERY welcome.

Here is my VERY basic program...

#!/usr/bin/perl -w
use strict;
use CGI::Carp qw(fatalsToBrowser);
use CGI qw:)standard escape escapeHTML);

# Force the buffer to flush
$|++;

my $i;

print << "EOF";

<!DOCTYPE html PUBLIC "-//W3C//DTD XHTML 1.0 Strict//EN" "http://
www.w3.org/TR/xhtml1/DTD/xhtml1-strict.dtd">
<html xmlns="http://www.w3.org/1999/xhtml" xml:lang="en">

Where is the content type header?

Without a header, my browser downloads the results as a file and only lets
me open that file once it is done. Once I add a proper header, the page
is rendered in my browser incrementally, as you expected.

Xho

--
-------------------- http://NewsReader.Com/ --------------------
The costs of publication of this article were defrayed in part by the
payment of page charges. This article must therefore be hereby marked
advertisement in accordance with 18 U.S.C. Section 1734 solely to indicate
this fact.
 
J

Jim Gibson

To clarify - it all appears at once in my web browser.

So it seems I must rephrase my question and perhaps address it to a
different Group as apparently the problem isn't with my Perl as
such...but I am writing this in Perl so I guess I still have a Perl
related question: Is it possible to achieve what I want i.e. to
request my perl program from my browser and have the program tell me
the progress it is making from time to time and have that appear in my
browser in real time? My impression from what I read in the faq (and
elsewhere) was that such a thing IS possible, but perhaps I've
misunderstood. In the worked examples I saw, they talked about
'Forking' - which I understand to mean that one part of the program
responds to the browser, whilst another part carries on with the
processing. I have to say I couldn't really get my head around the
examples I saw which is why I was trying to start off simple! Thanks
for the response anyway.

You need the "client-pull" refresh feature of HTTP that causes your
browser to periodically request a page (or CGI):

<http://www.stonehenge.com/merlyn/LinuxMag/col39.html>
 
P

Peter J. Holzer

The whole lot appears at once _WHERE_?

Run your sample program from the command line and you will notice,that
there is the desired one second pause between each "Please wait!<br
/>".

Of course your web server typically will not transmit the HTTP response
in such small chuncks but wait until it recieves EOF.

Why "of course"? The most common web server (Apache) DOES transmit
responses from CGI scripts as it receives them - it doesn't gather them
into bigger chunks. Other web servers I've worked with behave the same.
So which "typical" web server are you thinking about?
And a web browser may not display any HTML-page until it has been
downloaded completely.

Right. Although that's extremely rare these days. All the major
graphical HTML rendering engines (IE, Gecko, KHTML, ...) are able to
render a page incrementally. However, many text based browsers (e.g.,
lynx, w3m) aren't - if Nigel is using one of them that might be the
problem.

hp
 
P

Peter J. Holzer

I would go something like that:
[basically the same as the OP's code]
hopefully your web server will support "chunked" content-lenght as you
do not know how long the response size is going to be...

Neither the server nor the browser needs to know that for HTTP/1.0. And
for HTTP/1.1, chunked is mandatory.

hp
 
I

Ilya Zakharevich

[A complimentary Cc of this posting was NOT [per weedlist] sent to
Peter J. Holzer
Right. Although that's extremely rare these days. All the major
graphical HTML rendering engines (IE, Gecko, KHTML, ...) are able to
render a page incrementally. However, many text based browsers (e.g.,
lynx, w3m) aren't - if Nigel is using one of them that might be the
problem.

lynx is incremental as well...

Yours,
Ilya
 
E

Eric Pozharski

Ilya Zakharevich said:
[A complimentary Cc of this posting was NOT [per weedlist] sent to
Peter J. Holzer said:
Right. Although that's extremely rare these days. All the major
graphical HTML rendering engines (IE, Gecko, KHTML, ...) are able to
render a page incrementally. However, many text based browsers (e.g.,
lynx, w3m) aren't - if Nigel is using one of them that might be the
problem.
lynx is incremental as well...

elinks too...
 
M

Martijn Lievaart

Hi there,

I hope you can help - I'm writing a perl program and I want it report
back to the browser
on its progress. I read the perl faq and saw that by setting $| true the
buffer would be
flushed each time I printed instead of waiting until the program
terminates. But I can't
get it to work.

I'm expecting the web page to say that it's on the job, then every
second (for 10 seconds) to say Please Wait and then at the end to say
it's finished. But what happens is I wait ten seconds for a response and
the whole lot appears at once. Any advice would be VERY welcome.

Some versions (all?) of IE needs 256 characters before they will behave
correctly. Try to print 256 spaces somewhere. I wrote scripts that did
behave like what you told until I added the spaces.

HTH,
M4
 
P

Peter J. Holzer

Not the version I have here (2.8.7dev9) which appears to be almost the
newest developer version (there's a 2.8.7dev10 on lynx.isc.org). So I
wouldn't expect the stable release (2.8.6) to be incremental, either.
elinks too...

Yup. That's the exception I am aware of.

hp
 
N

Nigel

Some versions (all?) of IE needs 256 characters before they will behave
correctly. Try to print 256 spaces somewhere. I wrote scripts that did
behave like what you told until I added the spaces.

HTH,
M4

Some versions (all?) of IE needs 256 characters before they will behave
correctly. Try to print 256 spaces somewhere. I wrote scripts that did
behave like what you told until I added the spaces.

HTH,
M4

Thank you all for your input. In the end I decided to adapt the
solution suggested by Jim Gibson:
You need the "client-pull" refresh feature of HTTP that causes your
browser to periodically request a page (or CGI):

<http://www.stonehenge.com/merlyn/LinuxMag/col39.html>

My needs are rather specific as I'm developing a web front-ended
database for a company not a website for use by the public. Anyway my
approach may interest some others: I have a program that collects the
parameters with which the User wishes to run the batch process in
question. Once their input is validated I open a new window with the
URL of my batch program + the parameters. The batch program in turns
runs some intitialisation processing before sending a page back to the
browser. In that page I have an automatic redirect back to my batch
program with slightly amend parameters so that it now starts to
process the required records. After a predetermined number of records
or a period of time (as yet undecided about that) it again sends a
page to the browser - again with an automatic redirect. This cycle
continues until the processing is complete at which point the Batch
Program sends a last page to the browser (without the automatic
redirect!) to tell the user the result of the process. I use some
parameters to store information between iterations of the batch
process to avoid it having to repeat too much processing each time it
is invoked. I also store some information in cookies so the function
that 'submits' the batch process can keep track of its progress. If
the batch process fails or the window is closed, the cookies expire
and the next time the user runs the submission function it tidies up
any submitted process(es) that no longer have a cookie i.e. have
failed in one way or another.

So in conclusion - I am using the Client-Pull method, but I'm not
keeping the batch process active all the time with the use of a fork
as described in the article above. Why? Purely pragmatic reasons - I'm
a completely self taught perl programer and the article gets into
aspects of perl and server side prcessing that are well outside my
level of understanding. The processing time is not critical and I
don't think it will be huge anyway. In any case the architecture of
the program is pretty similar to that outlined in the article so if it
becomes necessary I could change it in the future.

One particular point about the article that I couldn't get my head
around is how, when the batch program is re-invoked by the client-
pull, one avoids that simply starting up another instance of the
program instead of interupting the existing instance to get an update
on its progress. Can anyone enlighten me?

Thanks again for the input!

Nigel
 
I

Ilya Zakharevich

[A complimentary Cc of this posting was NOT [per weedlist] sent to
Peter J. Holzer
Not the version I have here (2.8.7dev9) which appears to be almost the
newest developer version (there's a 2.8.7dev10 on lynx.isc.org). So I
wouldn't expect the stable release (2.8.6) to be incremental, either.

Mine is lynx2-8-5-pre4. Maybe you are not aware that lynx will behave
as you ask it? My config file is about 3K large... E.g., I have

# Set the threshold # of lines Lynx must render before it
# redraws the screen in PARTIAL mode. Anything < 0 implies
# use of the screen size.
PARTIAL_THRES:6

Hope this helps,
Ilya
 
J

Jim Gibson

Thank you all for your input. In the end I decided to adapt the
solution suggested by Jim Gibson:



One particular point about the article that I couldn't get my head
around is how, when the batch program is re-invoked by the client-
pull, one avoids that simply starting up another instance of the
program instead of interupting the existing instance to get an update
on its progress. Can anyone enlighten me?

The program uses the session id to indicate that a request is a
follow-up for a previous new request. The first request does not have a
session id, so param('session') in line 9 of the sample program returns
false (undef). Each subsequent refresh request contains the 'session'
parameter, and the value of that parameter is used to fetch the results
so far, as in line 11.

Good luck!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,768
Messages
2,569,574
Members
45,051
Latest member
CarleyMcCr

Latest Threads

Top