Python CGI and Browser timeout

S

skulka3

Hello,

I am creating a simple cgi script which needs to retrieve and process
a huge number of records from the database (more than 11,000) and
write the results to a file on disk and display some results when
processing is complete.

However, nothing needs to be displayed while the processing is on. I
was facing browser timeout issue due to the time it takes to process
these records.

In order to work around this problem, I started printing empty strings
(i.e. print "") so that the browser does not timeout.

Is there a better solution to avoid browser timeouts?

Thanks,
 
G

Gabriel Genellina

En Thu said:
I am creating a simple cgi script which needs to retrieve and process
a huge number of records from the database (more than 11,000) and
write the results to a file on disk and display some results when
processing is complete.

However, nothing needs to be displayed while the processing is on. I
was facing browser timeout issue due to the time it takes to process
these records.

In order to work around this problem, I started printing empty strings
(i.e. print "") so that the browser does not timeout.

Is there a better solution to avoid browser timeouts?
Click to expand...

You could spawn another process or thread, reporting the progress
somewhere.
Then redirect to another page showing the progress (and auto-reloading
itself each few seconds).
When it detects that processing is complete, redirect to another page
showing the final results.
 
S

Sebastian Bassi

In order to work around this problem, I started printing empty strings
(i.e. print "") so that the browser does not timeout.
Click to expand...

How do you print something while doing the query and waiting for the results?
I saw some pages that display something like: "This page will be
updated in X seconds to show the results" (X is an estimated time
depending of server load), after a JS countdown, it refresh itself and
show the result or another "This page will be updated in X seconds to
show the results".
 
S

Sebastian Bassi

Is there a better solution to avoid browser timeouts?
Click to expand...

Raising timeout in Apache, by default is 300 seconds.
Limiting jobs size (both in the html form and from script size since
you should not trust on client validations).
 
S

skulka3

Thanks for the response.

To further clarify the details:

I am printing the empty strings in a for loop. So the processing
happens in a loop when all the results from the query have been
already retrieved and each record is now being processed inside the
loop.

I also update the display periodically with the total number of
records processed(which is approximately after every 1/5th chunk of
the total number of records in the result).

Thanks,
Salil Kulkarni
 
S

Steve Holden

Sebastian said:
How do you print something while doing the query and waiting for the results?
I saw some pages that display something like: "This page will be
updated in X seconds to show the results" (X is an estimated time
depending of server load), after a JS countdown, it refresh itself and
show the result or another "This page will be updated in X seconds to
show the results".
Click to expand...

The usual way is by "client pull": send the content you want the user to
see, and include a Refresh: header - the easiest way is to include a
META tag in the html content <head> section like

<meta http-equiv="refresh" content="N; URL=other-web-address">

So the page can continually check whether the user's job is finished, if
it isn't just sending out the same content and then when it is printing
the details.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get Python in your .sig and on the web. Blog and lens
holdenweb.blogspot.com squidoo.com/pythonology
tag items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------
 
S

Steve Holden

Steve said:
The usual way is by "client pull": send the content you want the user to
see, and include a Refresh: header - the easiest way is to include a
META tag in the html content <head> section like

<meta http-equiv="refresh" content="N; URL=other-web-address">

So the page can continually check whether the user's job is finished, if
it isn't just sending out the same content and then when it is printing
the details.
Click to expand...
I should have pointed out that N is the number of seconds to wait before
refreshing.

regards
Steve
--
Steve Holden +1 571 484 6266 +1 800 494 3119
Holden Web LLC/Ltd http://www.holdenweb.com
Skype: holdenweb http://del.icio.us/steve.holden
------------------ Asciimercial ---------------------
Get Python in your .sig and on the web. Blog and lens
holdenweb.blogspot.com squidoo.com/pythonology
tag items: del.icio.us/steve.holden/python
All these services currently offer free registration!
-------------- Thank You for Reading ----------------
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Similar Threads

Image upload not working in browser 4
CGI NET::SSH browser problem 1
Replacing spreadsheets by Python and the browser 5
CGI question 3
CGI, POST, and file uploads 0
Python CGI 3
cgi and popen 2
pipes python cgi and gnupg 1

Members online

No members online now.
Total: 40 (members: 1, guests: 39)
Robots: 207

Forum statistics

Threads
473,764
Messages
2,569,564
Members
45,039
Latest member
CasimiraVa

Latest Threads

Top