Dynamic Script Insertion & caching (Randy?)

[Peter Michaux]

Hi,

I'm playing with dynamic script insertion to make a request to the
server for a JavaScript file to be automatically run when it arrives
in the browser. It works but...

The page caching is too good. When I revisit the page or refresh the
page, and then redo the script insertion, the browser doesn't even hit
the server to check for a newer version of the JavaScript file. The
same old script runs with each insertion.

If I insert the same script in a page repeatedly (without a page
reload) the server is not hit looking for a newer version but the
script does run each time.

If I attach a query string with a random number to the script's URL
then the server is hit. However if the script hasn't changed this is
an unnecessary waste of download because the cached version never gets
used.

Any ideas on getting caching working better for dynamic script
insertion?

Thanks,
Peter

 

[pcx99]

Hi,

I'm playing with dynamic script insertion to make a request to the
server for a JavaScript file to be automatically run when it arrives
in the browser. It works but...

The page caching is too good. When I revisit the page or refresh the
page, and then redo the script insertion, the browser doesn't even hit
the server to check for a newer version of the JavaScript file. The
same old script runs with each insertion.

If I insert the same script in a page repeatedly (without a page
reload) the server is not hit looking for a newer version but the
script does run each time.

If I attach a query string with a random number to the script's URL
then the server is hit. However if the script hasn't changed this is
an unnecessary waste of download because the cached version never gets
used.

Any ideas on getting caching working better for dynamic script
insertion?

Thanks,
Peter

var timestamp = new Date();
var headID = document.getElementsByTagName("head")[0];
var newScript = document.createElement('script');
newScript.src =
'http://www.somedomain.com/somescript.js?'+timestamp.getTime();
headID.appendChild(newScript);


The unique url will foil the cache.

 

[Jim Ley]

If I attach a query string with a random number to the script's URL
then the server is hit. However if the script hasn't changed this is
an unnecessary waste of download because the cached version never gets
used.

Any ideas on getting caching working better for dynamic script
insertion?

Make sure the headers sent by the server are appropriate - e.g.
must-revalidate

Jim.

 

[optimistx]

Peter Michaux wrote:
....

Any ideas on getting caching working better for dynamic script
insertion?

Thanks,
Peter

I have my javascript files in my homebased Apache- server as php-files (!).
Every file has php-code in the beginning . The code e.g. constructs headers
for
caching (how many minutes or days, old versions can be kept in cache).
In addition, in this way it is easy to define gzip (packing) to be applied
in the
server, if the requesting browser accepts it.
This is a hobby environment, not for real production work, but this seems
to work nicely with 'Grade A browsers' (YAHOO-terminology...).
Packing mimized javascript reduces the file size to 1/3 typically.
Even with dynamic generation of images the headers work wonderfully,
one can see how the user picks images from the cache, if it is at most 60
seconds
old, else the browser retrieves it from the server.

Hopefully nobody says that the bee cannot fly, it is too heavy for that

 

[VK]

Any ideas on getting caching working better for dynamic script
insertion?

To know if a new updated info is available the client has to query the
server - there is no other way around. The only alternative is to set
a timed update (every 5, 10, x sec)

If working within the same domain or over server-side content grabber
(so within the same-origin limits) then the best IMO to use ajaxoid
sending HEAD request with some reasonable frequency (1sec or so) and
make full GET or POST request as soon as new update reported in
response headers. That will minimize the traffic and overall script
engine load.

For cross-domain RPCs - thus for the <script> implants techniques - a
timed request seems the only option. In this case on no new data
server may return something like
/*NOP*/;
or
"NOP";
to minimize parser's work.

Could be a better ways of course.

 

[Randy Webb]

Peter Michaux said the following on 2/1/2007 8:47 AM:

Hi,

I'm playing with dynamic script insertion to make a request to the
server for a JavaScript file to be automatically run when it arrives
in the browser. It works but...

The page caching is too good. When I revisit the page or refresh the
page, and then redo the script insertion, the browser doesn't even hit
the server to check for a newer version of the JavaScript file. The
same old script runs with each insertion.

If I insert the same script in a page repeatedly (without a page
reload) the server is not hit looking for a newer version but the
script does run each time.

If I attach a query string with a random number to the script's URL
then the server is hit. However if the script hasn't changed this is
an unnecessary waste of download because the cached version never gets
used.

Any ideas on getting caching working better for dynamic script
insertion?

Precisely and exactly what Jim said. Set the headers for it. Anything
else you do is a crutch for not doing it right

 

[Randy Webb]

pcx99 said the following on 2/1/2007 10:30 AM:

Hi,

I'm playing with dynamic script insertion to make a request to the
server for a JavaScript file to be automatically run when it arrives
in the browser. It works but...

The page caching is too good. When I revisit the page or refresh the
page, and then redo the script insertion, the browser doesn't even hit
the server to check for a newer version of the JavaScript file. The
same old script runs with each insertion.

If I insert the same script in a page repeatedly (without a page
reload) the server is not hit looking for a newer version but the
script does run each time.

If I attach a query string with a random number to the script's URL
then the server is hit. However if the script hasn't changed this is
an unnecessary waste of download because the cached version never gets
used.

Any ideas on getting caching working better for dynamic script
insertion?

Thanks,
Peter

var timestamp = new Date();
var headID = document.getElementsByTagName("head")[0];
var newScript = document.createElement('script');
newScript.src =
'http://www.somedomain.com/somescript.js?'+timestamp.getTime();
headID.appendChild(newScript);


The unique url will foil the cache.

And how will that tell the browser whether there is a new file or not?
Re-read the next to last paragraph of Peter's post and he explains -
verbatim - "However if a script hasn't changed this is an unnecessary
waste of download" which your script doesn't handle the way it needs to
be handled.

 

[Peter Michaux]

Make sure the headers sent by the server are appropriate - e.g.
must-revalidate

Hi Jim and Randy,

Thanks both. I hadn't tried setting the cache header because I figured
this might be something to do with the browsers reaction to dynamic
script insertion. (and I'm no Apache wizz.) However just adding the
following to my Apache config file seemed to make all work well.

Header set 'Cache-Control' 'no-cache'

Randy, have you done any work on an article summarizing the script
insertion results? Just curious and I'm interested to ready your
interpretation and how you use it. If you are thinking of including
those things. After thinking about one application for dynamic script
insertion today I can really see that it has important powers that XHR
doesn't. Most importantly it can take advantage of browser caching
easily and as far as I know XHR would make a struggle (or impossible?)

Thanks again,
Peter

 

[Randy Webb]

VK said the following on 2/1/2007 1:48 PM:

To know if a new updated info is available the client has to query the
server - there is no other way around. The only alternative is to set
a timed update (every 5, 10, x sec)

The "only alternative"? That line, in and of itself, speaks volumes
about you.

If working within the same domain or over server-side content grabber
(so within the same-origin limits) then the best IMO to use ajaxoid
sending HEAD request with some reasonable frequency (1sec or so) and
make full GET or POST request as soon as new update reported in
response headers. That will minimize the traffic and overall script
engine load.

You still don't get it.

For cross-domain RPCs - thus for the <script> implants techniques - a
timed request seems the only option. In this case on no new data
server may return something like

For cross-domain requests you *cant* do a "implant technique".

Could be a better ways of course.

There is, the only thing I had trouble with was coming up with a *worse*
way to attempt to do it.

 

[Randy Webb]

Peter Michaux said the following on 2/2/2007 1:49 AM:

Hi Jim and Randy,

Thanks both. I hadn't tried setting the cache header because I figured
this might be something to do with the browsers reaction to dynamic
script insertion. (and I'm no Apache wizz.) However just adding the
following to my Apache config file seemed to make all work well.

Header set 'Cache-Control' 'no-cache'

As you discovered, it doesn't matter how the file is requested, it is
still controlled by Headers - if the browser honors the header.

Randy, have you done any work on an article summarizing the script
insertion results? Just curious and I'm interested to ready your
interpretation and how you use it. If you are thinking of including
those things. After thinking about one application for dynamic script
insertion today I can really see that it has important powers that XHR
doesn't. Most importantly it can take advantage of browser caching
easily and as far as I know XHR would make a struggle (or impossible?)

I haven't had time yet to be honest, when I do you will be one of the
first to know about it.

 

[Peter Michaux]

Peter Michaux said the following on 2/2/2007 1:49 AM:







As you discovered, it doesn't matter how the file is requested, it is
still controlled by Headers - if the browser honors the header.

I'm still a little nervous that inserting a script with the same src
attribute twice may only hit the server once for a particular page.
This would be regardless of the headers sent the first time the script
is inserted. Do you want to add another column to your table of
results and we can try?

I take it you already use these techniques in production?

Peter

 

[Randy Webb]

Peter Michaux said the following on 2/3/2007 12:16 PM:

I'm still a little nervous that inserting a script with the same src
attribute twice may only hit the server once for a particular page.

It will unless you do something to prevent it.

This would be regardless of the headers sent the first time the script
is inserted. Do you want to add another column to your table of
results and we can try?

We can try it, I don't mind at all. I want to split that page into two
pages and adding another column with regards to headers may end up being
a third page to it.

I take it you already use these techniques in production?

I use a time stamp and force a re-download from the server. The files I
use are so small that the download isn't an impact though. I think there
is one file out of 100 or so that is larger than 1K.

 

[Peter Michaux]

<snip about dynamic script insertion and ensuring the browser doesn't
just use the cached version the second time the same script is
inserted in a page>

We can try it, I don't mind at all. I want to split that page into two
pages and adding another column with regards to headers may end up being
a third page to it.


I use a time stamp and force a re-download from the server. The files I
use are so small that the download isn't an impact though. I think there
is one file out of 100 or so that is larger than 1K.

A URL query string time stamp? While we are at it that deserves a test
too.

If we are going to go to the trouble of firing up 40+ browsers can we
change the tests? Instead of using alerts for each we could have the
HTML to be inserted in the results table appear in a textarea element.
I think that is supported in browsers as far back as...well...a long
time ago.

Peter

 

[pcx99]

Yea, my bad. I don't always have all the time in the world to read
this group and it seemed like a simple straight-forward issue until you
get to the 2nd to last paragraph. Still useful info tho, even if not
necessarily for Mr. Michaux

pcx99 said the following on 2/1/2007 10:30 AM:

Hi,

I'm playing with dynamic script insertion to make a request to the
server for a JavaScript file to be automatically run when it arrives
in the browser. It works but...

The page caching is too good. When I revisit the page or refresh the
page, and then redo the script insertion, the browser doesn't even hit
the server to check for a newer version of the JavaScript file. The
same old script runs with each insertion.

If I insert the same script in a page repeatedly (without a page
reload) the server is not hit looking for a newer version but the
script does run each time.

If I attach a query string with a random number to the script's URL
then the server is hit. However if the script hasn't changed this is
an unnecessary waste of download because the cached version never gets
used.

Any ideas on getting caching working better for dynamic script
insertion?

Thanks,
Peter

var timestamp = new Date();
var headID = document.getElementsByTagName("head")[0];
var newScript = document.createElement('script');
newScript.src =
'http://www.somedomain.com/somescript.js?'+timestamp.getTime();
headID.appendChild(newScript);


The unique url will foil the cache.

And how will that tell the browser whether there is a new file or not?
Re-read the next to last paragraph of Peter's post and he explains -
verbatim - "However if a script hasn't changed this is an unnecessary
waste of download" which your script doesn't handle the way it needs to
be handled.

 

[dd]

I'm playing with dynamic script insertion to make a request to the
server for a JavaScript file to be automatically run when it arrives
in the browser. It works but...

The page caching is too good. When I revisit the page or refresh the
page, and then redo the script insertion, the browser doesn't even hit
the server to check for a newer version of the JavaScript file. The
same old script runs with each insertion.

If I insert the same script in a page repeatedly (without a page
reload) the server is not hit looking for a newer version but the
script does run each time.

If I attach a query string with a random number to the script's URL
then the server is hit. However if the script hasn't changed this is
an unnecessary waste of download because the cached version never gets
used.

Any ideas on getting caching working better for dynamic script
insertion?

I think you need to be doing the random query string. If you don't,
then the browser can make up it's own mind how to deal with caching.
Sure, all the time-to-live server settings in the world can tell the
browser
how to deal with the request when the file is already in cache, but IE
at least, will make up it's own mind quite often - based on the
caching
settings chosen by the user. I'm in exactly the same situation as you,
I'd like to use cache if it hasn't been changed on the server, but I
always
add a query string. It's either the .getTime( ) on a date object, or I
use
Math.random( ).

 

[Jim Ley]

I'm still a little nervous that inserting a script with the same src
attribute twice may only hit the server once for a particular page.
This would be regardless of the headers sent the first time the script
is inserted. Do you want to add another column to your table of
results and we can try?

This is re-inserting on the same page? Moz won't make 2 requests in
that situation no matter what the headers, you'll have to change the
URI's - between loads headers can deal with it, and in page IE at
least is better behaved.

Cheers,

Jim.

 

[VK]

The "only alternative"? That line, in and of itself, speaks volumes
about you.

So you know a way to be informed if file on server is changed without
contacting the server? I'm anxious to see a sample. AFAIK scripts with
pre-programmed sixth sense and other paranormal abilities are not
invented yet - even in the US.

You still don't get it.

I do perfectly - but you still trying to mix two flavors in one bottle
(which is your brand style IMO): i) do I have new data to load? ii) if
yes then how to enforce new data to load - instead of occasional cache
reusing. I'm answering on the first one so far.

For cross-domain requests you *cant* do a "implant technique".

Eh? For cross-domain requests "implats" (dynamically generated and
inserted <script> elements) _is_ the only technique - besides server-
side content grabbers of course. RMI over SOAP/Web service call is the
new cross-browser option also, but it's not really "script load"
discussed in this thread.

 

[Peter Michaux]

This is re-inserting on the same page?
Yes

Moz won't make 2 requests in
that situation no matter what the headers, you'll have to change the
URI's

Precisely why I'd like to test. Firefox 2 seems to make multiple
requests.

between loads headers can deal with it, and in page IE at
least is better behaved.

Peter

 

[Peter Michaux]

So you know a way to be informed if file on server is changed without
contacting the server? I'm anxious to see a sample. AFAIK scripts with
pre-programmed sixth sense and other paranormal abilities are not
invented yet - even in the US.

I think there are a few problems with your original statement.

"...there is no other way around. The only alternative..." is a
contradiction.

For your alternative the client will still have to query the server.
The client being the browser and the queries being automated based on
a timer. So the alternative is actually the same as the first
suggestion but with more detail on implementation.

Given what you probably meant that "client" is "user" it seems you
mean the user has to click some sort of button called "refresh".
However if the user takes an action that is known to make the browsers
data cache stale then the page script could be smart enough to know to
make a check.

Randy may have had some more clever ideas.

Peter

 

[Peter Michaux]

On Feb 1, 2:47 pm, "Peter Michaux" <[email protected]> wrote:

I think you need to be doing the random query string. If you don't,
then the browser can make up it's own mind how to deal with caching.
Sure, all the time-to-live server settings in the world can tell the
browser
how to deal with the request when the file is already in cache, but IE
at least, will make up it's own mind quite often - based on the
caching
settings chosen by the user. I'm in exactly the same situation as you,
I'd like to use cache if it hasn't been changed on the server, but I
always
add a query string. It's either the .getTime( ) on a date object, or I
use
Math.random( ).

I think the query string can fail because browser caching doesn't
necessarily have to recognize two URLs with only different query
strings as different resources.

Peter