how do APIs such as Google get around the Ajax sandbox?

[Jake Barnes]

I looked at this FAQ but found no answers related to Ajax:

http://www.irt.org/script/misc.htm


I looked at this FAQ, but there was very little information:

http://www.javascripter.net/faq/


I looked at this FAQ but didn't find an answer to my question:

http://www.jibbering.com/faq/#ajax


My problem is simple enough. Go here and you'll see that the links to
the comments say "many" where they should, instead, be reporting the
number of comments:

http://www.calculatedriskblog.com/

The comments show up over here:

http://www.hoocoodanode.org/node/6536

The hoocoodanode.org server has a PHP script that will tell me how
many comments each post has. The data is returned in JSON format:

http://www.hoocoodanode.org/comment...2=/343830017414438466&id_3=/93467763726066064

It is easy enough to write an Ajax request on calculatedriskblog.com
that could get this data, except I run into the Ajax sandbox.

My question: lots of services, such as the Google Map API, seem to get
around the restrictions imposed on on Ajax requests. How is this done?
I know that in the past I've populated maps that drew data from the
Google Maps API, and I've had this render on domains other than
google.com. How do they break out of the sandbox?

 

[Laser Lips]

I looked at this FAQ but found no answers related to Ajax:

http://www.irt.org/script/misc.htm

I looked at this FAQ, but there was very little information:

http://www.javascripter.net/faq/

I looked at this FAQ but didn't find an answer to my question:

http://www.jibbering.com/faq/#ajax

My problem is simple enough. Go here and you'll see that the links to
the comments say "many" where they should, instead, be reporting the
number of comments:

http://www.calculatedriskblog.com/

The comments show up over here:

http://www.hoocoodanode.org/node/6536

The hoocoodanode.org server has a PHP script that will tell me how
many comments each post has. The data is returned in JSON format:

http://www.hoocoodanode.org/comment_cnt?id_1=/3294774681922076612&id_....

It is easy enough to write an Ajax request on calculatedriskblog.com
that could get this data, except I run into the Ajax sandbox.

My question: lots of services, such as the Google Map API, seem to get
around the restrictions imposed on on Ajax requests. How is this done?
I know that in the past I've populated maps that drew data from the
Google Maps API, and I've had this render on domains other than
google.com. How do they break out of the sandbox?

If you mean you want to use Ajax to retrieve content from other web
servers other than the one your running your site from then I've done
it using a 'relay' script or' jump' script written in PHP.
Point your Ajax script at the PHP script on your server and pass it
all the parameters you need including the external URL you want to
query. Then get the PHP script to retrieve the page and echo out the
output. It's down to you to deal with the response form the page in
PHP.

Hope that helps.
Graham Vincent

 

[Jorge]

(...)
My question: lots of services, such as the Google Map API, seem to get
around the restrictions imposed on on Ajax requests. How is this done?
I know that in the past I've populated maps that drew data from the
Google Maps API, and I've had this render on domains other than
google.com. How do they break out of the sandbox?

Almost invariably, they use the so called "script tag hack". There's
currently no better way to do it. The idea is:

1.- Create and insert in your page an empty <script> tag whose .src
contains the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName"

2.- The server extracts the paramenters, processes the request and
formats the answer as a json text :

e.g. {'cityName': 'Some City'}

3.- Note that the client is expecting a <script>. A script gets
executed inmediatly after arriving, and it does so at the global
context. So, for example, assigning the json text to a global variable
would do, and this would be a valid <script> to send back:

mapsResponse= {'cityName': 'Some City'};

4.- At the client-side, the data becomes accessible as the global var
mapsResponse.

This is the idea. Data travels encapsulated in scripts, because the
scripts are not subject to the same origin policy. The example above
makes (I hope) it easy to grasp the whole process, although, the truth
is that passing the returned data to a global var as above is not the
most convenient way to do it. Using a callback function is much
better:

1.- Create and insert in your page a <script> tag whose .src contains
the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName&callBack=callBackFuncitonName"

Note the last parameter "callBack=callBackFuncitonName". It tells the
server that we want to have the returned data passed to the function
"callBackFuncitonName".

2.- The server extracts the parameters, processes the request and
respond with a script like this :

callBackFuncitonName({'cityName': 'Some City'});

3.- At the client side, when the script finally arrives it gets
automatically executed (in the global context), IOW, the function
callBackFuncitonName() gets called and receives the data as a
parameter.

Obviously, the callback must be accesible from the global context.
Most of the times you'll find yourself attaching the callBack to your
apps' global object: myApp.mapsThings.callBack() or something like
that. Just pass the "fully qualified name" to the server:
"&callback=myApp.mapsThings.callBack".

HTH,

 

[Tad J McClellan]

If you mean you want to use Ajax to retrieve content from other web
servers other than the one your running your site from then I've done
it using a 'relay' script or' jump' script

The usual name for such a thing is a "proxy server".

http://en.wikipedia.org/wiki/Proxy_server

 

[Eric Bednarz]

Almost invariably, they use the so called "script tag hack". There's
currently no better way to do it.

Huh? The easiest way I know is Apache’s mod_rewrite proxy throughput,
and I think most server-side programming languages allow you to make an
HTTP request and mangle the response data.

 

[Jorge]

Huh? The easiest way I know is Apache’s mod_rewrite proxy throughput,
and I think most server-side programming languages allow you to make an
HTTP request and mangle the response data.

That's fine, and if your server provides such a service you can do
cross-domain XMLHTTPRequest(s).
But most of Google's and Yahoo's json apis currently don't require
that, as they're based on the script tag hack.

 

[slebetman]

That's fine, and if your server provides such a service you can do
cross-domain XMLHTTPRequest(s).
But most of Google's and Yahoo's json apis currently don't require
that, as they're based on the script tag hack.

Actually, most of Yahoo's newer apis use flash to do the ajax request.
Flash does not suffer from same origin policy. Of course, now your
clients need flash installed. Then again, YouTube has made almost
everyone want to install flash.

 

[Jorge]

Actually, most of Yahoo's newer apis use flash to do the ajax request.
Flash does not suffer from same origin policy. Of course, now your
clients need flash installed. Then again, YouTube has made almost
everyone want to install flash.

"Using JSON with Yahoo! Web Services" :
http://developer.yahoo.com/common/json.html#output

 

[Lawrence Krubner]

The usual name for such a thing is a "proxy server".

   http://en.wikipedia.org/wiki/Proxy_server

Sorry, I should have explained, the first site is hosted by Blogger,
which doesn't allow us any server side tricks. So we've got to pull
this off without any of the usual tricks. But that shouldn't be
impossible, because we are trying to replace comment systems like
Haloscan and JS-Kit, and they manage to do what we want to do, and
they face the same constraints that we face. If they figured out a way
to do it, then clearly it must be possible.

 

[Lawrence Krubner]

Almost invariably, they use the so called "script tag hack". There's
currently no better way to do it. The idea is:

1.- Create and insert in your page an empty <script> tag whose .src
contains the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName"

2.- The server extracts the paramenters, processes the request and
formats the answer as a json text :

e.g. {'cityName': 'Some City'}

3.- Note that the client is expecting a <script>. A script gets
executed inmediatly after arriving, and it does so at the global
context. So, for example, assigning the json text to a global variable
would do, and this would be a valid <script> to send back:

mapsResponse= {'cityName': 'Some City'};

If I understand this correctly, you're suggesting I embed something
like this into the HTML at calculatedriskblog.com:

<script type="text/javascript" src="http://www.hoocoodanode.org/js/
comment_cnt?id_1=95403982934&id_2=4949586954439202"></script>

And this script should contain the JSON response and its assignment,
so the text (that is, the Javascript code) that is included looks like
this:

commentCounts = { ”/3294774681922076612”: “55”, ”/343830017414438466”:
“181”, ”/93467763726066064”: “251” }

And you're also saying that the variable commentCounts will suddenly
be available in the global context?

 

[Jorge]

If I understand this correctly, you're suggesting I embed something
like this into the HTML at calculatedriskblog.com:

<script type="text/javascript" src="http://www.hoocoodanode.org/js/
comment_cnt?id_1=95403982934&id_2=4949586954439202"></script>

And this script should contain the JSON response and its assignment,
so the text (that is, the Javascript code) that is included looks like
this:

commentCounts = { ”/3294774681922076612”: “55”, ”/343830017414438466”:
“181”, ”/93467763726066064”: “251” }

And you're also saying that the variable commentCounts will suddenly
be available in the global context?

Yes, exactly :

http://jorgechamorro.com/cljs/051/

 

[Bart Van der Donck]

It is easy enough to write an Ajax request on calculatedriskblog.com
that could get this data, except I run into the Ajax sandbox.

My question: lots of services, such as the Google Map API, seem to get
around the restrictions imposed on on Ajax requests. How is this done?
I know that in the past I've populated maps that drew data from the
Google Maps API, and I've had this render on domains other than
google.com. How do they break out of the sandbox?

This issue had inspired me to write Ajax-Cross-Domain:

http://www.ajax-cross-domain.com/

 

[lawrence]

Almost invariably, they use the so called "script tag hack". There's
currently no better way to do it. The idea is:

1.- Create and insert in your page an empty <script> tag whose .src
contains the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName"

How should I create and insert this script tag? Do I use document.write
()? Or DOM methods? Either?

 

[lawrence]

Almost invariably, they use the so called "script tag hack". There's
currently no better way to do it. The idea is:

1.- Create and insert in your page an empty <script> tag whose .src
contains the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName"

2.- The server extracts the paramenters, processes the request and
formats the answer as a json text :

e.g. {'cityName': 'Some City'}

3.- Note that the client is expecting a <script>. A script gets
executed inmediatly after arriving, and it does so at the global
context. So, for example, assigning the json text to a global variable
would do, and this would be a valid <script> to send back:

mapsResponse= {'cityName': 'Some City'};

4.- At the client-side, the data becomes accessible as the global var
mapsResponse.

This is the idea. Data travels encapsulated in scripts, because the
scripts are not subject to the same origin policy. The example above
makes (I hope) it easy to grasp the whole process, although, the truth
is that passing the returned data to a global var as above is not the
most convenient way to do it. Using a callback function is much
better:

1.- Create and insert in your page a <script> tag whose .src contains
the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName&callBack=callBackFuncitonName"

Hmm, I tried this:

document.write("<script type='text/javascript' src='" +
fullUrlForToGetCommentCountFromHoocoodanode + "'><script>");
alert(commentCounts);

The variable fullUrlForToGetCommentCountFromHoocoodanode points at a
page that simply has this on it, and nothing else:

commentCounts = { '/93467763726066064': '255', '/3687512778221280499':
'154', '/7469483397667998711': '69', '/3546694631903279790': '86', '/
7826626581502083668': '257', '/7863118590857837142': '3', '/
4528442172302293469': '121', '/4746505715184542507': '125', '/
3734446988393833799': '242', '/3855898527917724656': '190', '/
2322556671825296195': '89', '/7828790542573728707': '83', '/
7789277747613265886': '137', '/856477599963783632': '109', '/
170917561786384883': '119', '/2415194963852019364': '66', '/
4594614219998113399': '106', '/8621485541364545191': '297', '/
2546912594140426303': '159', '/2124986657277402452': '284', '/
294853826990957498': '112' }

And yet, when the alert() gets called, I get an error:

Error: commentCounts is not defined

Is that because the other page hasn't loaded yet, or am I building the
script tag incorrectly?

 

[lawrence]

Almost invariably, they use the so called "script tag hack". There's
currently no better way to do it. The idea is:

1.- Create and insert in your page an empty <script> tag whose .src
contains the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName"

2.- The server extracts the paramenters, processes the request and
formats the answer as a json text :

e.g. {'cityName': 'Some City'}

3.- Note that the client is expecting a <script>. A script gets
executed inmediatly after arriving, and it does so at the global
context. So, for example, assigning the json text to a global variable
would do, and this would be a valid <script> to send back:

mapsResponse= {'cityName': 'Some City'};

4.- At the client-side, the data becomes accessible as the global var
mapsResponse.

This is the idea. Data travels encapsulated in scripts, because the
scripts are not subject to the same origin policy. The example above
makes (I hope) it easy to grasp the whole process, although, the truth
is that passing the returned data to a global var as above is not the
most convenient way to do it. Using a callback function is much
better:

1.- Create and insert in your page a <script> tag whose .src contains
the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName&callBack=callBackFuncitonName"

Note the last parameter "callBack=callBackFuncitonName". It tells the
server that we want to have the returned data passed to the function
"callBackFuncitonName".

2.- The server extracts the parameters, processes the request and
respond with a script like this :

callBackFuncitonName({'cityName': 'Some City'});

3.- At the client side, when the script finally arrives it gets
automatically executed (in the global context), IOW, the function
callBackFuncitonName() gets called and receives the data as a
parameter.

Obviously, the callback must be accesible from the global context.
Most of the times you'll find yourself attaching the callBack to your
apps' global object: myApp.mapsThings.callBack() or something like
that. Just pass the "fully qualified name" to the server:
"&callback=myApp.mapsThings.callBack".

Okay, I tried adding in a call back:

document.write("<script type='text/javascript' src='" +
fullUrlForToGetCommentCountFromHoocoodanode +
"&callBack=updateTheCommentCountsOnEachPost'><script>");


But the callback doesn't seem to ever be called.

Here is the wider context of the script so far:



// 04-06-09 - We want to provide a count of how many comments have
// been posted to each post, so after the front page of Calculated
Risk loads
// we need to loop over all the comment-link nodes on the page to get
the comment
// URLs. We want to know how many comments are appearing
hoocoodanode.org nodes
// such as this:
//
// http://www.hoocoodanode.org/node/6533
//
// This URL:
//
// http://www.hoocoodanode.org/comment...2=/343830017414438466&id_3=/93467763726066064
//
// Gives me this JSON response:
//
// { �/3294774681922076612�: �55�, �/343830017414438466�: �181�, �/
93467763726066064�: �251� }
//
// so we need to collect all the ids from all the comment links on CR,
and put them
// in a big string that we can send to hoocoodanode.org as an Ajax
request.
addLoadEvent(function() {
var arrayOfNodesWithUrlsForComments = getElementsByClass("comment-
link");
encodeQueryStringsInCommentLinks(arrayOfNodesWithUrlsForComments);
var stringOfIdsToBeUsedToQueryHoocoodanode =
getAllIdsAsAQueryStringReadyToBeSentToHoocoodanode
(arrayOfNodesWithUrlsForComments);
var fullUrlForToGetCommentCountFromHoocoodanode = "http://
www.hoocoodanode.org/comment_cnt?" +
stringOfIdsToBeUsedToQueryHoocoodanode;
document.write("<script type='text/javascript' src='" +
fullUrlForToGetCommentCountFromHoocoodanode +
"&callBack=updateTheCommentCountsOnEachPost'><script>");







});

function updateTheCommentCountsOnEachPost() {
alert(commentCounts);
}



What am I doing wrong?

 

[lawrence]

Almost invariably, they use the so called "script tag hack". There's
currently no better way to do it. The idea is:

1.- Create and insert in your page an empty <script> tag whose .src
contains the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName"

2.- The server extracts the paramenters, processes the request and
formats the answer as a json text :

e.g. {'cityName': 'Some City'}

3.- Note that the client is expecting a <script>. A script gets
executed inmediatly after arriving, and it does so at the global
context. So, for example, assigning the json text to a global variable
would do, and this would be a valid <script> to send back:

mapsResponse= {'cityName': 'Some City'};

4.- At the client-side, the data becomes accessible as the global var
mapsResponse.

This is the idea. Data travels encapsulated in scripts, because the
scripts are not subject to the same origin policy. The example above
makes (I hope) it easy to grasp the whole process, although, the truth
is that passing the returned data to a global var as above is not the
most convenient way to do it. Using a callback function is much
better:

1.- Create and insert in your page a <script> tag whose .src contains
the parameters:

e.g. script.src= "example.com/mapsData.json?
lat=0.0&long=0.0&q=cityName&callBack=callBackFuncitonName"

Note the last parameter "callBack=callBackFuncitonName". It tells the
server that we want to have the returned data passed to the function
"callBackFuncitonName".

2.- The server extracts the parameters, processes the request and
respond with a script like this :

callBackFuncitonName({'cityName': 'Some City'});

3.- At the client side, when the script finally arrives it gets
automatically executed (in the global context), IOW, the function
callBackFuncitonName() gets called and receives the data as a
parameter.

Obviously, the callback must be accesible from the global context.
Most of the times you'll find yourself attaching the callBack to your
apps' global object: myApp.mapsThings.callBack() or something like
that. Just pass the "fully qualified name" to the server:
"&callback=myApp.mapsThings.callBack".

Instead of using document.write(), I've started using DOM methods:

// 04-13-09 - the next 3 lines are borrowed from here:
// http://userscripts.org/topics/782
var hoocoodanodeScript = document.createElement('script');
hoocoodanodeScript.setAttribute('src',
fullUrlForToGetCommentCountFromHoocoodanodeWithCallbackAppended);
document.body.appendChild(hoocoodanodeScript);


However, these don't work any better. The callback doesn't seem to get
called. What am I missing?

 

[beegee]

However, these don't work any better. The callback doesn't seem to get
called. What am I missing?

I think the problem may be in your server side code. I don't know
what serve side technology you are using but if you were using php it
would need to look like this:

<?php

$retStr = "";
if (isset($_GET["callback"]))
{
$callback = $_GET["callback"];
$retStr = "$callback(" . $the_JSON_you_want_to_return . ");";
}
header('Content-Type: text/javascript');
echo $retStr;
?>

This is pretty cool and I had no idea you could do it. Thanks Jorge.
However:

http://javascript.crockford.com/script.html

Crockford says, don't do it.

Bob

 

[beegee]

I think the problem may be in your server side code.  I don't know
what serve side technology you are using but if you were using php it
would need to look like this:

....SNIP...

Ooops. This is kind of confusing. Jorge specified two links in his
replies. One was the yahoo link that I assume you were trying.
However, he also specified a link to his example page:
http://jorgechamorro.com/cljs/051/ where he accomplishes the desired
task without server code and without a callback. I'm not actually
sure which one you are trying or whether you're the OP?!?

In any case, my reply concerned using the yahoo approach. You
definitely need to write the server code in that case but I think it's
a cool api.

Bob

 

[Lawrence Krubner]

...SNIP...

Ooops.  This is kind of confusing.  Jorge specified two links in his
replies.  One was the yahoo link that I assume you were trying.
However, he also specified a link to his example page:http://jorgechamorro.com/cljs/051/where he accomplishes the desired
task without server code and without a callback.  I'm not actually
sure which one you are trying or whether you're the OP?!?

In any case, my reply concerned using the yahoo approach.  You
definitely need to write the server code in that case but I think it's
a cool api.

Okay, maybe I misunderstood. Right now our PHP page sends this to the
screen:

commentCounts = { '/93467763726066064': '255', '/3687512778221280499':
'154', '/7469483397667998711': '69', '/3546694631903279790': '86', '/
7826626581502083668': '257', '/7863118590857837142': '3', '/
4528442172302293469': '121', '/4746505715184542507': '125', '/
3734446988393833799': '242', '/3855898527917724656': '190', '/
2322556671825296195': '89', '/7828790542573728707': '83', '/
7789277747613265886': '137', '/856477599963783632': '109', '/
170917561786384883': '119', '/2415194963852019364': '66', '/
4594614219998113399': '106', '/8621485541364545191': '297', '/
2546912594140426303': '159', '/2124986657277402452': '284', '/
294853826990957498': '112' }

And then, on our main page, which the public sees, we have a script
tag that imports this page. But having read a bit more, I'm now under
the impression that the PHP should be sending something like this to
the screen:

updateComments({ '/93467763726066064': '255', '/3687512778221280499':
'154', '/7469483397667998711': '69', '/3546694631903279790': '86', '/
7826626581502083668': '257', '/7863118590857837142': '3', '/
4528442172302293469': '121', '/4746505715184542507': '125', '/
3734446988393833799': '242', '/3855898527917724656': '190', '/
2322556671825296195': '89', '/7828790542573728707': '83', '/
7789277747613265886': '137', '/856477599963783632': '109', '/
170917561786384883': '119', '/2415194963852019364': '66', '/
4594614219998113399': '106', '/8621485541364545191': '297', '/
2546912594140426303': '159', '/2124986657277402452': '284', '/
294853826990957498': '112' })

And then, on the page that is importing this "script" (the output of
the PHP) we need to define a function called "updateComments" which
can then do something useful with this JSON?

 

[Lawrence Krubner]

Instead of using document.write(), I've started using DOM methods:

        // 04-13-09 - the next 3 lines are borrowed from here:
        //http://userscripts.org/topics/782
        var hoocoodanodeScript = document.createElement('script');
        hoocoodanodeScript.setAttribute('src',
fullUrlForToGetCommentCountFromHoocoodanodeWithCallbackAppended);
        document.body.appendChild(hoocoodanodeScript);

However, these don't work any better. The callback doesn't seem to get
called. What am I missing?

I'm looking at this tutorial here:

http://userscripts.org/topics/782

I'm wondering if this is the correct way to add in the script tag?

var myscript = document.createElement('script');
myscript.src = 'http://www.mydomain.com/my.js';
document.body.appendChild(myscript);

Does the code get executed, when you do this?