AJAX Mash-up Sites?

[VUNETdotUS]

My research I did a while ago showed there was no possibility to get
web page content from a third-party website with AJAX only, without
using a server side technology. Now I have to re-investigate this case
and look for a workaround, perhaps, to allow client side to get the
content of the external page, living on another server. In case you
are wondering, there is no content stealing: both parties agree to
exchange data.
Please, advise if you know of any examples, links or suggestion as to
how a client can request external page content.
Thanks.

 

[David Dorward]

My research I did a while ago showed there was no possibility to get
web page content from a third-party website with AJAX only, without
using a server side technology. Now I have to re-investigate this case
and look for a workaround, perhaps, to allow client side to get the
content of the external page, living on another server. In case you
are wondering, there is no content stealing: both parties agree to
exchange data.

If the third party can be trusted you can have them provide the data
as a piece of JavaScript, and pass data to them in the query string:

<script type="text/javascript">
function myFunction(third_party_code) {
// etc
}
</script>
<script src="http://example.com/foo/?hello=world">
// which would contain something like:
myFunction({ foo: 'bar', baz: [1,2,3,4] });
</script>

The script element that sourced the third party data could be
dynamically generated.

 

[Peter Michaux]

My research I did a while ago showed there was no possibility to get
web page content from a third-party website with AJAX only, without
using a server side technology. Now I have to re-investigate this case
and look for a workaround, perhaps, to allow client side to get the
content of the external page, living on another server. In case you
are wondering, there is no content stealing: both parties agree to
exchange data.
Please, advise if you know of any examples, links or suggestion as to
how a client can request external page content.
Thanks.

If your site is foo.com and the other is bar.net then you can play a
trick...

Set up the domain name servers so that bar.foo.com points to bar.net

Then in your JavaScript write

document.domain = 'foo.com';

Now you can make Ajax requests to both foo.com and bar.foo.com. It's
just like you can make requests to foo.com and bar.net.

This works around the XMLHttpRequest "same origin policy".

I believe that I read this trick on Ajaxian some time this year.

Peter

 

[David Mark]

My research I did a while ago showed there was no possibility to get
web page content from a third-party website with AJAX only, without
using a server side technology. Now I have to re-investigate

this case

That's not true.

and look for a workaround, perhaps, to allow client side to get the
content of the external page, living on another server. In case you
are wondering, there is no content stealing: both parties agree to
exchange data.
Please, advise if you know of any examples, links or suggestion as to
how a client can request external page content.

The same way it requests any other content. It only fails if a user's
browser settings restrict cross-domain requests. Since this may rule
it out for your particular application, you can use dynamically
created script elements as described in a previous post.

 

[Thomas 'PointedEars' Lahn]

If your site is foo.com and the other is bar.net then you can play a
trick...

Set up the domain name servers so that bar.foo.com points to bar.net

Then in your JavaScript write

document.domain = 'foo.com';

Now you can make Ajax requests to both foo.com and bar.foo.com. It's
just like you can make requests to foo.com and bar.net.

This works around the XMLHttpRequest "same origin policy".

It doesn't. This works for DOM Level 0 objects only.


PointedEars

 

[Peter Michaux]

It doesn't. This works for DOM Level 0 objects only.

What do you mean?

Peter

 

[Thomas 'PointedEars' Lahn]

What do you mean?

The Same Origin Policy was introduced with DOM Level 0 objects where
properties could be tainted; some properties were tainted and others were
not. The tainting was dropped later but the policy and affected properties
remained. Setting `document.domain' therefore was and is a way to work
around the SOP for those objects if there is the same second-level domain
(as you described).

http://docs.sun.com/source/816-6409-10/sec.htm#1021266

However, that does not work for XHR (as that is not part of DOM Level 0),
and that, at least partly, is good so.

http://web.archive.org/web/20050404...=javascript:xmlhttprequest:behaviour#security
http://www.mozilla.org/projects/security/components/jssec.html

This can be tested easily. Execute the following in the context of
<http://www.google.com/>:

try
{
document.domain = "google.com";

var x = new XMLHttpRequest();
x.open("GET", "http://groups.google.com/", false);
x.send(null);
window.alert(x.responseText);
}
catch (e)
{
// "Permission denied to call method XMLHttpRequest.open"
// even though document.domain was set
window.alert(e);
}

Tested with Firebug 1.05 on Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7

It might be that some UAs work differently (although IE 6+7 and Opera 9.23
behaved much the same in my tests), however that would be a security issue
that would be fixed soon.


PointedEars

 

[Peter Michaux]

The Same Origin Policy was introduced with DOM Level 0 objects where
properties could be tainted; some properties were tainted and others were
not. The tainting was dropped later but the policy and affected properties
remained. Setting `document.domain' therefore was and is a way to work
around the SOP for those objects if there is the same second-level domain
(as you described).

http://docs.sun.com/source/816-6409-10/sec.htm#1021266

However, that does not work for XHR (as that is not part of DOM Level 0),
and that, at least partly, is good so.

http://web.archive.org/web/20050404...a.org/projects/security/components/jssec.html

This can be tested easily. Execute the following in the context of
<http://www.google.com/>:

try
{
document.domain = "google.com";

var x = new XMLHttpRequest();
x.open("GET", "http://groups.google.com/", false);
x.send(null);
window.alert(x.responseText);
}
catch (e)
{
// "Permission denied to call method XMLHttpRequest.open"
// even though document.domain was set
window.alert(e);
}

Tested with Firebug 1.05 on Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7

It might be that some UAs work differently (although IE 6+7 and Opera 9.23
behaved much the same in my tests), however that would be a security issue
that would be fixed soon.

Well, it looks like my memory has betrayed me on this one. I only
played with the document.domain property once over a year ago. I
looked around on Ajaxian and couldn't find the article I remember
reading about playing a trick with the domain name servers. There is
some trick of some kind out there somewhere.

Thanks,
Peter

 

[Peter Michaux]

[...] Thomas 'PointedEars' Lahn [...] wrote:
Peter Michaux wrote:
If your site is foo.com and the other is bar.net then you can play a
trick...
Set up the domain name servers so that bar.foo.com points to bar.net
Then in your JavaScript write
document.domain = 'foo.com';
Now you can make Ajax requests to both foo.com and bar.foo.com. It's
just like you can make requests to foo.com and bar.net.
This works around the XMLHttpRequest "same origin policy".
It doesn't. This works for DOM Level 0 objects only.
What do you mean?

The Same Origin Policy was introduced with DOM Level 0 objects where
properties could be tainted; some properties were tainted and others were
not. The tainting was dropped later but the policy and affected properties
remained. Setting `document.domain' therefore was and is a way to work
around the SOP for those objects if there is the same second-level domain
(as you described).

However, that does not work for XHR (as that is not part of DOM Level 0),
and that, at least partly, is good so.

This can be tested easily. Execute the following in the context of
<http://www.google.com/>:

try
{
document.domain = "google.com";

var x = new XMLHttpRequest();
x.open("GET", "http://groups.google.com/", false);
x.send(null);
window.alert(x.responseText);
}
catch (e)
{
// "Permission denied to call method XMLHttpRequest.open"
// even though document.domain was set
window.alert(e);
}

Tested with Firebug 1.05 on Mozilla/5.0 (Windows; U; Windows NT 5.1; en-US;
rv:1.8.1.7) Gecko/20070914 Firefox/2.0.0.7

It might be that some UAs work differently (although IE 6+7 and Opera 9.23
behaved much the same in my tests), however that would be a security issue
that would be fixed soon.

Well, it looks like my memory has betrayed me on this one. I only
played with the document.domain property once over a year ago. I
looked around on Ajaxian and couldn't find the article I remember
reading about playing a trick with the domain name servers. There is
some trick of some kind out there somewhere.

Perhaps something like this...

<URL: http://www.xml.com/pub/a/2005/11/09/fixing-ajax-xmlhttprequest-considered-harmful.html?page=2>

In relation to my previous example, this shows using apache to proxy

http://foo.com/bar/ to http://bar.net/

Peter

 

[VUNETdotUS]

Is IFRAME trick possible if I load IFRAME with src='anotherdomain.com'
and access with document.getElementById("myiframe").contentWindow?
It does get a reference to IFRAME object but I cannot find a way to
get its innerText or innerHTML property.
Thanks.

 

[Erwin Moller]

My research I did a while ago showed there was no possibility to get
web page content from a third-party website with AJAX only, without
using a server side technology. Now I have to re-investigate this case
and look for a workaround, perhaps, to allow client side to get the
content of the external page, living on another server. In case you
are wondering, there is no content stealing: both parties agree to
exchange data.
Please, advise if you know of any examples, links or suggestion as to
how a client can request external page content.
Thanks.

Hi,

As you have seen in the other responses, it will be tricky: changing
policy in browser, or using an iframe.
Which brings me to: WHY do you avoid serverside scripting?
Getting a random page via AJAX is simple WITH serverside scripting.

In PHP it would be simple as:

<?php
$requestedPage = $_POST["requestedpage"];
echo file_get_contents($requestedPage);
?>

What is wrong with a little serverside help?

Regards,
Erwin Moller

 

[VUNETdotUS]

My research I did a while ago showed there was no possibility to get
web page content from a third-party website with AJAX only, without
using a server side technology. Now I have to re-investigate this case
and look for a workaround, perhaps, to allow client side to get the
content of the external page, living on another server. In case you
are wondering, there is no content stealing: both parties agree to
exchange data.
Please, advise if you know of any examples, links or suggestion as to
how a client can request external page content.
Thanks.

Hi,

As you have seen in the other responses, it will be tricky: changing
policy in browser, or using an iframe.
Which brings me to: WHY do you avoid serverside scripting?
Getting a random page via AJAX is simple WITH serverside scripting.

In PHP it would be simple as:

<?php
$requestedPage = $_POST["requestedpage"];
echo file_get_contents($requestedPage);
?>

What is wrong with a little serverside help?

Regards,
Erwin Moller

It is a very high traffic problem.

 

[Peter Michaux]

Is IFRAME trick possible if I load IFRAME with src='anotherdomain.com'
and access with document.getElementById("myiframe").contentWindow?
It does get a reference to IFRAME object but I cannot find a way to
get its innerText or innerHTML property.

If the second-level domains do not match then you cannot get around
the same origin policy. A page from foo.com cannot inspect the
contents of an iframe from bar.net.

Peter

 

[Aaron Saray]

If the second-level domains do not match then you cannot get around
the same origin policy. A page from foo.com cannot inspect the
contents of an iframe from bar.net.

Peter

You can do some fun DNS/subdomain tricks - but it requires
communication with the mashup service providers.

*shameless self promotion* check this link:
http://www.102degrees.com/blog/2007/09/19/cross-domain-ajax-a-quick-anatomy-of-a-mashup/
I pointed to two of the technical articles, and then filled in the
blanks on the things I thought they were missing.