proxy baffement

Discussion in 'Java' started by Roedy Green, Nov 18, 2011.

  1. Roedy Green

    Roedy Green Guest

    This is not so much a question about Java as about HTTP.

    I wrote a program called Brokenlinks that tests all the links on my
    website and tells me about permanent redirects and sites that have
    stayed dead for 6+ days. (It filters out temporary outages).

    Something strange is happening with http://ask.com

    My program, written in Java, says it is always dead. Firefox often
    thinks it is alive.

    I fired up Wireshark to see if I could figure out what was going on.

    The first thing I noticed was this

    GET
    /tbproxy/lh/fixurl?hl=en-US&sd=ca&url=http%3A%2F%2Fask.com%2F&sourceid=chrome&error=connectionfailure
    HTTP/1.1
    Host: linkhelp.clients.google.com

    Somehow my request is sometimes going through a Google proxy. Buy
    WHY? What has google got to do with this? I am am using Firefox, not
    Chrome.

    Further, I notice Chrome often says "resolving proxy" and dithers for
    a long time loading a LOCAL file off my hard disk. Why would a web
    proxy be involved at all?

    Opera has been completely unusable for about two years because it
    takes about a minute to load a file off local disk (it is fine on the
    web). I thought had something to do with Google Javascript fetching
    ads and translate slowly, but maybe that too is a proxy problem. The
    Opera people ignored me every time I reported the problem.

    I always thought proxies just did a bit of caching, but otherwise you
    could ignore them. Yet it seem clearly my browser is speaking a
    different protocol from usual. It seems to be aware of its presence.

    My questions:

    If my IAP is using a proxy, what are the benefits?

    Is there a way to sneak around it in case it is screwing things up?

    Why google? Do the host proxies for IAPs? Is ASK owned by Google?

    Is my Java program supposed to be proxy-aware and do something
    different?

    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 18, 2011
    #1
    1. Advertising

  2. Roedy Green

    Roedy Green Guest

    On Thu, 17 Nov 2011 18:31:51 -0800, Roedy Green
    <> wrote, quoted or indirectly quoted
    someone who said :

    >
    >Is there a way to sneak around it in case it is screwing things up?


    I posted what I have discovered so far at
    http://mindprod.com/jgloss/proxy.html

    Another puzzle has come out. Where does Windows get its its notion of
    what proxy to use from? It is not part of the DHCP protocol. The
    docs I have found suggest it has to learn by manually configuring an
    IP. I most definitely did not do that. I wonder if some Google
    product did it.

    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 18, 2011
    #2
    1. Advertising

  3. Roedy Green <> wrote:
    > My program, written in Java, says it [ask.com] is always dead.
    > Firefox often thinks it is alive.
    > I fired up Wireshark to see if I could figure out what was going on.
    > The first thing I noticed was this
    > GET
    > /tbproxy/lh/fixurl?hl=en-US&sd=ca&url=http%3A%2F%2Fask.com%2F&sourceid=chrome&error=connectionfailure
    > HTTP/1.1
    > Host: linkhelp.clients.google.com


    The "error=connectionfailure" makes it look like the result of some
    other software that gets triggered by failed connect-attempts and
    tries to retrieve separate information for the failure. It could
    be a browser-addon, but perhaps could also be software installed
    at system level (chrome surely looks quite suspicious for that).

    Doesn't wireshark show some SYN packets going to ask.com before this
    line?

    > Opera has been completely unusable for about two years because it
    > takes about a minute to load a file off local disk (it is fine on the
    > web).


    Cannot reproduce that with opera on my machine. (and perhaps the
    opera-people weren't able, either.) Perhaps you use network-drives
    mounted locally? Or an over-eager virus-scanner, that intercepts
    local file reads? Or such without the "-scanner"...

    > Is there a way to sneak around it in case it is screwing things up?


    Subscribe to a different one. But your problem doesn't so far look
    like it was the IAP's responsibility.
    While they could force a more or less transparent proxy upon all your
    connections to the outside world, they surely have no means to interfere
    with local file accesses, unless something goes wrong in the browser.
    Andreas Leitgeb, Nov 18, 2011
    #3
  4. Roedy Green

    Roedy Green Guest

    On 18 Nov 2011 08:11:30 GMT, Andreas Leitgeb
    <> wrote, quoted or indirectly quoted
    someone who said :

    >Doesn't wireshark show some SYN packets going to ask.com before this
    >line?


    I was only looking at GETs
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 18, 2011
    #4
  5. Roedy Green

    Roedy Green Guest

    On 18 Nov 2011 08:11:30 GMT, Andreas Leitgeb
    <> wrote, quoted or indirectly quoted
    someone who said :

    >> Opera has been completely unusable for about two years because it
    >> takes about a minute to load a file off local disk (it is fine on the
    >> web).

    >
    >Cannot reproduce that with opera on my machine


    It would be helpful if you performed this experiment.

    Download a typical page from my website, e.g.
    http://mindprod.com/jgloss/wireshark.html

    and save it on local hard disk.

    The try loading it with Opera.

    If it loads fine that suggests there is something weird with my
    machine.
    If it is very slow to render, that suggests there is something strange
    about my pages.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 18, 2011
    #5
  6. Roedy Green

    Roedy Green Guest

    On 18 Nov 2011 08:11:30 GMT, Andreas Leitgeb
    <> wrote, quoted or indirectly quoted
    someone who said :

    >Cannot reproduce that with opera on my machine. (and perhaps the
    >opera-people weren't able, either.) Perhaps you use network-drives
    >mounted locally? Or an over-eager virus-scanner, that intercepts
    >local file reads? Or such without the "-scanner"...


    In trying to track this down I turned off dynamic virus checking,
    Windows index, Copernic index. I have most of the accelerators, e.g.
    Open Office that load at boot time turned off. This proxy thing looks
    like the best lead so far since it is in there without my permission.

    The was another anomaly than seems to have gone away. Timeouts in my
    Java code HTTP Get code sometimes would not time out. They just sat
    there forever.

    Maybe it is time for another Windows 7 install to scape off the
    barnacles.

    Where does your father keep his barnacles?
    ~ One of Charles Darwin’s children to a school friend.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 18, 2011
    #6
  7. On Fri, 18 Nov 2011 02:44:08 -0800, Roedy Green
    <> wrote:

    >On 18 Nov 2011 08:11:30 GMT, Andreas Leitgeb
    ><> wrote, quoted or indirectly quoted
    >someone who said :
    >
    >>> Opera has been completely unusable for about two years because it
    >>> takes about a minute to load a file off local disk (it is fine on the
    >>> web).

    >>
    >>Cannot reproduce that with opera on my machine

    >
    >It would be helpful if you performed this experiment.
    >
    >Download a typical page from my website, e.g.
    >http://mindprod.com/jgloss/wireshark.html
    >
    >and save it on local hard disk.
    >
    >The try loading it with Opera.
    >
    >If it loads fine that suggests there is something weird with my
    >machine.
    >If it is very slow to render, that suggests there is something strange
    >about my pages.


    I do not use Opera, but I tried this with Firefox 8.0 under
    Windows XP SP 3. The render from file was fine.

    Sincerely,

    Gene Wirchenko
    Gene Wirchenko, Nov 18, 2011
    #7
  8. Roedy Green

    Roedy Green Guest

    On Fri, 18 Nov 2011 09:44:48 -0800, Gene Wirchenko <>
    wrote, quoted or indirectly quoted someone who said :

    > I do not use Opera, but I tried this with Firefox 8.0 under
    >Windows XP SP 3. The render from file was fine.


    For me Chrome work quite well from local disk. Opera is impossible,
    and Firefox is usually ok, but sometimes takes a long time.

    The catch is Chrome can't do Applets, so I have to toggle back and
    forth between Chrome and FIrefox. Firefox has a weird stuttering
    problem with Java. All is going fine and it gets into its head to
    reload the page without the Java. I manually reload the page and it
    comes back. Even when it is working fine, FIrefox calls my init
    methods twice when a page loads.


    The browser people seem to be trying as hard as Microsoft to kill
    Java.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 18, 2011
    #8
  9. On 18/11/11 02:31, Roedy Green wrote:
    > GET
    > /tbproxy/lh/fixurl?hl=en-US&sd=ca&url=http%3A%2F%2Fask.com%2F&sourceid=chrome&error=connectionfailure
    > HTTP/1.1
    > Host: linkhelp.clients.google.com
    >
    > Somehow my request is sometimes going through a Google proxy. Buy
    > WHY? What has google got to do with this? I am am using Firefox, not
    > Chrome.


    Are you sure the request is from your program? Does User-Agent show it
    to be Java, at least? You could set it to be more specific, to be sure
    it's /your/ Java program.

    --
    ss at comp dot lancs dot ac dot uk
    Steven Simpson, Nov 18, 2011
    #9
  10. Roedy Green <> wrote:
    > I fired up Wireshark to see if I could figure out what was going on.
    > The first thing I noticed was this
    > GET
    > /tbproxy/lh/fixurl?hl=en-US&sd=ca&url=http%3A%2F%2Fask.com%2F&sourceid=chrome&error=connectionfailure
    > HTTP/1.1
    > Host: linkhelp.clients.google.com


    Another thing: I googled for "tbproxy fixurl" and got hits mostly talking
    about some google-script to be used for a site's 404-page.
    Andreas Leitgeb, Nov 19, 2011
    #10
  11. Roedy Green <> wrote:
    > It would be helpful if you performed this experiment.
    > Download a typical page from my website, e.g.
    > http://mindprod.com/jgloss/wireshark.html
    > and save it on local hard disk.


    Did that, but opened it in a textviewer instead, and saw a couple
    of <script ... src="http://..."> in it. So much for "local."
    Andreas Leitgeb, Nov 19, 2011
    #11
  12. Roedy Green

    Roedy Green Guest

    On Thu, 17 Nov 2011 18:31:51 -0800, Roedy Green
    <> wrote, quoted or indirectly quoted
    someone who said :

    >Is my Java program supposed to be proxy-aware and do something
    >different?


    In poking around to figure out what is going on I discovered a proxy
    can be configured in the Windows control panel. There is also a place
    to configure one in the Java Control panel.

    I found a note suggesting that if you ticked "automatically detect
    settings" in the Windows control panel, you might end up with a proxy.

    There are three Java system properties.
    System.setProperty( "proxySet", "true" );
    System.setProperty( "http.proxyHost", proxyHostName );
    System.setProperty( "http.proxyPort", Integer.toString( proxyHostPort
    ) );

    It seems all you need to hook up a proxy is its dns name or ip. Even
    though there are about 5 differerent proxy protocols, I gather the
    machines sort that out themselves.

    I also discovered inside Google Chrome in "under the hood" you can
    change proxy settings. However, I think that is just a hook into the
    Windows control panel.


    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 19, 2011
    #12
  13. Roedy Green

    Roedy Green Guest

    On 19 Nov 2011 11:18:15 GMT, Andreas Leitgeb
    <> wrote, quoted or indirectly quoted
    someone who said :

    >Did that, but opened it in a textviewer instead, and saw a couple
    >of <script ... src="http://..."> in it. So much for "local."


    What you saw were google adsense ads and a google translate widget.

    The odd thing is when the same page is loaded from a website it works
    fine. It is WAY slower when loaded off local hard disk.

    This may simply be Google screwing up, or deliberately hosing other
    browsers besides Google Chrome which works fairly well.

    Yet I think some people don't have trouble with loading from local
    hard disk.
    --
    Roedy Green Canadian Mind Products
    http://mindprod.com
    I can't come to bed just yet. Somebody is wrong on the Internet.
    Roedy Green, Nov 21, 2011
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. vidy5300
    Replies:
    0
    Views:
    862
    vidy5300
    Jun 7, 2004
  2. Replies:
    0
    Views:
    1,641
  3. Karsten Wutzke
    Replies:
    1
    Views:
    876
    Steven Simpson
    Aug 20, 2007
  4. Klein Stéphane
    Replies:
    3
    Views:
    592
    Steve Holden
    Dec 20, 2009
  5. Alejandro Molina

    Proxy and non Proxy configurations

    Alejandro Molina, Jul 15, 2004, in forum: ASP .Net Security
    Replies:
    0
    Views:
    148
    Alejandro Molina
    Jul 15, 2004
Loading...

Share This Page