use java.nio for http client?

Discussion in 'Java' started by Corey_G, May 7, 2004.

  1. Corey_G

    Corey_G Guest

    I am writing some code to create an HTTP client. URLConnection (and
    HttpURLConnection) seem to be too limited, so I want to do it at the
    socket level. It looks like I can choose java.io or java.nio for
    doing this. I read nio is newer and "high performance". I am going
    to eventually use this as the basis of a load testing tool, so
    scalabilty is important.

    any advice? is there any reason not to use nio instead of io?

    regards,
    -Corey Goldberg
    Corey_G, May 7, 2004
    #1
    1. Advertising

  2. Corey_G

    Sudsy Guest

    Corey_G wrote:
    > I am writing some code to create an HTTP client. URLConnection (and
    > HttpURLConnection) seem to be too limited, so I want to do it at the
    > socket level. <snip>


    Ah, so you want to reinvent the wheel! Good for you!!
    One question: how does HttpURLConnection "seem to be too limited"?
    Could you give us an all an example of what's missing, enough to
    justify designing/developing/debugging code which implements a
    standard protocol?
    Are you then going to extend it further and write some native code
    just so you can REALLY get down to the "nuts and bolts"?
    Please do yourself a favour and reconsider.
    Very few applications need more than what is already provided. If
    you truly believe that your situation is exceptional then please
    explain. There just might be a simpler (and standard) mechanism...
    Sudsy, May 7, 2004
    #2
    1. Advertising

  3. Corey_G

    Roedy Green Guest

    On Thu, 06 May 2004 22:45:06 -0400, Sudsy <>
    wrote or quoted :

    >
    >Ah, so you want to reinvent the wheel! Good for you!!


    IT could just be for the learning experience.

    It is good for one's soul to write such code, as it writing some sort
    of compiler.

    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
    Roedy Green, May 7, 2004
    #3
  4. On Fri, 07 May 2004 04:38:08 GMT, Roedy Green wrote:

    > On Thu, 06 May 2004 22:45:06 -0400, Sudsy <>
    > wrote or quoted :

    ...
    >>Ah, so you want to reinvent the wheel! Good for you!!

    >
    > IT could just be for the learning experience.


    The bit at the top of the post you trimmed.. viz.
    "I am writing some code to create an HTTP client. URLConnection (and
    HttpURLConnection) seem to be too limited, so I want to do it at the
    socket level"

    Seems to suggest 'No'.

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
    Andrew Thompson, May 7, 2004
    #4
  5. Corey_G

    Tony Morris Guest

    > any advice?
    HTTP client using java.nio:
    http://java.sun.com/developer/technicalArticles/releases/nio/

    > is there any reason not to use nio instead of io?

    When supporting VM versions < 1.4

    --
    Tony Morris
    (BInfTech, Cert 3 I.T.)
    Software Engineer
    (2003 VTR1000F)
    Sun Certified Programmer for the Java 2 Platform (1.4)
    Sun Certified Developer for the Java 2 Platform
    Tony Morris, May 7, 2004
    #5
  6. Corey_G

    Corey_G Guest

    > Ah, so you want to reinvent the wheel! Good for you!!
    > One question: how does HttpURLConnection "seem to be too limited"?
    > Could you give us an all an example of what's missing, enough to
    > justify designing/developing/debugging code which implements a
    > standard protocol?


    Thanks for the positive and encouraging response!

    My question wasn't "should I write my own http client?" My question
    was "I AM GOING TO write my own http client, what should I use". I am
    new to java, but have written socket level clients in many other
    languages. So part of the purpose is a learning experience in how to
    write java.. part of the purpose is to learn network and socket
    programming in java.

    > Please do yourself a favour and reconsider.


    reconsider what? learning socket programming in java by plugging in
    some pre-baked code thats already written? If this was small piece of
    larger client project, I would most likely use something like the
    Jakarta Commons HttpClient.
    But its rather difficult to learn socket programming without actually
    doing socket programming.. don't you think?

    thanks again for your support.
    Corey_G, May 7, 2004
    #6
  7. On 7 May 2004 05:59:29 -0700, Corey_G wrote:

    > My question wasn't "should I write my own http client?" My question
    > was "I AM GOING TO write my own http client, what should I use".


    OoooK..

    >...I am
    > new to java, but have written socket level clients in many other
    > languages. So part of the purpose is a learning experience in how to
    > write java.. part of the purpose is to learn network and socket
    > programming in java.


    So _this_ is the bit Sudsy was supposed to see in
    his Crystal Ball? I got it.

    Problem is, the only 'crystal' Sudsy's got is
    better suited to the containmnent a far more
    important thing, beer. ;-)

    --
    Andrew Thompson
    http://www.PhySci.org/ Open-source software suite
    http://www.PhySci.org/codes/ Web & IT Help
    http://www.1point1C.org/ Science & Technology
    Andrew Thompson, May 7, 2004
    #7
  8. Sudsy <> wrote:
    > Corey_G wrote:
    >> I am writing some code to create an HTTP client. URLConnection (and
    >> HttpURLConnection) seem to be too limited, so I want to do it at the
    >> socket level. <snip>

    >
    > Ah, so you want to reinvent the wheel! Good for you!!
    > One question: how does HttpURLConnection "seem to be too limited"?
    > Could you give us an all an example of what's missing, enough to
    > justify designing/developing/debugging code which implements a
    > standard protocol?
    > Are you then going to extend it further and write some native code
    > just so you can REALLY get down to the "nuts and bolts"?
    > Please do yourself a favour and reconsider.
    > Very few applications need more than what is already provided. If
    > you truly believe that your situation is exceptional then please
    > explain. There just might be a simpler (and standard) mechanism...


    I'm sure this is unrelated to why he thinks it is too limited,
    but I too was thinking of doing my own "HttpURLConnection" class for
    this reason:

    There is no way to set a reasonable time out value for the connection.

    I am interested in writing 2 apps. A website mirroring tool, and a proxy
    checker. Each of which will fire up several threads. Most good website
    mirroring tools allow the user to specify the timeout period, and I was
    thinking of doing the same.

    Any ideas ?
    electric sheep, May 7, 2004
    #8
  9. electric sheep wrote:


    > I'm sure this is unrelated to why he thinks it is too limited,
    > but I too was thinking of doing my own "HttpURLConnection" class for
    > this reason:
    >
    > There is no way to set a reasonable time out value for the connection.
    >
    > I am interested in writing 2 apps. A website mirroring tool, and a proxy
    > checker. Each of which will fire up several threads. Most good website
    > mirroring tools allow the user to specify the timeout period, and I was
    > thinking of doing the same.
    >
    > Any ideas ?


    Jakarta commons HttpClient allows you to specifiy the timeouts.

    --
    Kind regards,
    Christophe Vanfleteren
    Christophe Vanfleteren, May 7, 2004
    #9
  10. Corey_G wrote:
    > My question wasn't "should I write my own http client?" My question
    > was "I AM GOING TO write my own http client, what should I use".


    And you got your answer: URLConnection.

    > I am
    > new to java,


    So this gives you enough competence to tell us all that URLConnection is
    to limited for your purpose? For the purpose which you refused to state
    until now? Oh no, wait, you originally wrote:

    >>> I am going
    >>> to eventually use this as the basis of a load testing tool, so
    >>> scalabilty is important.


    But now you say

    > So part of the purpose is a learning experience


    And why didn't you tell us this the first time? People here usually
    don't program for the learning experience, but to solve real world
    problems. Consider to have your learning experience in
    comp.lang.java.help instead.

    > in how to
    > write java.


    And you got your answer: Use the pre-existing classes on the highest
    possible abstraction layer. This is how things are done in Java.

    > reconsider what?


    Your attitude. Your way of formulating questions. Your way of handling
    the good advice you got. Your bad case of "not invented here" syndrome.

    > learning socket programming in java by plugging in
    > some pre-baked code thats already written?


    Nowhere in your original post you mentioned that you want to learn Java
    socket programming. Don't blame us for your inability to express your
    wishes.

    /Thomas
    Thomas Weidenfeller, May 7, 2004
    #10
  11. Christophe Vanfleteren <> wrote:
    > electric sheep wrote:
    >
    >
    >> I'm sure this is unrelated to why he thinks it is too limited,
    >> but I too was thinking of doing my own "HttpURLConnection" class for
    >> this reason:
    >>
    >> There is no way to set a reasonable time out value for the connection.
    >>
    >> I am interested in writing 2 apps. A website mirroring tool, and a proxy
    >> checker. Each of which will fire up several threads. Most good website
    >> mirroring tools allow the user to specify the timeout period, and I was
    >> thinking of doing the same.
    >>
    >> Any ideas ?

    >
    > Jakarta commons HttpClient allows you to specifiy the timeouts.
    >

    awesome !
    that looks like a great package.
    thanks.
    electric sheep, May 7, 2004
    #11
  12. Corey_G

    Corey_G Guest

    > Your attitude. Your way of formulating questions. Your way of handling
    > the good advice you got.


    apologies on my part then (although im still somewhat confused how
    that was inferred from posts ;)

    I've since written clients using URLConnection, Java.io, Java.nio, and
    the Jakarta Commons HTTP Client.

    I was hoping to be enlightened on the pros and cons of using Java.nio
    for socket programming, not to start trouble. appologies again.. I
    will ask questions elsewhere next time.

    regards.
    Corey_G, May 7, 2004
    #12
  13. Corey_G

    Yu SONG Guest

    "electric sheep" <> wrote:
    > Sudsy <> wrote:
    > > Corey_G wrote:

    >
    > There is no way to set a reasonable time out value for the connection.
    >
    > I am interested in writing 2 apps. A website mirroring tool, and a proxy
    > checker. Each of which will fire up several threads. Most good website
    > mirroring tools allow the user to specify the timeout period, and I was
    > thinking of doing the same.
    >
    > Any ideas ?


    You can write a customized timeout class, which can give you more
    flexibility and control.

    --
    Song E-mail: Y*Song#d$cs*war$wick*ac*uk
    (Replace "*" -> "." AND "#" -> "@" , Remove "$")
    http://www.dcs.warwick.ac.uk/~esubbn/
    Yu SONG, May 10, 2004
    #13
  14. Corey_G

    Marc Slemko Guest

    In article <c7mev5$7jv$>, Yu SONG wrote:
    > "electric sheep" <> wrote:
    >> Sudsy <> wrote:
    >> > Corey_G wrote:

    >>
    >> There is no way to set a reasonable time out value for the connection.
    >>
    >> I am interested in writing 2 apps. A website mirroring tool, and a proxy
    >> checker. Each of which will fire up several threads. Most good website
    >> mirroring tools allow the user to specify the timeout period, and I was
    >> thinking of doing the same.
    >>
    >> Any ideas ?

    >
    > You can write a customized timeout class, which can give you more
    > flexibility and control.


    No, no you can't unless you want to leave threads sitting around stuck,
    unable to kill them or make progress.

    You can't set a socket timeout since the code behind HttpURLConnection
    doesn't let you get at the socket.

    And even in 1.5, they "fixed" this problem but obviously still don't
    get the idea of timeouts since they added a read and connect timeout but
    no write timeout.

    There is a bunch of other stuff Sun does behind the scenes that is
    pretty nasty for certain applications, I don't know if they have fixed
    them in 1.5 or not.

    Anyone who has used Java's built in HttpURLConnection class for any
    significant volume of requests to arbitrary remote webservers will have
    numerous reasons why using it is a non-starter. It is pretty shameful
    how many really basic things like this do not have any useful standard
    implementations that are included in J2SE.
    Marc Slemko, May 10, 2004
    #14
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Avizz
    Replies:
    3
    Views:
    13,791
    Andy Fish
    Sep 29, 2003
  2. iksrazal

    NIO with timeouts != NIO?

    iksrazal, Jun 17, 2004, in forum: Java
    Replies:
    1
    Views:
    6,246
    iksrazal
    Jun 18, 2004
  3. Replies:
    4
    Views:
    406
  4. brieuc
    Replies:
    5
    Views:
    2,172
    Esmond Pitt
    Mar 20, 2007
  5. Replies:
    2
    Views:
    195
    Petri Heinilä
    Feb 15, 2013
Loading...

Share This Page