java download manager

Discussion in 'Java' started by Jean-Daniel Gamache, Jul 14, 2004.

  1. Hi,

    I want to write a Java application that can download a file (url)
    fast.

    To do this, I want to open multiple connections on the same file(url)
    and each connection will download a chunk of the file.

    Any clue how I can do this ?

    thanks,
    JD
     
    Jean-Daniel Gamache, Jul 14, 2004
    #1
    1. Advertising

  2. Jean-Daniel Gamache

    Roedy Green Guest

    On 13 Jul 2004 21:08:00 -0700, (Jean-Daniel
    Gamache) wrote or quoted :

    >To do this, I want to open multiple connections on the same file(url)
    >and each connection will download a chunk of the file.
    >
    >Any clue how I can do this ?

    see http://mindprod.com/fileio.html

    Tell it you want to do a CGI get.

    Look up the HTTP headers to control the offset and length of the chunk
    you want, and spin off three threads after you figure out how big the
    whole thing is with a HEAD.


    --
    Canadian Mind Products, Roedy Green.
    Coaching, problem solving, economical contract programming.
    See http://mindprod.com/jgloss/jgloss.html for The Java Glossary.
     
    Roedy Green, Jul 14, 2004
    #2
    1. Advertising

  3. Jean-Daniel Gamache

    Alan Meyer Guest

    "Jean-Daniel Gamache" <> wrote in message
    news:...
    > Hi,
    >
    > I want to write a Java application that can download a file (url)
    > fast.
    >
    > To do this, I want to open multiple connections on the same

    file(url)
    > and each connection will download a chunk of the file.
    >
    > Any clue how I can do this ?
    >
    > thanks,
    > JD


    Note: I would expect your technique to be SLOWER,
    not faster.

    The speed of the download will almost certainly be
    determined by the speed of the physical link between
    the two computers. Creating two separate TCP/IP
    connections over the same link (e.g., modem and
    telephone line, modem and cable, etc.) will increase
    the overhead, i.e., the number of bits dedicated to
    connection creation and control, but will not increase
    the total number of bits per second that can flow over
    the link. So a smaller number of data bits from the file
    you want will be downloaded per second than if you
    had one connection with more of the bits dedicated
    to data instead of control.

    If you have two physical links, e.g., two phone lines
    that you can use at once, that's another story.

    However, even in that case, you'd also need to get
    the server to cooperate with you. To the best of
    my knowledge, neither the http nor the ftp protocols
    will download two halves of a file in this way. So
    you'd need a program on the server to split the
    file in half into two separate files - then go after
    each half on each connection. If it's a server that's
    not under your control, I don't think you'll be able
    to do that.

    Alan
     
    Alan Meyer, Jul 14, 2004
    #3
  4. Jean-Daniel Gamache

    Carl Howells Guest

    Alan Meyer wrote:
    >
    > Note: I would expect your technique to be SLOWER,
    > not faster.
    >


    [snip]

    >
    > However, even in that case, you'd also need to get
    > the server to cooperate with you. To the best of
    > my knowledge, neither the http nor the ftp protocols
    > will download two halves of a file in this way. So
    > you'd need a program on the server to split the
    > file in half into two separate files - then go after
    > each half on each connection. If it's a server that's
    > not under your control, I don't think you'll be able
    > to do that.


    Well, that technique has made a significant speed increase in the past,
    due to a couple of factors you're missing.

    First, many servers cap their upload speed *per connection*. If you
    open multiple connections, you can get much greater speed, up to the
    available bandwidth between your computer and the server.

    Second, the server does cooperate with the user on this, if it supports
    "resume" style functionality. Many web and FTP servers do support
    resuming a download, and therefore support this technique.

    And third... Many programs made when this style of download manager was
    popular included functionality to search for the same file on multiple
    servers, and download different parts of it from different servers.
    When this worked correctly (not that often, which is why it's pretty
    much abandoned for this type of downloading now), it also provided a
    giant speed boost.
     
    Carl Howells, Jul 14, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Ryan Taylor
    Replies:
    2
    Views:
    953
    Ryan Taylor
    Nov 10, 2004
  2. Rafal Majda
    Replies:
    5
    Views:
    2,307
    Rafal Majda
    Apr 18, 2005
  3. Replies:
    1
    Views:
    408
    Roedy Green
    Dec 2, 2005
  4. padma
    Replies:
    0
    Views:
    351
    padma
    Oct 3, 2007
  5. Metalone
    Replies:
    0
    Views:
    346
    Metalone
    Jan 6, 2010
Loading...

Share This Page