Iterated mega file upload - invented wheel wanted

Discussion in 'Javascript' started by john_ramsden@sagitta-ps.com, Mar 3, 2005.

  1. Guest

    I have to prepare an Apache/PHP based (or perl if easier) web
    page that allows a user to upload very large files, possibly
    over 10 Gbytes and maybe up to 100 Gbytes.

    I'm aware that file upload limits can be increased by config
    changes; but I'd rather minimize the need for changes like
    this, both for admin reasons and to avoid possible (?) memory problems
    or even bugs.

    In principle there's a simple solution whereby the user runs
    an initial page, in which they input the file name, and the
    Submit button then runs JavaScript in the page.

    The JavaScript opens the file, positions at a start offset
    (initially zero), and reads and uploads up to a fixed length
    of the contents, along with a continuation offset (or zero
    to indicate the end reached), and the server code re-runs
    the script in a loop while this returned offset is non-zero.
    Subsequent runs of the script would bypass the data input
    and execute the JavaScript directly.

    Anyway, the question I have is whether anyone has written
    anything similar, ideally with a progress bar?

    The only problem I anticipate is possible delays, and even
    timeouts, while the code (client or server) positions to
    the next offset to read or write (respectively) the next
    chunk of a very large, e.g. 50 GByte, file.

    Any suggestions gratefully received.

    (It's a shame Apache doesn't seem to have an embedded FTP
    server in some way. I guess FTP would be the obvious choice,
    but the client doesn't wish to have a separate FTP server
    running.)


    Cheers

    John R Ramsden (jramsden at glassouse.com)
    , Mar 3, 2005
    #1
    1. Advertising

  2. wrote in news:1109862480.383312.250480
    @z14g2000cwz.googlegroups.com:

    > The JavaScript opens the file, positions at a start offset
    > (initially zero), and reads and uploads up to a fixed length
    > of the contents, along with a continuation offset (or zero
    > to indicate the end reached), and the server code re-runs
    > the script in a loop while this returned offset is non-zero.
    > Subsequent runs of the script would bypass the data input
    > and execute the JavaScript directly.


    Do you have a Perl question?

    Sinan
    A. Sinan Unur, Mar 3, 2005
    #2
    1. Advertising

  3. Guest

    A. Sinan Unur wrote:
    >
    > wrote in news:1109862480.383312.250480
    > @z14g2000cwz.googlegroups.com:
    > >
    > > I have to prepare an Apache/PHP based (or perl if easier)
    > > [...]
    > >
    > > The JavaScript opens the file, positions at a start offset
    > > (initially zero), and reads and uploads up to a fixed length
    > > of the contents, along with a continuation offset (or zero
    > > to indicate the end reached), and the server code re-runs
    > > the script in a loop while this returned offset is non-zero.
    > > Subsequent runs of the script would bypass the data input
    > > and execute the JavaScript directly.

    >
    > Do you have a Perl question?


    Read the first line, numbnuts: "(or perl if easier)"
    , Mar 3, 2005
    #3
  4. Anno Siegel Guest

    <> wrote in comp.lang.perl.misc:
    >
    > A. Sinan Unur wrote:
    > >
    > > wrote in news:1109862480.383312.250480
    > > @z14g2000cwz.googlegroups.com:


    > > Do you have a Perl question?

    >
    > Read the first line, numbnuts: "(or perl if easier)"


    So long then.

    Anno
    Anno Siegel, Mar 3, 2005
    #4
  5. On Thu, 03 Mar 2005 07:08:00 -0800, john_ramsden wrote:

    > (It's a shame Apache doesn't seem to have an embedded FTP server in some
    > way.


    Apache 2.x does, I think. At least the MPM architecture allows for it, I
    don't know if one has been written. As has been pointed out, FTP
    belongs to the age of a nicer internet, but httpd2 will let you do it if
    you really want to ...

    R
    Richard Gration, Mar 3, 2005
    #5
  6. J. Gleixner Guest

    wrote:
    > I have to prepare an Apache/PHP based (or perl if easier) web
    > page that allows a user to upload very large files, possibly
    > over 10 Gbytes and maybe up to 100 Gbytes.

    [...]
    > Anyway, the question I have is whether anyone has written
    > anything similar, ideally with a progress bar?

    [...]
    > Any suggestions gratefully received.


    Out of the 3 newsgroups you chose, all 3 are off topic. This isn't a
    "programming" question.

    The wheel is to build a Web page, in HTML, and put it on your Web site
    and let the browser do it for you. That's it, problem solved.

    Just have a link to the file and the client clicks on the link or does a
    right click...Save Link as. That's it. You could use PHP/CGI to build
    the list of files, but there's probably no need to call a script to
    serve the file, just use HTTP. You don't need to do anything to Apache
    or your PHP installation.

    Forget about a progress bar. Most (All?) browsers create their own
    download manager and they likely have their own progress display. If you
    really want to make your own, then an Applet is your best bet.
    J. Gleixner, Mar 3, 2005
    #6
  7. Mr. Wilson Guest

    > wrote:
    >
    >> ...that allows a user to **upload** very large files...


    [idiotic non-answer snipped]

    Didn't watch enough Sesame Street to learn the difference between up and
    down, eh?

    Heh, what a maroon.

    --Mr. Wilson
    Mr. Wilson, Mar 3, 2005
    #7
  8. Csaba2000 Guest

    This is an interesting question, but I must be missing something:
    If I understand you right, you're trying to get a large file from
    the client (browser) side to your (the server) side. And you
    want the user's browser (possibly with the aid of code you
    provide) to cooperate in this endeavor so you can get past
    the usual (including timeout) limitations.

    But javascript (really the only thing available generically) on
    the client side does not have access to the user's disk. So
    I am curious to know what I am not understanding here.
    If browsers are not involved, then I would turn to cURL
    within command line PHP (on the client side) or a similar
    variant and have the PHP on the Apache end reassemble
    the file. Haven't done it, but it seems straightforward.
    Probably would try with a command line version of cURL.

    As for progress bar, on the client side, you can use COM
    methods on recent Windows systems to put up an IE and
    keep changing the IE page as you progress along (see the
    php.net/com area for starter examples).

    Csaba Gabor from Vienna

    <> wrote in message
    news:...
    >
    > I have to prepare an Apache/PHP based (or perl if easier) web
    > page that allows a user to upload very large files, possibly
    > over 10 Gbytes and maybe up to 100 Gbytes.
    >
    > I'm aware that file upload limits can be increased by config
    > changes; but I'd rather minimize the need for changes like
    > this, both for admin reasons and to avoid possible (?) memory problems
    > or even bugs.
    >
    > In principle there's a simple solution whereby the user runs
    > an initial page, in which they input the file name, and the
    > Submit button then runs JavaScript in the page.
    >
    > The JavaScript opens the file, positions at a start offset
    > (initially zero), and reads and uploads up to a fixed length
    > of the contents, along with a continuation offset (or zero
    > to indicate the end reached), and the server code re-runs
    > the script in a loop while this returned offset is non-zero.
    > Subsequent runs of the script would bypass the data input
    > and execute the JavaScript directly.
    >
    > Anyway, the question I have is whether anyone has written
    > anything similar, ideally with a progress bar?
    >
    > The only problem I anticipate is possible delays, and even
    > timeouts, while the code (client or server) positions to
    > the next offset to read or write (respectively) the next
    > chunk of a very large, e.g. 50 GByte, file.
    >
    > Any suggestions gratefully received.
    >
    > (It's a shame Apache doesn't seem to have an embedded FTP
    > server in some way. I guess FTP would be the obvious choice,
    > but the client doesn't wish to have a separate FTP server
    > running.)
    >
    >
    > Cheers
    >
    > John R Ramsden (jramsden at glassouse.com)
    >
    Csaba2000, Mar 3, 2005
    #8
  9. juglesh Guest

    <> wrote in message
    news:...
    >
    > I have to prepare an Apache/PHP based (or perl if easier) web
    > page that allows a user to upload very large files, possibly
    > over 10 Gbytes and maybe up to 100 Gbytes.


    maybe this
    http://jupload.biz/

    havent tried it for large files (working great so far with smallish images),
    but the website says no limit...
    juglesh, Mar 4, 2005
    #9
  10. Guest

    Mr. Wilson wrote:
    >
    > > wrote:
    > >
    > >> ...that allows a user to **upload** very large files...

    >
    > [idiotic non-answer snipped]
    >
    > Didn't watch enough Sesame Street to learn the difference
    > between up and down, eh?
    >
    > Heh, what a maroon.
    >
    > --Mr. Wilson


    My post was about copying files from a client to a server,
    as should have been clear to anyone with a 3-digit IQ who
    skimmed it in more than a second.

    I bow to your superior knowledge of kids' TV. But Sesame
    Street must use a different definition of "upload" to the
    one in http://www.vivtek.com/rfc1867.html.
    , Mar 4, 2005
    #10
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Harvey
    Replies:
    0
    Views:
    665
    Harvey
    Jul 16, 2004
  2. Harvey
    Replies:
    1
    Views:
    817
    Daniel
    Jul 16, 2004
  3. Phlip

    Who invented Jemmy?

    Phlip, Aug 4, 2004, in forum: Java
    Replies:
    5
    Views:
    604
    Martin Brehovsky
    Aug 4, 2004
  4. Peter Bailey
    Replies:
    2
    Views:
    101
    Peter Bailey
    Sep 21, 2007
  5. Replies:
    9
    Views:
    114
    juglesh
    Mar 4, 2005
Loading...

Share This Page