Using AJAX to do inline, inpage file uploads

Discussion in 'Javascript' started by JMB, Jan 2, 2006.

  1. JMB

    JMB Guest

    Hello,

    I was wondering if anyone knew of any projects extending the inline upload
    progress bar to utilize an inpage image uploader with bar, without having to
    refresh or go to a seperate page, nor opening a second box for display of
    the progress bar.

    I had been using the MegaUpload that was adapted from Raditha's script at
    http://www.raditha.com/upload.php . The MegaUpload script I have been using
    takes the progress bar inpage, instead of opening a new page/box for the
    display of the bar.

    Anyhow, here is what I have been wondering. I have projects with large
    amounts of 'application' usage and Web2.0 features, I guess you can say.
    This includes dynamic form submissions, inline graphical interfaces,
    realtime chat, realtime drawing/whiteboarding, etc. The thing I have not
    been able to nail is inline progress bar usage with uploaded images/files.
    Does anyone know about any projects currently working on this very thing
    that I could contribute to? Or, see some demo code to have a better idea.

    I am guessing my request to the server should be a put, but then I would
    need what from there modifed to handle this? As PHP sux for it, and the
    closest I have gotten is using a modified verision of OB_ functions. The
    Perl solution seems more feasible, but I would not know where to begin using
    the async nature of AJAX, I think sending the file via PUT method, and the
    form variables(if any), submitted during this display of the bar from a
    seperate thread to the server for storage.

    I know the uploader is PHP/Perl, but with it being an AJAX-savy script,
    posting here was a starting point.

    Thanks in advance for direction to any/all projects, as well as suggestions,
    help and comments.
     
    JMB, Jan 2, 2006
    #1
    1. Advertising

  2. JMB

    Sean Berry Guest

    "JMB" <> wrote in message
    news:i1%tf.33841$az4.26548@trndny03...
    > Hello,
    >
    > I was wondering if anyone knew of any projects extending the inline upload
    > progress bar to utilize an inpage image uploader with bar, without having
    > to refresh or go to a seperate page, nor opening a second box for display
    > of the progress bar.
    >
    > I had been using the MegaUpload that was adapted from Raditha's script at
    > http://www.raditha.com/upload.php . The MegaUpload script I have been
    > using takes the progress bar inpage, instead of opening a new page/box for
    > the display of the bar.
    >
    > Anyhow, here is what I have been wondering. I have projects with large
    > amounts of 'application' usage and Web2.0 features, I guess you can say.
    > This includes dynamic form submissions, inline graphical interfaces,
    > realtime chat, realtime drawing/whiteboarding, etc. The thing I have not
    > been able to nail is inline progress bar usage with uploaded images/files.
    > Does anyone know about any projects currently working on this very thing
    > that I could contribute to? Or, see some demo code to have a better idea.
    >
    > I am guessing my request to the server should be a put, but then I would
    > need what from there modifed to handle this? As PHP sux for it, and the
    > closest I have gotten is using a modified verision of OB_ functions. The
    > Perl solution seems more feasible, but I would not know where to begin
    > using the async nature of AJAX, I think sending the file via PUT method,
    > and the form variables(if any), submitted during this display of the bar
    > from a seperate thread to the server for storage.
    >
    > I know the uploader is PHP/Perl, but with it being an AJAX-savy script,
    > posting here was a starting point.
    >
    > Thanks in advance for direction to any/all projects, as well as
    > suggestions, help and comments.
    >


    I just did something like this a few months ago and would be happy to share
    the code. I have it working in both PERL and Python. The PERL is a
    modified version of the MegaUpload project and the Python version is my own.
    In either case I can send you the source for it. Do you also want the
    javascript? That part is pretty straight forward.

    Let me know via email and I will pass it along.
     
    Sean Berry, Jan 2, 2006
    #2
    1. Advertising

  3. JMB

    VK Guest

    JMB wrote:
    > I have projects with large
    > amounts of 'application' usage and Web2.0 features, I guess you can say.
    > This includes dynamic form submissions, inline graphical interfaces,
    > realtime chat, realtime drawing/whiteboarding, etc. The thing I have not
    > been able to nail is inline progress bar usage with uploaded images/files.
    > Does anyone know about any projects currently working on this very thing
    > that I could contribute to? Or, see some demo code to have a better idea.
    >
    > I am guessing my request to the server should be a put, but then I would
    > need what from there modifed to handle this? As PHP sux for it, and the
    > closest I have gotten is using a modified verision of OB_ functions. The
    > Perl solution seems more feasible, but I would not know where to begin using
    > the async nature of AJAX, I think sending the file via PUT method, and the
    > form variables(if any), submitted during this display of the bar from a
    > seperate thread to the server for storage.
    >
    > I know the uploader is PHP/Perl, but with it being an AJAX-savy script,
    > posting here was a starting point.


    The question of upload bar is very old (since Netscape 3.0 at least).
    It is one of Perpetuum Mobile's of the Web: it cannot be solved in the
    given physical conditions but still many people are trying to invent it
    instead of reading physics books :)

    The limitation is build into the HTTP upload model. Unlike FTP, HTTP
    upload is a start/stop event w/o intermediary states: you have no idea
    about uploading data until submission starts and you still have no idea
    of the actual data to handle until the upload is done.
    The second crutial limitation of HTTP upload is that the data is sent
    in BASE64 encoding. It means that the actual submission size is *always
    bigger* than the summary of file sizes to upload. The actual growth
    depends heavily on the binary structure of files to upload. Roughly it
    is no less than +20% and up to +100%. This is why all endless attempts
    to make progress bar based on file(s) size + bandwidth speed failed so
    miserably.

    So in order to have a reliable progress bar for HTTP upload you have to
    have *on client-side*:
    1) A mechanics to know the exact upload size of files after BASE64
    encoded.
    2) Per packed-sent notification from HTTP upload module.

    Neither of this is possible on the existing browsers. You still *can*
    write your own ActiveX / plugin to take over the whole HTTP upload
    functionality (or to have an FTP uploader right on the page). Nothing
    you can do by just using server-side + slient-side scripting in any
    shall perform combination.

    P.S. Thermodynamic laws never stopped people from trying to invent a
    perpetuum mobile. I'm affraid it is the case with the progress bar
    neither. :)

    P.P.S. No current browser supports PUT method except W3C Amaya. Just to
    save you from of one of dead branches.
     
    VK, Jan 2, 2006
    #3
  4. JMB

    Jim Ley Guest

    On 2 Jan 2006 03:01:27 -0800, "VK" <> wrote:

    >So in order to have a reliable progress bar for HTTP upload you have to
    >have *on client-side*:
    >1) A mechanics to know the exact upload size of files after BASE64
    >encoded.


    A progress bar could reasonably be done on non-base64 encoded sizes,
    the only assumption would then be that the base64 encoding is
    approximately linear - that would be a sufficiently accurate approach
    for what is already a very rough ready visual indicator.

    >2) Per packed-sent notification from HTTP upload module.


    Or a callback from the server announcing "I've got N bytes", indeed
    this may well be more accurate as it counts bytes recieved not sent
    (some may have gone missing...)

    So all that is really missing from the entire system is the total
    upload size. Certainly you need a seperate communication channel from
    the server to the client - a script available socket or
    xmlhttprequest, or an open iframe - to send the "I've got N bytes"
    messages.

    It's a relatively simple job in fact to have an "uploaded 10k, 20k,30k
    etc." message, indeed I implemented it one about 2001.

    Jim.
     
    Jim Ley, Jan 2, 2006
    #4
  5. JMB

    VK Guest

    Duncan Booth wrote:
    > VK wrote:
    >
    > > The second crutial limitation of HTTP upload is that the data is sent
    > > in BASE64 encoding. It means that the actual submission size is *always
    > > bigger* than the summary of file sizes to upload. The actual growth
    > > depends heavily on the binary structure of files to upload. Roughly it
    > > is no less than +20% and up to +100%. This is why all endless attempts
    > > to make progress bar based on file(s) size + bandwidth speed failed so
    > > miserably.
    > >

    > This is rubbish.
    >
    > Base64 encoding always increases the size of the data by one third. It does
    > not depend on the 'binary structure' of the data: quite simply 3 input
    > bytes are encoded as 4 printable characters.


    My bad! I was thinking about zipping instead of base64-encoding. The
    resulting file size is never less then source size + 30%
    At the same time for MIME base64 it is never strictly equal to
    binarySize+binarySize/100*30. Because of additional service data is
    approx 33%.
    On 5-10-more files package this "approx" transforms a strict
    calculation into guessing game.
    Also there is such things as i) bandwidth fluctuations and ii) thread
    priority management where you have no control whatsoever. The latter is
    caused by the fact that a script is just another process within the
    host process (browser). Periodically system will decide that it has
    something more important to do and your script can wait till another
    system tick (or two). On a long run it adds up significally.
    The only reliable counter would be from a flag "another packet sent"
    but you have none of such.

    Still no one is forbidden to invent perpetuum mobiles and I expect to
    see a lot of "perfectly working models" linked soon.
     
    VK, Jan 2, 2006
    #5
  6. JMB

    Sean Berry Guest

    "JMB" <> wrote in message
    news:i1%tf.33841$az4.26548@trndny03...
    > Hello,
    >
    > I was wondering if anyone knew of any projects extending the inline upload
    > progress bar to utilize an inpage image uploader with bar, without having
    > to refresh or go to a seperate page, nor opening a second box for display
    > of the progress bar.
    >
    > I had been using the MegaUpload that was adapted from Raditha's script at
    > http://www.raditha.com/upload.php . The MegaUpload script I have been
    > using takes the progress bar inpage, instead of opening a new page/box for
    > the display of the bar.
    >
    > Anyhow, here is what I have been wondering. I have projects with large
    > amounts of 'application' usage and Web2.0 features, I guess you can say.
    > This includes dynamic form submissions, inline graphical interfaces,
    > realtime chat, realtime drawing/whiteboarding, etc. The thing I have not
    > been able to nail is inline progress bar usage with uploaded images/files.
    > Does anyone know about any projects currently working on this very thing
    > that I could contribute to? Or, see some demo code to have a better idea.
    >
    > I am guessing my request to the server should be a put, but then I would
    > need what from there modifed to handle this? As PHP sux for it, and the
    > closest I have gotten is using a modified verision of OB_ functions. The
    > Perl solution seems more feasible, but I would not know where to begin
    > using the async nature of AJAX, I think sending the file via PUT method,
    > and the form variables(if any), submitted during this display of the bar
    > from a seperate thread to the server for storage.
    >
    > I know the uploader is PHP/Perl, but with it being an AJAX-savy script,
    > posting here was a starting point.
    >
    > Thanks in advance for direction to any/all projects, as well as
    > suggestions, help and comments.
    >


    JMB,

    The posts above are quite right about Base64 encoding and size fluctuations.
    However, in my case I had clients who wanted to have some sort of progress
    indicator. When someone has to upload a 50 magabyte file for instance they
    want to know that the upload is still in progress after 1 minute.

    My program works by using the CONTENT_LENGTH of the request as the total
    size. Then, using AJAX, it retrieves the current size of the file on the
    server and creates a progress bar based on the that.

    Is it 100% accurate? NO!

    Does it satisfy the customer by providing them with exactly what they
    wanted? YES!

    And does it actually work with a fair amount of accuracy? YES!
     
    Sean Berry, Jan 2, 2006
    #6
  7. JMB

    JMB Guest

    Hello, and thank you for all of the replies...

    However, in the applications I am currently using, I am doing an inpage
    upload progress that is about 90% accurate with time and rate of tranfer, so
    this is already covered. In fact, a 100k image to a 5mb to(we had reason
    to) a 1.8GB file, all displayed near-correct size, time-to-fin and percent
    done in all speed cases tried. You just needed a small modifier on readout,
    and an incremental for timeFromStart of the upload.

    What I am having problem with, is using the XMLHttpRequest object and
    asyncronous actions to allow for a file to be placed onto the server, have a
    filename callback/src designation to be placed back onto that page(I know
    how to do this, so, once again, this is not a problem), but all the while,
    maintain the ability of the app to display a progress bar while uploading is
    going on in the background(my bane with letting the progress bar do it's
    thing).

    I know that it can be done, but I hit a wall, and am not having any luck
    going around it...seems like my programming attempts on this are trying to
    push me through it!:)

    Thanks again, and I look forward to any more replies that the group may be
    willing to send,
    JMB

    "VK" <> wrote in message
    news:...
    >
    > Duncan Booth wrote:
    >> VK wrote:
    >>
    >> > The second crutial limitation of HTTP upload is that the data is sent
    >> > in BASE64 encoding. It means that the actual submission size is *always
    >> > bigger* than the summary of file sizes to upload. The actual growth
    >> > depends heavily on the binary structure of files to upload. Roughly it
    >> > is no less than +20% and up to +100%. This is why all endless attempts
    >> > to make progress bar based on file(s) size + bandwidth speed failed so
    >> > miserably.
    >> >

    >> This is rubbish.
    >>
    >> Base64 encoding always increases the size of the data by one third. It
    >> does
    >> not depend on the 'binary structure' of the data: quite simply 3 input
    >> bytes are encoded as 4 printable characters.

    >
    > My bad! I was thinking about zipping instead of base64-encoding. The
    > resulting file size is never less then source size + 30%
    > At the same time for MIME base64 it is never strictly equal to
    > binarySize+binarySize/100*30. Because of additional service data is
    > approx 33%.
    > On 5-10-more files package this "approx" transforms a strict
    > calculation into guessing game.
    > Also there is such things as i) bandwidth fluctuations and ii) thread
    > priority management where you have no control whatsoever. The latter is
    > caused by the fact that a script is just another process within the
    > host process (browser). Periodically system will decide that it has
    > something more important to do and your script can wait till another
    > system tick (or two). On a long run it adds up significally.
    > The only reliable counter would be from a flag "another packet sent"
    > but you have none of such.
    >
    > Still no one is forbidden to invent perpetuum mobiles and I expect to
    > see a lot of "perfectly working models" linked soon.
    >
     
    JMB, Jan 2, 2006
    #7
  8. JMB

    VK Guest

    Sean Berry wrote:
    > The posts above are quite right about Base64 encoding and size fluctuations.
    > However, in my case I had clients who wanted to have some sort of progress
    > indicator. When someone has to upload a 50 magabyte file for instance they
    > want to know that the upload is still in progress after 1 minute.


    50Mb over HTTP ?! That's at least 15.5MB of extra base64 trash on each
    upload! How about environment protection (not talking about wasted
    time) ? ;-)

    There are plenty of FTP modules to plug in into your page for such huge
    uploads.

    And on relatively small uploads the "oopsy miscalculation" can easily
    be up to 50% of the predicted time.

    But it is indeed possible (and useful) to have an animated "currently
    uploading" sign on the page. With a bit of trickery it can be done.
     
    VK, Jan 2, 2006
    #8
  9. JMB

    Sean Berry Guest

    "VK" <> wrote in message
    news:...
    >
    > Sean Berry wrote:
    >> The posts above are quite right about Base64 encoding and size
    >> fluctuations.
    >> However, in my case I had clients who wanted to have some sort of
    >> progress
    >> indicator. When someone has to upload a 50 magabyte file for instance
    >> they
    >> want to know that the upload is still in progress after 1 minute.

    >
    > 50Mb over HTTP ?! That's at least 15.5MB of extra base64 trash on each
    > upload! How about environment protection (not talking about wasted
    > time) ? ;-)
    >


    You are preaching to the choir with this argument.

    The problem is end-users. Our clients for the most part do not deal with
    (nor are they themselves) the smartest people on the planet. Heck, most of
    them do not have what I would consider the basic aptitude needed for
    survival... but they prove me wrong every day.

    > There are plenty of FTP modules to plug in into your page for such huge
    > uploads.
    >


    In my particular case the upload area is a bit more complex and has users
    with different access and abilities. It also needed to have added
    functionalites like email notifications for new uploads, description
    requirements and a whole bunch of other crap. But, trying to teach my
    clients to teach their clients how to successfully upload a file via FTP
    wouldn't have been my first choice anyway.
    > And on relatively small uploads the "oopsy miscalculation" can easily
    > be up to 50% of the predicted time.
    >
    > But it is indeed possible (and useful) to have an animated "currently
    > uploading" sign on the page. With a bit of trickery it can be done.
    >


    As far as the progress bar, "people" like to have something really fun and
    exciting like a progress bar that tells them that something is going on. At
    first, I used an animation once the upload started by changing a static
    image to an animated gif or something. However, I found that most "people"
    didn't understand that when they are uploading a 50 megabyte file it may
    take more than ten seconds and became frustrated thinking it "locked up
    again". Anyway, what really matters - in my case anyway - is that the end
    product does exactly what they, the brilliant people I get to deal with,
    want it to do. It gives a visual indication through a recognizable and very
    familiar way of how much progress has been made.

    SMB
     
    Sean Berry, Jan 3, 2006
    #9
  10. JMB

    VK Guest

    Sean Berry wrote:
    > You are preaching to the choir with this argument.
    >
    > The problem is end-users. Our clients for the most part do not deal with
    > (nor are they themselves) the smartest people on the planet. Heck, most of
    > them do not have what I would consider the basic aptitude needed for
    > survival... but they prove me wrong every day.


    The problem of "sub-zero" users is well known to me either :)
    <http://groups.google.com/group/comp.lang.javascript/browse_frm/thread/355b3f86ed2f11fa/17aa578042f08022>

    But it is a wrong thinking IMHO to correlate "primitive user" with
    "primitive solution". Mostly it's just opposite: a very "primitive
    user" requires a very sophisticated solution - but with a very simple
    interface covering all internal complexity. In the above linked case I
    had to end up by designing a whole new set of pseudo-inputs paired with
    <input type="hidden"... for submission. On a normal run I would try to
    avoid such pervertion. :)

    HTTP binary exchange is not a tool to replace FTP. It is merely a
    convenience addon to a form for relatively small attachments. Besides
    the file growth because of base64 format, it has a bounch of other
    implications like buffer mechanics. See for example
    <http://support.microsoft.com/default.aspx?scid=kb;en-us;329781> I
    don't think that we can talk about usability improvement if
    upload/download takes 1hr instead of needed 10 min - but user gets an
    animation for the entire hour :)

    There are FTP plugins like say Internet Transfer control from
    Microsoft. One can use them to have FTP and emulate as close as needed
    (if needed) the usual <input type="file"... interface.

    But everyone is the king of his own solution - so it's all just
    IMHighlyHO.
     
    VK, Jan 3, 2006
    #10
  11. JMB

    VK Guest

    PJ wrote:
    > I have been using a component from http://www.abcupload.com/ for years.
    > After creating a unique id that is appended to the action attribute on the
    > form that does the post for the upload, I can then use this id in get
    > requests to get informatation on the progress of the upload. I perform
    > these get requests with XHR requests on the client side and handle the
    > requests on the server side, where I send the data on the progress of the
    > upload back in xml. The javascript updates an html progress bar, and byte
    > count / total bytes of the upload.
    >
    > This "approx" "working model" handles 80 Gig ( yes, GIG ) uploads as
    > accurately as my eye is able to discern.
    >
    > I love people that have looked into a solution, failed to find one, and thus
    > determine their problem is "impossible" to solve.


    That is a subject substitution. I did not say that there were not ways
    to have a status report for upload/download operations in browser. You
    see such report in action every time you're downloading something from
    the Web.

    The problem is not solvable by using only default browser tools (HTML
    elements and scripting) and server responces. If there is a need to
    have status report at any price, you can do it by extending
    browser/server core functionality. And you can do it without privacy
    exposure like in the mentioned case. There are plenty of FTP Java
    applets and ActiveX controls to embed onto your page.

    P.S. 80 Gigs over FTP I hope? (I did not read abcupload.com proposal in
    details).
    Otherwise it would be an athem to uneffectiveness.
     
    VK, Jan 4, 2006
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Darko

    Problem with inpage link

    Darko, Jul 30, 2007, in forum: HTML
    Replies:
    9
    Views:
    461
    Information Man
    Jul 31, 2007
  2. Replies:
    0
    Views:
    293
  3. Replies:
    8
    Views:
    2,300
  4. Kal
    Replies:
    10
    Views:
    618
    David Wang
    Sep 17, 2008
  5. Sven Wolf
    Replies:
    7
    Views:
    211
    Tad McClellan
    Dec 3, 2004
Loading...

Share This Page