cgi upload of very large files (>2GB)

Discussion in 'Perl Misc' started by darksaga, Apr 25, 2006.

  1. darksaga

    darksaga Guest

    hi ppl,

    i have to upload very big files (could be bigger 2gb) web-based via
    http (input type="file") to our intranet tomcat server. Therefore i use
    following cgi script
    (http://www.freeware-base.de/freeware-base-artikel11663.html) which i
    modified to our needs. It indicates upload speed, time left etc. (see
    link). Everything works pretty fine with files up to 2GB, speed is very
    nice (transfer of 1GB in around 1minute), but i have problems to upload
    files bigger than 2GB.
    Some files i need to upload are 3-4gb (sequence databases).
    If you click the browse butten to select such a file (bigger than 2GB),
    file size is shown in the popped up filechooser as "0" and you are not
    able to select it.
    Biggest file i could select was (2140686574Bytes = 1,99GB), but some i
    need to upload, as said above, are ~4000000000Byte=4GB.
    I think the problem is not the cgi script, its the html form input
    type="file".

    Anybody knows if there's a maximum file size it can handle and if there
    is a workaround?

    Help would be very appriciated

    greets darksaga
     
    darksaga, Apr 25, 2006
    #1
    1. Advertising

  2. "darksaga" <> wrote in news:1145952614.140124.210460
    @u72g2000cwu.googlegroups.com:

    > hi ppl,


    Please avoid childish spelling.

    > i have to upload very big files (could be bigger 2gb) web-based via
    > http (input type="file") to our intranet tomcat server. Therefore i use
    > following cgi script
    > (http://www.freeware-base.de/freeware-base-artikel11663.html) which i


    I guess it is not as bad as it could be but it is close.


    > modified to our needs. It indicates upload speed, time left etc. (see
    > link). Everything works pretty fine with files up to 2GB, speed is very
    > nice (transfer of 1GB in around 1minute), but i have problems to upload
    > files bigger than 2GB.


    You are hitting either a perl limit or OS limit.

    1. Does the server have the ability to create and access files larger than
    2GB?

    2. Does perl -V indicate:

    uselargefiles=define

    Sinan
     
    A. Sinan Unur, Apr 25, 2006
    #2
    1. Advertising

  3. darksaga

    darksaga Guest

    have to check it tomorrow @ work, if uselargefiles is defined

    do I have any other possibilities to upload such big files to the
    server, maybe ftp? i'm not that familiar with perl&cgi (i'm new in perl
    programming).

    the only problem i see: everything has to be done from a html page. The
    method i'm using now, is imo pretty nice & user friendly, and works
    very good for files < 2 GB.

    greets

    darksaga
     
    darksaga, Apr 25, 2006
    #3
  4. darksaga

    Guest

    darksaga wrote:
    > do I have any other possibilities to upload such big files to the
    > server, maybe ftp?


    If the operating system (or, really, the filesystem) won't let you
    create a file larger than 2 GB then there is no possible way to create
    a file larger than 2 GB. It does not matter HOW you try to create it.

    I happen to know that, on IBM AIX, the "default" journaled file system
    has a 2 GB filesize limitation. On reasonably modern versions of AIX,
    it is possible to create a JFS filesystem that is "Large File Enabled"
    (and does not have this limitation) but this must be done when the
    filesystem is initially created.

    --
    http://DavidFilmer.com
     
    , Apr 25, 2006
    #4
  5. "darksaga" <> wrote in
    news::

    > do I have any other possibilities to upload such big files to the
    > server, maybe ftp?


    That's not going to help if the OS does not allow.

    > i'm not that familiar with perl


    One idea might be to save the uploaded data in 1GB sized chunks.

    > &cgi


    Perl is not CGI. CGI is not Perl.

    > (i'm new in perl programming).


    Please read the posting guidelines and the FAQ.

    > the only problem i see: everything has to be done from a html page.


    You might want to try the CGI group:

    comp.infosystems.www.authoring.cgi

    Sinan
    --
    A. Sinan Unur <>
    (remove .invalid and reverse each component for email address)

    comp.lang.perl.misc guidelines on the WWW:
    http://augustmail.com/~tadmc/clpmisc/clpmisc_guidelines.html
     
    A. Sinan Unur, Apr 26, 2006
    #5
  6. darksaga

    darksaga Guest

    darksaga, Apr 26, 2006
    #6
  7. darksaga

    Guest

    "darksaga" <> wrote:
    > hi ppl,
    >
    > i have to upload very big files (could be bigger 2gb) web-based via
    > http (input type="file") to our intranet tomcat server. Therefore i use
    > following cgi script
    > (http://www.freeware-base.de/freeware-base-artikel11663.html) which i
    > modified to our needs. It indicates upload speed, time left etc. (see
    > link). Everything works pretty fine with files up to 2GB, speed is very
    > nice (transfer of 1GB in around 1minute), but i have problems to upload
    > files bigger than 2GB.
    > Some files i need to upload are 3-4gb (sequence databases).
    > If you click the browse butten to select such a file (bigger than 2GB),
    > file size is shown in the popped up filechooser as "0" and you are not
    > able to select it.


    The file chooser is part of your web browser. This seems to be a
    limitation of your web browser, and is probably unrelated to Perl or the
    download program you linked to. (Not there could not also be a problem
    there, but you are not getting far enough in the process to activate such a
    problem, if any, in that part.)

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
     
    , Apr 26, 2006
    #7
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    14
    Views:
    4,740
    JIMCO Software
    Nov 11, 2005
  2. =?Utf-8?B?U2FuZHk=?=

    upload very large files

    =?Utf-8?B?U2FuZHk=?=, Nov 2, 2006, in forum: ASP .Net
    Replies:
    2
    Views:
    425
    =?Utf-8?B?U2FuZHk=?=
    Nov 3, 2006
  3. Rolf Schroedter

    Accessing large >2GB file succeeds fails with open/read

    Rolf Schroedter, Feb 15, 2005, in forum: C Programming
    Replies:
    6
    Views:
    549
    Olof Lagerkvist
    Feb 16, 2005
  4. Alona

    stat() fails for large file (2GB)

    Alona, Mar 14, 2008, in forum: C Programming
    Replies:
    5
    Views:
    1,233
    Ian Collins
    Mar 15, 2008
  5. Ams Lo

    Transpose a large file(>2GB)

    Ams Lo, Apr 27, 2008, in forum: Ruby
    Replies:
    17
    Views:
    268
    Ken Bloom
    Apr 30, 2008
Loading...

Share This Page