HELP: Joining 5 large text files

Discussion in 'Perl' started by Stuart, Aug 19, 2004.

  1. Stuart

    Stuart Guest

    Hi,

    Please can anyone help me join 5 large (1.8gb) text files togeather
    to create 1 very large file.

    I have some code in PHP but it bombs out at 2gb (seems there is a limit
    and php needs re compiling, something thats not an option for me)

    I don't want to take up all the servers resources so something that uses
    little memory would be very good indeed!

    here's the php code if it help...

    <?php
    set_time_limit(14400000);

    $file[0] = "file1.txt";
    $file[1] = "file2.txt";
    $file[2] = "file3.txt";
    $file[3] = "file4.txt";
    $file[4] = "file5.txt";

    $target = "output.txt";
    $handle = fopen($target, 'a');

    foreach ($file as $var) {
    $c_handle = fopen($var, 'r');
    do {
    $content = fread($c_handle,1000000);
    fwrite($handle, $content);
    } while (!empty($content));
    fclose($c_handle);
    }
    ?>


    all the best

    Stu
     
    Stuart, Aug 19, 2004
    #1
    1. Advertising

  2. Stuart

    Guest

    (Stuart) wrote in message news:<>...

    > I have some code in PHP but it bombs out at 2gb (seems there is a limit
    > and php needs re compiling, something thats not an option for me)


    2gigs is a magic number for the maximum file size on some OSes; 2^31.
    (You loose a bit due to signed variable usage on some file systems.)

    What OS, Architecture are you using? What type of file system i.e.
    XFS, EXT2, WINDoze

    Cheers!
     
    , Aug 19, 2004
    #2
    1. Advertising

  3. Stuart

    Guest

    (Stuart) wrote in message news:<>...
    > Newsgroup: comp.lang.perl


    > I have some code in PHP


    Perhaps you should try a newsgroup that:

    1) Exists (this one doesn't)
    2) Relates to PHP (this one wouldn't, if it existed)
     
    , Aug 19, 2004
    #3
  4. Stuart

    Joe Smith Guest

    Stuart wrote:

    > create 1 very large file.
    >
    > I have some code in PHP but it bombs out at 2gb (seems there is a limit
    > and php needs re compiling, something thats not an option for me)


    You're screwed. Unless PHP is recompiled with LARGE_FILE is enabled,
    you will *not* be able to write anything past byte number 2147483647.
    -Joe
     
    Joe Smith, Aug 19, 2004
    #4
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. JHenstay
    Replies:
    3
    Views:
    564
    osmium
    Jan 11, 2004
  2. Azalar ---
    Replies:
    1
    Views:
    146
    Rob Biedenharn
    Mar 28, 2010
  3. Stuart

    Help: joining 5 LARGE text files

    Stuart, Aug 19, 2004, in forum: Perl Misc
    Replies:
    7
    Views:
    124
    Joe Smith
    Sep 16, 2004
  4. Miki Tebeka
    Replies:
    0
    Views:
    172
    Miki Tebeka
    Dec 5, 2012
  5. Ian Kelly
    Replies:
    0
    Views:
    161
    Ian Kelly
    Dec 5, 2012
Loading...

Share This Page