reading text-file with very long lines

Discussion in 'Java' started by McGregor, Jan 29, 2009.

  1. McGregor

    McGregor Guest

    Hello!

    I'm trying to read a file line by line with following code-snippet:

    -----------------------------**-------------------------------
    BufferedReader test = new BufferedReader(new FileReader(datei));
    String tmp = "";
    Vector data = new Vector();
    boolean first = true;
    int breaker = 0;
    int position = 0;
    while ((tmp = test.readLine()) != null) {
    if (!first) data.addElement(tmp); else first = false;
    if (tmp.contains("%%EOF")) breaker = position;
    position++;
    }
    while (data.size() > breaker) data.remove(data.size()-1);
    -----------------------------**-------------------------------

    there are many "%%EOF" - lines within this file. I need to get the
    last one and this is my solution to get the whole data from the second
    line to the last line with "%%EOF"

    But I encountered some problems. Some lines are very long, i.e. more
    than 8000 characters (varied, depends on the file). When reading the
    file using the above code I only get aprox. 1520 characters and the
    remaing characters are lost and reading stops with no error at all.
    So I need some other solution to read my file. Keep in mind, that I
    need to compare strings...

    Does anyone know any (nice) solution?
    Thanks for your help.
    McGregor
    McGregor, Jan 29, 2009
    #1
    1. Advertising

  2. McGregor

    Stefan Ram Guest

    McGregor <> writes:
    >file using the above code I only get aprox. 1520 characters and the


    If replying to my post, please do not quote my complete
    post, but only those parts you directly refer to.
    Especially, please do not quote the complete source code.

    public class Main
    {
    static final int NUMBER_OF_LINES = 100;
    static final int MAXIMUM_LINE_LENGTH = 64000;
    static final int MINIMUM_LINE_LENGTH = 1521;

    public static void write( final java.lang.String[] args )
    throws java.lang.Throwable
    {
    final java.io.PrintWriter out = new java.io.PrintWriter
    ( new java.io.FileOutputStream( "tmp.txt" ), true );
    for( int l = 0; l < NUMBER_OF_LINES; ++l )
    { java.lang.System.err.println( "writing line " + l );
    out.print( l );
    out.print( ' ' );
    if( java.lang.Math.random() < 0.1 )out.print( "%%EOF" );
    final int length =MINIMUM_LINE_LENGTH +
    ( int )
    ( java.lang.Math.random() *
    ( 1 + MAXIMUM_LINE_LENGTH - MINIMUM_LINE_LENGTH ));
    for( int p = 0; p < length; ++p )out.print( "x" );
    out.print( "\n" ); }}

    public static void read( final java.lang.String[] args )
    throws java.lang.Throwable
    {
    final java.util.Scanner scanner = new java.util.Scanner
    ( new java.io.File( "tmp.txt" ));
    java.lang.String result = null;
    int l = 0;
    while( scanner.hasNextLine() )
    { java.lang.System.err.println( "reading line " + l++ );
    final java.lang.String line = scanner.nextLine();
    if( line.contains( "%%EOF" ))result = line; }
    if( result != null )
    { java.lang.System.out.printf( "%.78s%n", result );
    java.lang.System.out.println( result.length() ); }
    else
    { java.lang.System.out.println( result ); }}

    public static void main( final java.lang.String[] args )
    throws java.lang.Throwable
    {
    write( args );
    read( args ); }}

    94 %%EOFxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxxx
    9877
    Stefan Ram, Jan 29, 2009
    #2
    1. Advertising

  3. McGregor

    Tom Anderson Guest

    On Thu, 29 Jan 2009, McGregor wrote:

    > I'm trying to read a file line by line with following code-snippet:
    >
    > -----------------------------**-------------------------------
    > BufferedReader test = new BufferedReader(new FileReader(datei));
    > String tmp = "";
    > Vector data = new Vector();
    > boolean first = true;
    > int breaker = 0;
    > int position = 0;
    > while ((tmp = test.readLine()) != null) {
    > if (!first) data.addElement(tmp); else first = false;
    > if (tmp.contains("%%EOF")) breaker = position;
    > position++;
    > }
    > while (data.size() > breaker) data.remove(data.size()-1);
    > -----------------------------**-------------------------------
    >
    > there are many "%%EOF" - lines within this file. I need to get the
    > last one and this is my solution to get the whole data from the second
    > line to the last line with "%%EOF"
    >
    > But I encountered some problems. Some lines are very long, i.e. more
    > than 8000 characters (varied, depends on the file). When reading the
    > file using the above code I only get aprox. 1520 characters and the
    > remaing characters are lost and reading stops with no error at all.


    Odd. Could you try running this test program:

    http://urchin.earth.li/~twic/tmp/LongLines.java

    ?

    If there are any problems with reading lines up to 8000 characters long,
    it should fail with an IOException. On my machine (java 1.5.0_16), it
    completes successfully.

    The thing is, your code looks like it should work to me. It's not quite
    how i would have written it, but it ought to work.

    > So I need some other solution to read my file. Keep in mind, that I
    > need to compare strings...


    Here's how i'd do it:

    public static List<String> readToLastMarker(BufferedReader in, String marker) throws IOException {
    List<String> acceptedLines = new ArrayList<String>();
    List<String> candidateLines = new ArrayList<String>();
    String firstLine = in.readLine(); // ignore the first line
    String line;
    while ((line = in.readLine()) != null) {
    if (line.contains(marker)) {
    acceptedLines.addAll(candidateLines);
    candidateLines.clear();
    candidateLines.add(line); // do you really want to do this?
    }
    else {
    candidateLines.add(line);
    }
    }
    return acceptedLines;
    }

    tom

    --
    I am the best at what i do.
    Tom Anderson, Jan 29, 2009
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. shanx__=|;-

    very very very long integer

    shanx__=|;-, Oct 16, 2004, in forum: C Programming
    Replies:
    19
    Views:
    1,593
    Merrill & Michele
    Oct 19, 2004
  2. Abhishek Jha

    very very very long integer

    Abhishek Jha, Oct 16, 2004, in forum: C Programming
    Replies:
    4
    Views:
    409
    jacob navia
    Oct 17, 2004
  3. Vlad Dogaru

    Reading long lines from a file

    Vlad Dogaru, Aug 14, 2007, in forum: C Programming
    Replies:
    7
    Views:
    314
    Spiros Bousbouras
    Aug 20, 2007
  4. ShaunJ

    m// on very long lines leaks memory

    ShaunJ, Mar 13, 2008, in forum: Perl Misc
    Replies:
    5
    Views:
    105
    Uri Guttman
    Mar 13, 2008
  5. gry
    Replies:
    15
    Views:
    136
    Chris Angelico
    Apr 11, 2013
Loading...

Share This Page