how to reach end of file?

[ruds]

Hello,
I'm reading a file which is sbout 3.5 GB in size. And extracting some
fields from the file.
For smaller file the program works fine but for this file , it is not
reaching the end of file and exits.
How should I force the program to reach end of file?

To read the file I'm using:
while((str=br.readLine())!=null)
{
//set of operations
}

My sample code is:

public void getLine(String file) throws IOException
{
fr=new FileReader(file);
br=new BufferedReader(fr);
no=0;
i=0;
j=0;
while((str=br.readLine())!=null)
{
no++;

if(str.startsWith("$SUBCASE"))
{
st1= new StringTokenizer(str," ");
while(st1.hasMoreTokens())
{
tok1=st1.nextToken();
if(tok1.equals("="))
{
tok2=st1.nextToken();
if(tok2.equals(id))
continue;
else
{
id=tok2;
subcase[j]=Integer.parseInt(id);
line=no-5;
System.out.println("Subcase="+subcase[j]);
i++;
j++;

}
}
}
}
else
continue;
}
System.out.println("end of getLine");
fr.close();

}

Please help ! Its urgent.

 

[Andrew Thompson]

I'm reading a file which is sbout 3.5 GB in size. And extracting some
fields from the file.
For smaller file the program works fine but for this file , it is not
reaching the end of file and exits.

And what exactly is the error output?
(* Or would you like us to simply guess?)

How should I force the program to reach end of file?

Hit it with a stick?

* Maybe it is just lazy.

To read the file I'm using: ...
My sample code is:

I term that a sample 'snippet', and always
recommend (as opposed to code snippets)
compilable, runnable code.

...

Please help ! Its urgent.

Pay me, then I might care about your schedule.

Andrew T.

 

[ruds]

It does not give any error...
It just exits the program before printing "end of getLine".

 

[Andrew Thompson]

It does not give any error...

I am surprised.

It just exits the program before printing "end of getLine".

...hmm. Can you supply an SSCCE that does the
same*?
<http://www.physci.org/codes/sscce.html>

I strongly suspect this is an OutOfMemoryError
(this 3.5 Gb file goes somewhat beyond the standard
64 Meg. assigned to a regular Java app.), but
your stated results do not support that (no OOME
stacktrace), perhaps I am mistaken.

* it would be very handy if it could also
generate a File of approx. 3.5 Gig, before
attempting to read that file, and failing.

Andrew T.

 

[Patricia Shanahan]

I am surprised.


..hmm. Can you supply an SSCCE that does the
same*?
<http://www.physci.org/codes/sscce.html>

I strongly suspect this is an OutOfMemoryError
(this 3.5 Gb file goes somewhat beyond the standard
64 Meg. assigned to a regular Java app.), but
your stated results do not support that (no OOME
stacktrace), perhaps I am mistaken.

* it would be very handy if it could also
generate a File of approx. 3.5 Gig, before
attempting to read that file, and failing.

One of my Java programs reads a 6,933,389,378 byte file. I can tell it
reads the whole file, because it generates two output files that each
have one line per line of the input, and the line counts match.

Here are a couple of possibilities:

1. A logic error in the program that is triggered by some condition that
exists in the full file but not in shorter excerpts.

2. A catch statement somewhere in the program that is concealing a
condition such as running out of heap space.

Patricia

 

[=?ISO-8859-1?Q?Arne_Vajh=F8j?=]

I strongly suspect this is an OutOfMemoryError
(this 3.5 Gb file goes somewhat beyond the standard
64 Meg. assigned to a regular Java app.), but
your stated results do not support that (no OOME
stacktrace), perhaps I am mistaken.

A while loop with a BufferedReader readLine should
not read the entire file into memory. Going through
a 3.5 GB file with just 64 MB of heap space should
not be a problem.

Arne

 

[Nigel Wade]

It does not give any error...
It just exits the program before printing "end of getLine".

What do you do with IOException?

Does the JVM/environment in which you are operating support files of this size?
Some 32bit environments are limited to 2GB files.

 

[su_dang]

What do you do with IOException?

Does the JVM/environment in which you are operating support files of this size?
Some 32bit environments are limited to 2GB files.

--
Nigel Wade, System Administrator, Space Plasma Physics Group,
University of Leicester, Leicester, LE1 7RH, UK
E-mail : (e-mail address removed)
Phone : +44 (0)116 2523548, Fax : +44 (0)116 2523555

Not sure if this is a problem or not, but you might have an overflow
problem. The maximum value of an int is 2,147,483,647, but your file
is a lot bigger than that. If you are using int to count something in
your program, try to change it to long instead.

 

[ruds]

I'm executing the program on AIX OS which as per my information does
support huge files(larger than 2GB)..
Please tell me, How do i detect what actual problem is there out of
all suggested ones ???