J2ME : fastest way to read lines from text resource into a string array ?

[Sam Iam]

I read text from a resource in my JAR into one big string then use a
split function of my own devising to seperate it into lines of a
string array based on \n delimiters. This method seems relatively
slow.

Is there a faster way ?

Due to suggestions from the group I sped up some numeric parsing using
DataInputStream but alas even DataInputStream's deprecated readline
method isn't in J2ME.

- Sam.

 

[Darryl L. Pierce]

I read text from a resource in my JAR into one big string then use a
split function of my own devising to seperate it into lines of a
string array based on \n delimiters. This method seems relatively
slow.

Is there a faster way ?

Due to suggestions from the group I sped up some numeric parsing using
DataInputStream but alas even DataInputStream's deprecated readline
method isn't in J2ME.

Try readUTF() and writeUTF().

 

[Roedy Green]

I read text from a resource in my JAR into one big string then use a
split function of my own devising to seperate it into lines of a
string array based on \n delimiters. This method seems relatively
slow.

Do the I/O in one fell swoop unbuffered. See
http://mindprod.com/fileio.html

I do this all the time and it is very fast.

I hope you are not reading it C-style char by char with an unbuffered
reader. THAT would be slow.

Use indexOf to find the \n and substring to extract the line.
Finallyi compile with Jet. See http://mindprod.com/jgloss/jet.html

The alternative for very large files is readline using a
BufferedReader with a whacking huge buffer. For details see
http://mindprod.com/fileio.html


See http://mindprod.com/jgloss/buffer.html

for hints on selecting optimum buffer sizes.

 

[Roedy Green]

Try readUTF() and writeUTF().

IIRC those are a binary format with a leading length on the string.
You must have created the file with Java.

To deal with 16 bit characters. Most of the methods deal with 8-bit
local character sets and translate.

Once you have the array parsed, you can save it as an array of strings
with ObjectOutputStream. If you read that back with a decent buffer
size it should be pretty quick. It does not have to scan because
there are length codes.

see http://mindprod.com/fileio.html
for how to read/write ObjectStreams.

 

[Darryl L. Pierce]

IIRC those are a binary format with a leading length on the string.
You must have created the file with Java.

Since he's creating the data before putting it into his JAR, he can easily
create the file with Java. I used the same solution in my last project to
put default i18n data into the MIDlet suite, and we supported several
non-Latin character sets in this manner.

 

[Roedy Green]

I read text from a resource in my JAR into one big string then use a
split function of my own devising to seperate it into lines of a
string array based on \n delimiters. This method seems relatively
slow.

In the jar it is in compressed form. You could store the elt in
uncompressed form, with a winzip tweak. It would fish out the jar
faster, but the jar would take longer to download.

 

[Roedy Green]

Since he's creating the data before putting it into his JAR, he can easily
create the file with Java. I used the same solution in my last project to
put default i18n data into the MIDlet suite, and we supported several
non-Latin character sets in this manner.

For that sort of thing, I just use ObjectStreams (which internally
uses writeUTF). There is then just a single line of code in the
Applet to restore the entire universe of data. It always works
perfectly no matter how complicated the data.

That's how the CurrCon currency converter app on my website works. See
http://mindprod.com/products.html#CURRCON and also the navigator if
you turn on frames that lets you jump anywhere in the glossary
directly. I also use that in the Replicator. see
http://mindprod.com/products.html#REPLICATOR. The manifest.ser file
is a serialised copy of everything your client needs to know.

This keeps the Applet or JWS client small since you have no baggage in
there to deal with parsing or even individual fields.

For sample code on how to do that see http://mindprod.com/fileio.html

Serialisation was pretty rocky at first. Now we know where the rocks
are and it has been fixed up. Look into it. It saves a TON of boring
hard to maintain code.

 

[Darryl L. Pierce]

For that sort of thing, I just use ObjectStreams (which internally
uses writeUTF).

Object serialization is not available in CLDC/MIDP.

There is then just a single line of code in the
Applet to restore the entire universe of data. It always works
perfectly no matter how complicated the data.

If only we were talking about Applets. MIDlets are a different beast
entirely. <g>

<snip>

 

[Sam Iam]

Do the I/O in one fell swoop unbuffered. See
http://mindprod.com/fileio.html

I do this all the time and it is very fast.

I hope you are not reading it C-style char by char with an unbuffered
reader. THAT would be slow.

Use indexOf to find the \n and substring to extract the line.
Finallyi compile with Jet. See http://mindprod.com/jgloss/jet.html

The alternative for very large files is readline using a
BufferedReader with a whacking huge buffer. For details see
http://mindprod.com/fileio.html


See http://mindprod.com/jgloss/buffer.html

for hints on selecting optimum buffer sizes.

Thanks to you & Darryl for all your help.

In Java I try to avoid doing any loops of single byte/char operations.
Below is some code (not mine) that's similar my current way of loading
data from my JAR.

InputStream.available() happens to return a # that is the size of the
entire resource on the phones I'm currently working with but I'm
worried that may not always be the case.

Is there a way of getting the overall resource size from an
InputStream() or a JAR resource ?

On the J2ME PC emulator I get back 0 from the InputStream.available()
method so I also have to include code (not included below) to do
looped single byte/char reads.

There seems to be a lot of J2ME sample code with single byte/char
operation loops when there should be faster ways.

I also wonder if it's faster to read in one big chunk & then loop to
do parsing then to loop with small reads & parse at the same time.

- Sam.

private byte[] getPic()
{
InputStream iStream = getClass().getResourceAsStream("mypic.jpg");

byte imageBytes[] = new byte[iStream.available()];
iStream.read(imageBytes);
return imageBytes;
}

 

[Darryl L. Pierce]

In Java I try to avoid doing any loops of single byte/char operations.
Below is some code (not mine) that's similar my current way of loading
data from my JAR.

InputStream.available() happens to return a # that is the size of the
entire resource on the phones I'm currently working with but I'm
worried that may not always be the case.

Is there a way of getting the overall resource size from an
InputStream() or a JAR resource ?

It depends on the implementation. The InputStream.available() returns only
the bytes available in the current buffer (and not the full size of the
resource). If you're creating the data, however, you can put at the
beginning a watermark that indicates the number of bytes or whatever is to
follow.

On the J2ME PC emulator I get back 0 from the InputStream.available()
method so I also have to include code (not included below) to do
looped single byte/char reads.

There seems to be a lot of J2ME sample code with single byte/char
operation loops when there should be faster ways.

That's for blind reading of the input data. What you can do is this if you
don't know the total size and don't want to read a single byte at a time:

InputStream input = // whatever;
byte[] buffer = new byte[4096];
boolean done = false;

while(!done)
{
int bytesread = input.read(buffer);

if(bytesread == -1) // we've reached the end of the stream
{
done = true;
}
else
{
// process the data just read in
}
}