StringTokenizer ignores tokens without content

[Per Magnus L?vold]

Hi,
I'm working with a Java application that parses a semi-kolon (
separated file.
For the parsing I use StringTokenizer to cut each row into tokens. But
when a token has no content, the StringTokenizer ignores the token(!).
When I try to structure the parsed content, I now miss the "columns"
with no content, and the table I am trying to create becomes corrupt.

Example:
myFile:
***start***
a0;b0;c0;d0;e0;f0;
a1;b1;c1;;e1;f1;
a2;;c2;d2;e2;;
***end***

I'd rather want each of these rows to have 6 tokens (inkluding blank
tokens).
Row 1 has 6 tokens
Row 2 has 5 tokens (the third token is ignored)
Row 3 has 4 tokens (2nd and 6th are ignored)

When I try to make a table out of this, e1 appears in the "d" columnm,
f1 appears in the "e" column etc.

Can anybody help me solve this problem?

Regards, Per Magnus Løvold

 

[Tor Iver Wilhelmsen]

For the parsing I use StringTokenizer to cut each row into tokens. But
when a token has no content, the StringTokenizer ignores the token(!).

Yes, as documented.

Can anybody help me solve this problem?

Ues the constructor StringTokenizer(String, delims, true) to have it
return delimiters, and count those.

But for newer JREs, use String.split() or other variants on regular
expressions instead.

 

[Jacob]

I'm working with a Java application that parses a semi-kolon (
separated file.
For the parsing I use StringTokenizer to cut each row into tokens. But
when a token has no content, the StringTokenizer ignores the token(!).

See

http://geosoft.no/software/index.html#tokenizer

for a string tokenizer that handles empty tokens.

 

[Real Gagnon]

For the parsing I use StringTokenizer to cut each row into tokens. But

when a token has no content, the StringTokenizer ignores the token(!).
When I try to structure the parsed content, I now miss the "columns"
with no content, and the table I am trying to create becomes corrupt.

See http://www.rgagnon.com/javadetails/java-0015.html
fro an alternate StringTokenizer.

Bye.

 

[Per Magnus L?vold]

Thank you, all!
I finally ended parsing the input strings manually. Didn't require too
much code, and I felt like having more control over the results. ;-)

I used:

myToken1 = stringToBeParsed.substring(0,
stringToBeParsed.indexOf(';'));
stringToBeParsed = stringToBeParsed.substring(stringToBeParsed.indexOf(';')+1);

myToken2 = stringToBeParsed.substring(0,
stringToBeParsed.indexOf(';'));
stringToBeParsed = stringToBeParsed.substring(stringToBeParsed.indexOf(';')+1);
..
..
..
etc.
Works fine, and easy to handle special tokens. Ex. when the third
token has different delimiter (or X number of characters).

Thanks!
-P