Byte array

[Dirk Bruere at NeoPax]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast each
one individually?

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff

 

[markspace]

Is there a representation I can use such that I do not need to cast each
one individually?

Well, you can use -121 instead of 0x87, but I don't think that's what
you are after.

Why not just use char or int as the type instead of byte? Can you just
deal with a different type programmatically?

 

[Mayeul]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Goodness! Unless those hex bytes are already comma-separated, this will
be a pain to copy and paste anyway. I'd probably rather write a short
program that copies these bytes in a Java source file.

Actually you'll probably want to consider the option to just read those
bytes from your file at runtime, to avoid data duplication and be sure
your program and your data stay in sync.

Is there a representation I can use such that I do not need to cast each
one individually?

I couldn't think of anything, but you might want to try writing a simple
makeBytesArray(int[]) private static method that would enable you to write :

public static byte[] default0 = makeBytesArray(int[] {
0x87, 0x3f, ...
});

 

[Dirk Bruere at NeoPax]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast each
one individually?

Thanks for the replies.
The original text was a Windows .reg file, so it's got to be a cut and
paste job. Good news is that its all plain text comma separated.

One other question - any way I can get the array to see the text input
as hex rather than decimal by default? Otherwise I have to add "0x" to
each entry. Not a vast deal given my text editor, but I'd rather not.

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff

 

[markspace]

Actually you'll probably want to consider the option to just read those
bytes from your file at runtime, to avoid data duplication and be sure
your program and your data stay in sync.

Oh, I missed that part. Yeah 20k bytes is going to make an unreasonable
source file, to say the least.

I agree with Mayeul here: process the text file into bytes with a short
Java program. Read the bytes directly at runtime from a resource file.

 

[Dirk Bruere at NeoPax]

Oh, I missed that part. Yeah 20k bytes is going to make an unreasonable
source file, to say the least.

I agree with Mayeul here: process the text file into bytes with a short
Java program. Read the bytes directly at runtime from a resource file.

It's going to be just as hard to put them in a file and read it as hard
coding them in from the start. I'll make an int array and then convert
to bytes as I read from it.

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff

 

[Mayeul]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast
each one individually?

Thanks for the replies.
The original text was a Windows .reg file, so it's got to be a cut and
paste job. Good news is that its all plain text comma separated.

Well it doesn't *have to*, this only complicates automatic reading.
Keeping your data in sync might become a pain over time otherwise.

One other question - any way I can get the array to see the text input
as hex rather than decimal by default? Otherwise I have to add "0x" to
each entry. Not a vast deal given my text editor, but I'd rather not.

None that I can think of.

 

[Mayeul]

It's going to be just as hard to put them in a file and read it as hard
coding them in from the start.

Actually it will probably be harder, but it will enable you to keep this
data out of compiled form and easily check and replace it in any
deployment phase.

This will also ease keeping your data in sync.

 

[Dirk Bruere at NeoPax]

Actually it will probably be harder, but it will enable you to keep this
data out of compiled form and easily check and replace it in any
deployment phase.

This will also ease keeping your data in sync.

Testing it will be easy, and it will *never* upgraded.
It's a bodge of the most horrendous type, to be replaced in the near
future by the real thing. The s/w as a whole is 1.0, and 2.0 will follow
in about 3 months when things settle down a bit.

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff

 

[markspace]

It's going to be just as hard to put them in a file and read it as hard
coding them in from the start. I'll make an int array and then convert
to bytes as I read from it.

It is? Why? How is it harder to read a file than it is to deal with
20k items in a source file? There's a read method in InputStream that
takes an array, you know. Read the whole thing in one go.

Even reading the bytes as a text file from a resource would be
preferable, imo, to coding it into an array initialization in a source file.

 

[markspace]

Try Integer.parseInt(String s, int radix) with radix 16.

And the Scanner class has hasNextByte( int radix ) and nextByte( int
radix ) which is what I'd use to parse bytes out of a file.

 

[markspace]

However, this results in an easily recognizable bytecode pattern which
any compiler (whether JIT or AOT) can easily recognize as such and
transform into a mass initialization

"bastore". Hence, 4 to 11 bytecode bytes for each value byte. However,
.class are compressed and the Deflate algorithm which is used in Jar
files is prone to detect and exploit the redundancy of the pattern.

Just curious: do you have any evidence that the Java compiler does this
operations? I really am curious. It would be neat if it did.

Another way would be to store the bytes as a literal string. Any byte
value can be converted to an octal sequence "\xyz". At runtime, the

Resulting in an 80k string. No, I think I'd prefer a 20k binary file,
which is then loaded as a resource. I know that's as "compressed" as it
can possibly.

 

[Dirk Bruere at NeoPax]

It is? Why? How is it harder to read a file than it is to deal with
20k items in a source file? There's a read method in InputStream that
takes an array, you know. Read the whole thing in one go.

Even reading the bytes as a text file from a resource would be
preferable, imo, to coding it into an array initialization in a source
file.

Well, I may do as you suggest but for now I'll use a macro to edit it
for hard coding. I haven't got around to writing the setup file for the
app, so when I do that I might move the data.

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff

 

[Roedy Green]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast each
one individually?

Here are some ideas.

1. create the array as an int[] and then convert it to byte[].

2. write a throw-away program to reformat the data with the casts.

3. The way I would do it is save that parsed data as Serialized byte[]
compacted with GZIP or a DataOutputStream compacted, and include that
file as a resource. I doubt you really want the data exposed to
accidental editing in the source. see
http://mindprod.com/jgloss/resource.html

4. use CramFull http://mindprod.com/jgloss/cramfull.html

--
Roedy Green Canadian Mind Products
http://mindprod.com

If everyone lived the way people do in Vancouver, we would need three more entire planets to support us.
~ Guy Dauncey

 

[Dirk Bruere at NeoPax]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast each
one individually?

Here are some ideas.

1. create the array as an int[] and then convert it to byte[].

2. write a throw-away program to reformat the data with the casts.

3. The way I would do it is save that parsed data as Serialized byte[]
compacted with GZIP or a DataOutputStream compacted, and include that
file as a resource. I doubt you really want the data exposed to
accidental editing in the source. see
http://mindprod.com/jgloss/resource.html

4. use CramFull http://mindprod.com/jgloss/cramfull.html

Hopefully it is only a temporary measure, and I'm not short of memory
space or processing power for a change. Plus, I'm the only one that will
be editing it - unless I drop dead, in which case maintaining some Java
code will be the least of my problems.

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff

 

[Joshua Cranmer]

Just curious: do you have any evidence that the Java compiler does this
operations? I really am curious. It would be neat if it did.

Popping open the hotspot source code from OpenJDK and looking at the
memnode.cpp file in the opto folder. A function has this documentation
header:

// Coalesce subword constants into int constants and possibly
// into long constants. The goal, if the CPU permits,
// is to initialize the object with a small number of 64-bit tiles.
// Also, convert floating-point constants to bit patterns.
// Non-constants are not relevant to this pass.

It seems that the JIT compiler would transform the initializer into a
series of 64-bit store operations (as long as such instructions exist).

 

[Knute Johnson]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast
each one individually?

Thanks for the replies.
The original text was a Windows .reg file, so it's got to be a cut and
paste job. Good news is that its all plain text comma separated.

One other question - any way I can get the array to see the text input
as hex rather than decimal by default? Otherwise I have to add "0x" to
each entry. Not a vast deal given my text editor, but I'd rather not.

Dirk:

What's the file look like? Maybe there is a simple conversion that you
can use from the text.

 

[Dirk Bruere at NeoPax]

I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f, ... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from an
existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast
each one individually?

Thanks for the replies.
The original text was a Windows .reg file, so it's got to be a cut and
paste job. Good news is that its all plain text comma separated.

One other question - any way I can get the array to see the text input
as hex rather than decimal by default? Otherwise I have to add "0x" to
each entry. Not a vast deal given my text editor, but I'd rather not.

Dirk:

What's the file look like? Maybe there is a simple conversion that you
can use from the text.

Here's a line of it with a continuation backslash ending each line:
00,02,00,00,6e,06,00,00,00,00,00,00,58,50,43,52,01,00,10,00,01,5e,06,00,00,\

Right now I'm just using a macro on a text editor to reformat it. No big
problem.

--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff

 

[Knute Johnson]

Dirk Bruere at NeoPax wrote:
I need to fill a predefined byte array with hexadecimal bytes.
I can do it like this:

public static byte[] default0 = { (byte)0x87, (byte)0x3f,
... };

By casting each value.
Which is a bit of a pain considering I have to copy and paste from
an existing text source of hex bytes (about 20,000 bytes).

Is there a representation I can use such that I do not need to cast
each one individually?


Thanks for the replies.
The original text was a Windows .reg file, so it's got to be a cut
and paste job. Good news is that its all plain text comma separated.

One other question - any way I can get the array to see the text
input as hex rather than decimal by default? Otherwise I have to add
"0x" to each entry. Not a vast deal given my text editor, but I'd
rather not.

Dirk:

What's the file look like? Maybe there is a simple conversion that
you can use from the text.

Here's a line of it with a continuation backslash ending each line:
00,02,00,00,6e,06,00,00,00,00,00,00,58,50,43,52,01,00,10,00,01,5e,06,00,00,\


Right now I'm just using a macro on a text editor to reformat it. No big
problem.

Look at the Scanner class. You can read the data directly into a byte,
write it into a ByteArrayInputStream and then get a byte[] from that.

 

[Dirk Bruere at NeoPax]

Well, I may do as you suggest but for now I'll use a macro to edit it
for hard coding. I haven't got around to writing the setup file for the
app, so when I do that I might move the data.

Next problem I've got is that I get the message: "code too large" when I
try to compile. Have I got to break up the byte array(s) and put them in
separate constants classes?


--
Dirk

http://www.transcendence.me.uk/ - Transcendence UK
http://www.theconsensus.org/ - A UK political party
http://www.onetribe.me.uk/wordpress/?cat=5 - Our podcasts on weird stuff