convert byte stream to integers

  • Thread starter Toralf Förster
  • Start date
T

Toralf Förster

Ben said:
perldoc -f unpack

Yep, RTFM b/c I made the wrong question, ok again :

I'm unsure how do I read the 2nd 4byte integer in :

tfoerste@n22 ~/download $ dd if=cohnopafinal_0.65_sha512_portion04 of=4byte
bs=4 count=2
2+0 records in
2+0 records out
8 bytes (8 B) copied, 0.000438815 s, 18.2 kB/s

because this gives only the answer for the first 4 bytes :

tfoerste@n22 ~/download $ cat 4byte | perl -wane 'print unpack "N", $_;
print "\n"'
1831393600
 
S

Steve C


much like with any other language (perl functions in parens)

1) open the file (open)
2) loop through until end-of-file (while, eof)
3) read 32 bits from the file (read)
4) store the data in a variable (unpack)
5) process it - your code here
6) when done close the file (close)

The unpack might seem non-obvious. It takes a string and unpacks it as a perl value.
The reason this step is separate from the read is that perl variables have to know
what type of data they contain, since they can hold different types.
 
U

Uri Guttman

TF> Yep, RTFM b/c I made the wrong question, ok again :

TF> I'm unsure how do I read the 2nd 4byte integer in :

TF> tfoerste@n22 ~/download $ dd if=cohnopafinal_0.65_sha512_portion04 of=4byte
TF> bs=4 count=2
TF> 2+0 records in
TF> 2+0 records out
TF> 8 bytes (8 B) copied, 0.000438815 s, 18.2 kB/s

TF> because this gives only the answer for the first 4 bytes :

TF> tfoerste@n22 ~/download $ cat 4byte | perl -wane 'print unpack "N", $_;
TF> print "\n"'
TF> 1831393600

ever heard of a loop or a quantifier? N only converts one word so why
would you think it would 2 words? there is also a perlpacktut doc you
should read. pack/unpack have lots of formats and options to handle most
any binary data out there. and if you can't do it in one call, you can
use perl to grab data for use by pack/unpack.

uri
 
T

Toralf Förster

Steve said:
The unpack might seem non-obvious. It takes a string and unpacks it as a
perl value. The reason this step is separate from the read is that perl
variables have to know what type of data they contain, since they can hold
different types.

Yep thx, I thought it could be done in one step.

This is now sufficient for me :

perl -we 'open (FILE, "./4byte"); while (read (FILE, my $s, 16)) { my @N =
unpack "I*", $s; print join ("\t", @N), "\n"; } close (FILE)'
 
U

Uri Guttman

TF> Yep thx, I thought it could be done in one step.

it can be. you need to learn perl one liner tricks.

TF> This is now sufficient for me :

TF> perl -we 'open (FILE, "./4byte"); while (read (FILE, my $s, 16)) { my @N =
TF> unpack "I*", $s; print join ("\t", @N), "\n"; } close (FILE)'


with perl -0777n you can read the whole file into $_. then split to the
size you want unpack and print. much simpler than your code. there is
also no need for the close or the temp vars you have.

uri
 
C

C.DeRykus

Yep thx, I thought it could be done in one step.

This is now sufficient for me :

perl -we 'open (FILE, "./4byte"); while (read (FILE, my $s, 16)) { my @N =
unpack "I*", $s; print join ("\t", @N), "\n"; } close (FILE)'

That can be simplified a bit without getting golf-ish:


perl -we 'open(F,"./4byte"); $,="\t"; print unpack "I*",$s while read
F,$s,16'
 
J

John W. Krahn

C.DeRykus said:
perl -we 'open(F,"./4byte"); $,="\t"; print unpack "I*",$s while read
F,$s,16'

You need the -l switch for that to work the same as the OP's

perl -lne'BEGIN{$/=\16;$,="\t"}print unpack"I*",$_' ./4byte



John
 
S

Steve C

Yep thx, I thought it could be done in one step.

This is now sufficient for me :

perl -we 'open (FILE, "./4byte"); while (read (FILE, my $s, 16)) { my @N =
unpack "I*", $s; print join ("\t", @N), "\n"; } close (FILE)'


I forgot to mention that after opening a binary file, you should call
binmode on the filehandle to protect the data from unwanted text file
conversions by the I/O services. Something like this:

open (FILE, '<', "./4byte") or die "Unable to open file\n";
binmode FILE;
 
D

Dr.Ruud

I forgot to mention that after opening a binary file, you should call
binmode on the filehandle to protect the data from unwanted text file
conversions by the I/O services. Something like this:

open (FILE, '<', "./4byte") or die "Unable to open file\n";
binmode FILE;

Or just use the :raw layer, see perlio.

open my $fh, "<:raw", $fname
or die "Error with '$fname': ", $!;
 
S

sreservoir

You need the -l switch for that to work the same as the OP's

perl -lne'BEGIN{$/=\16;$,="\t"}print unpack"I*",$_' ./4byte

John

with 5.10, -E and say.

INIT does well, and unpack takes implicit $_. and a v-string for $,.

on the command line ./4byte should be the same thing as 4byte.

perl -nE'INIT{$/=\16;$,=v9}say unpack"I*"' 4byte # 9 chars shorter.
 
C

C.DeRykus

You need the -l switch for that to work the same as the OP's

perl -lne'BEGIN{$/=\16;$,="\t"}print unpack"I*",$_' ./4byte

Neat. Still, neither is safe on Win32
without binmode - both input/output.

perl -lne 'use open IO=>":raw"; ..."
 
J

John W. Krahn

Ben said:
No. STD{IN,OUT,ERR} are opened before open.pm is loaded, so it doesn't
affect them:

~% PERLIO=:perlio:crlf perl -Mopen=IO,:raw
-E'say for PerlIO::get_layers \*STDIN'
unix
perlio
crlf
~%

There isn't any way to get open.pm to push any layer other than
:encoding or :utf8 onto the STD handles (no, I don't understand why
either). I think the best you're going to get is

perl -lne'BEGIN{binmode$_ for<STD{IN,OUT}>;...}...'

(now *there's* a nasty golf trick...).

Yes but, Perl doesn't use STDIN with the -n or -p switch, it uses ARGV,
and if you use the -i switch as well it uses ARGVOUT instead of STDOUT.



John
 
S

sreservoir

Quoth (e-mail address removed):

True, and, good point :). It's not quite that simple, though:

~% export PERLIO=:perlio:crlf
~% echo foo>foo
~% perl -Mopen=IO,:raw -nE'say for PerlIO::get_layers \*ARGV' foo
unix
~% echo foo | perl -Mopen=IO,:raw
-nE'say for PerlIO::get_layers \*ARGV'
unix
perlio
crlf
~%

and STDOUT will still need explicitly binmoding in cases where you're
using it. (I have no idea where the :perlio went in the first case.)

conclusion: IO golfing is _hard_.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,902
Latest member
Elena68X5

Latest Threads

Top