Low level data manipulation in Perl

[Leonard Challis]

Hi everyone,

I have spent a few hours looking on Google, Perl.com, CPAN etc to try find
some information on messing about with low leveldata in Perl. I am talking
about opening files and looking at them in their very simplest format, 1s
and 0s.

What I have noticed from my searches so far is things like pack(), unpack(),
binmode() and some other stuff, but not really what I'm looking for, AFAIK.

If anyone could even just point me in the right direction, maybe just the
proper keywords to use in google to find some articles and tutorials on this
kind of task in Perl I would much appreciate it.

Thanks a lot,
Lenny Challis

 

[Brian McCauley]

I have spent a few hours looking on Google, Perl.com, CPAN etc to try find
some information on messing about with low leveldata in Perl. I am talking
about opening files and looking at them in their very simplest format, 1s
and 0s.

What I have noticed from my searches so far is things like pack(), unpack(),
binmode() and some other stuff, but not really what I'm looking for, AFAIK.

In what way are they not what you are looking at? Appart fromm those
three there's also vec().

 

[Gregory Toomey]

Hi everyone,

I have spent a few hours looking on Google, Perl.com, CPAN etc to try find
some information on messing about with low leveldata in Perl. I am talking
about opening files and looking at them in their very simplest format, 1s
and 0s.

What I have noticed from my searches so far is things like pack(),
unpack(), binmode() and some other stuff, but not really what I'm looking
for, AFAIK.

If anyone could even just point me in the right direction, maybe just the
proper keywords to use in google to find some articles and tutorials on
this kind of task in Perl I would much appreciate it.

Thanks a lot,
Lenny Challis

Pack/unpack easily translate to hexadecimal, and conversion to binary from
there is trivial.

But Perl has the full complement of bit twiddling operators anyway (similar
to C).

gtoomey

 

[Leonard Challis]

What I am saying is this: Is there any way of opening a file, and instead of
getting the text in side it, you actually see the 1s and0s or the
hexnumbers?

This would then allow me to use the bitwise operators etc to manipulate it.

If there isn't, what would anyone suggest trying? For instance, could i
convert the ascii data i receive in to hex or binary. I'm guessing it isn't
just a simple case of converting it back right?

Thanks for any tips or hints, even a book to read
Cheers,


Lenny

 

[xhoster]

What I am saying is this: Is there any way of opening a file, and instead
of getting the text in side it, you actually see the 1s and0s or the
hexnumbers?

The text is identical to the 1 and 0s. I highly doubt that you are able
to stare at a CPU and "see" the insides of the Perl and perl programs. So
what you see is not what is inside perl, but rather what you told Perl to
print to your monitor. If you want to see 1s and 0s, then tell Perl to
print them as 1s and 0s.

This would then allow me to use the bitwise operators etc to manipulate
it.

Why don't you read the docs on those bitwise operators?

If there isn't, what would anyone suggest trying? For instance, could i
convert the ascii data i receive in to hex or binary. I'm guessing it
isn't just a simple case of converting it back right?

Thanks for any tips or hints, even a book to read

I think you've already been given those tips and hints. Please, go
followup on them. After you read all of perlop and all of perlfunc, if
you don't understand, come back and ask specific questions about the
operators and functions you don't understand.

Xho

 

[Anno Siegel]

What I am saying is this: Is there any way of opening a file, and instead of
getting the text in side it, you actually see the 1s and0s or the
hexnumbers?

The text *is* the ones and zeroes, it's just a matter of interpretation.
To access individual bits, use vec() (perldoc -f vec).

This would then allow me to use the bitwise operators etc to manipulate it.

What is stopping you?

Here is an example: Compare the strings "a" and "A" bitwise and show the
bit number(s) where they differ:

my ( $lower, $upper) = ( 'a', 'A');
for ( 0 .. 7 ) { # eight bits
print "bits $_ differ\n" if vec( $lower, $_, 1) != vec( $upper, $_, 1);
}

For ASCII code, that prints "bits 5 differ". Check with an ASCII table.

Anno

 

[A. Sinan Unur]

[ Please do not top-post. If you do not know what that means, and even if
you do, please read the posting guidelines for this group. ]

What I am saying is this: Is there any way of opening a file, and
instead of getting the text in side it, you actually see the 1s and0s
or the hexnumbers?

This would then allow me to use the bitwise operators etc to
manipulate it.

You are misguided.

See

perldoc perlop

Sinan.

 

[Leonard Challis]

You don'tseem to get me straight...

The text is identical to the 1 and 0s. I highly doubt that you are able
to stare at a CPU and "see" the insides of the Perl and perl programs. So
what you see is not what is inside perl, but rather what you told Perl to
print to your monitor. If you want to see 1s and 0s, then tell Perl to
print them as 1s and 0s.

I don't want to look at perl's code, I just want to see the file I open. Say
for instance I have an image and I open it in a HEX editor, I see the binary
or hex as well as the text for that file. I was wondering if there was any
way of opening the file and inputting it in this format instead of doing a
lot of converting. I know you can convert stuff, google has plenty of stuff.

Why don't you read the docs on those bitwise operators?

I know, i was just stating thats what I was planning on doing so you had
more information. I have been reading up on them.

I think you've already been given those tips and hints. Please, go
followup on them. After you read all of perlop and all of perlfunc, if
you don't understand, come back and ask specific questions about the
operators and functions you don't understand.

I have been toldabout conversion to hex/binary and about bitwise operators..
NOT about opening a file and inputting in this format straight away. If you
don'thave an answer I don't mind, don't moan.

Lenny Challis

 

[Leonard Challis]

Awesome.

This is exactly what I have been wondering, whether converting from the
ASCII representation would give me what it is in Binary/HEXetc, and I have
been reading about vec() too, it's pretty cool!

Thanks alot for your example,
Lenny Challis

 

[A. Sinan Unur]

....

I have been toldabout conversion to hex/binary and about bitwise
operators.. NOT about opening a file and inputting in this format
straight away. If you don'thave an answer I don't mind, don't moan.

WTF are you talking about? There is no excuse for the problem
description above unless you are six years old.

#! /usr/bin/perl

use strict;
use warnings;

use File::Slurp;

my $file = shift or die "Provide an input file name.\n";
my $contents = read_file($file, binmode => ':raw');

use constant CHUNK_SIZE => 0x10;

my $pos = 0;
my $last = length $contents;

while($pos + CHUNK_SIZE < $last) {
print hexdump_line(substr $contents, $pos, CHUNK_SIZE);
$pos += CHUNK_SIZE;
}

print hexdump_line(substr $contents, $pos, $last - $pos) if($pos <
$last);

sub hexdump_line {
my ($row) = @_;
my @bytes = split '', $row;
my $output;
$output .= sprintf '%2.2X', ord($_) for @bytes;
$row =~ s/[^\x20-\x7e]/\./g;
$output .= "\t$row\n";
}
__END__

C:\Home> 050117-b.pl 050117-b.pl
2321202F7573722F62696E2F7065726C #! /usr/bin/perl
0D0A0D0A757365207374726963743B0D ....use strict;.
0A757365207761726E696E67733B0D0A .use warnings;..
0D0A7573652046696C653A3A536C7572 ..use File::Slur
703B0D0A0D0A6D79202466696C65203D p;....my $file =

Sinan

 

[jl_post]

I have spent a few hours looking on Google, Perl.com,
CPAN etc to try find some information on messing about
with low leveldata in Perl. I am talking about opening
files and looking at them in their very simplest format,
1s and 0s.

I do a lot of work with binary files. Sometimes I need to find the
value of a specific bit in a binary file. Here is how to convert a
binary file into its binary 0s and 1s:

# Unix:
perl -l -0777 -ne 'print unpack("B*", $_)' binary_file
# DOS (using ActiveState Perl 5.8 or later):
perl -l -Mopen=IO,:raw -0777 -ne "print unpack('B*', $_)" binary_file

(Note: Replace the 'B*' packstring with 'H*' to see the hexadecimal
equivalent.)

Hopefully this will help you, Leonard.

-- Jean-Luc

 

[Anno Siegel]

Awesome.

Looks like you finally got it.

[snip TOFU]

Anno

 

[Tad McClellan]

You don'tseem to get me straight...

Who is the "you" that you refer to?

The only attribution you've provided is yourself.

Are you talking to yourself?

Please provide an attribution when you quote someone, so that folks
can tell who said what.

don't moan.

Consider following your own advice.

 

[Leonard Challis]

I am referring to most people in this group who aren't newbies. If you do
plenty of searching first and then ask a question you will still get
flamed. People seem to enjoy flaming each other in this group. It's quite
sad

really. I fully agree with people just posting "I cant do it how do i do ^disagree (typo)
it!" questions, but when people show what they have searched already and
why they are still struggling do they deserve being flamed still?

Lenny Challis

 

[Arndt Jonasson]

I am referring to most people in this group who aren't newbies. If you do
plenty of searching first and then ask a question you will still get flamed.
People seem to enjoy flaming each other in this group. It's quite sad
really. I fully agree with people just posting "I cant do it how do i do
it!" questions, but when people show what they have searched already and why
they are still struggling do they deserve being flamed still?

The text you quote (*) mostly mentions attributions, and their absence
in your article. You wrote "You don't seem to get me straight", with
no indication on who or what you mean.

I don't know what "most people in this group who aren't newbies" like
to do. Many of them probably read this group hoping to learn something
now and then, and don't bother to post anything. Some jump in and post
something now and then when they think it might be useful and not
mentioned already. Some point out style and other errors, including
failure to follow the posting guidelines, as a matter of routine.

(*) In the wrong place. It's better and common practice to write your
comments after the text you're referring to. The opposite is called
"top-posting".

 

[Alfred Z. Newmane]

The text you quote (*) mostly mentions attributions, and their absence
in your article. You wrote "You don't seem to get me straight", with
no indication on who or what you mean.

I don't know about you but it seemed plainly obvious to me that he was
referring to the name attributed /below/ that line:

(An excerpt.)

You don'tseem to get me straight...

(End of excerpt.)

While he wasn't using "standard quoting" practice, is that hard to see
who he was referring to? (Nothing wrong with wanting to correct his
quoting technique, but I don't see why some people had to act like it
was so hard to tell...)

 

[Leonard Challis]

While he wasn't using "standard quoting" practice, is that hard to see who
he was referring to? (Nothing wrong with wanting to correct his quoting
technique, but I don't see why some people had to act like it was so hard
to tell...)

This is the point I was trying to put forward, but anyhow let's drop the
issue it doesn't matter.

Would you mind pointing me in the direction of the newsgroup guidelines and
quoting techniques and anything else you feel I ought to know. Thanks for
your time,
Lenny Challis

 

[Arndt Jonasson]

I don't know about you but it seemed plainly obvious to me that he was
referring to the name attributed /below/ that line:

(An excerpt.)

You don'tseem to get me straight...


(End of excerpt.)

Yes, but that's his _own_ name. But I think most points have been made
by now, and maybe taken too. Let's get back to Perl.

 

[Tad McClellan]

Would you mind pointing me in the direction of the newsgroup guidelines

They are posted here twice each week.

http://mail.augustmail.com/~tadmc/clpmisc.shtml

and
quoting techniques

There are pointers in the Guidelines to those.

 

[Leonard Challis]

Tad McClellan wrote

They are posted here twice each week.

http://mail.augustmail.com/~tadmc/clpmisc.shtml

Excellent - thanks for that Tad