Low level data manipulation in Perl

Discussion in 'Perl Misc' started by Leonard Challis, Jan 16, 2005.

  1. Hi everyone,

    I have spent a few hours looking on Google, Perl.com, CPAN etc to try find
    some information on messing about with low leveldata in Perl. I am talking
    about opening files and looking at them in their very simplest format, 1s
    and 0s.

    What I have noticed from my searches so far is things like pack(), unpack(),
    binmode() and some other stuff, but not really what I'm looking for, AFAIK.

    If anyone could even just point me in the right direction, maybe just the
    proper keywords to use in google to find some articles and tutorials on this
    kind of task in Perl I would much appreciate it.

    Thanks a lot,
    Lenny Challis
    Leonard Challis, Jan 16, 2005
    #1
    1. Advertising

  2. Leonard Challis wrote:

    > I have spent a few hours looking on Google, Perl.com, CPAN etc to try find
    > some information on messing about with low leveldata in Perl. I am talking
    > about opening files and looking at them in their very simplest format, 1s
    > and 0s.
    >
    > What I have noticed from my searches so far is things like pack(), unpack(),
    > binmode() and some other stuff, but not really what I'm looking for, AFAIK.


    In what way are they not what you are looking at? Appart fromm those
    three there's also vec().
    Brian McCauley, Jan 16, 2005
    #2
    1. Advertising

  3. Leonard Challis wrote:

    > Hi everyone,
    >
    > I have spent a few hours looking on Google, Perl.com, CPAN etc to try find
    > some information on messing about with low leveldata in Perl. I am talking
    > about opening files and looking at them in their very simplest format, 1s
    > and 0s.
    >
    > What I have noticed from my searches so far is things like pack(),
    > unpack(), binmode() and some other stuff, but not really what I'm looking
    > for, AFAIK.
    >
    > If anyone could even just point me in the right direction, maybe just the
    > proper keywords to use in google to find some articles and tutorials on
    > this kind of task in Perl I would much appreciate it.
    >
    > Thanks a lot,
    > Lenny Challis


    Pack/unpack easily translate to hexadecimal, and conversion to binary from
    there is trivial.

    But Perl has the full complement of bit twiddling operators anyway (similar
    to C).

    gtoomey
    Gregory Toomey, Jan 16, 2005
    #3
  4. What I am saying is this: Is there any way of opening a file, and instead of
    getting the text in side it, you actually see the 1s and0s or the
    hexnumbers?

    This would then allow me to use the bitwise operators etc to manipulate it.

    If there isn't, what would anyone suggest trying? For instance, could i
    convert the ascii data i receive in to hex or binary. I'm guessing it isn't
    just a simple case of converting it back right?

    Thanks for any tips or hints, even a book to read :)
    Cheers,


    Lenny



    "Gregory Toomey" <> wrote in message
    news:...
    > Leonard Challis wrote:
    >
    >> Hi everyone,
    >>
    >> I have spent a few hours looking on Google, Perl.com, CPAN etc to try
    >> find
    >> some information on messing about with low leveldata in Perl. I am
    >> talking
    >> about opening files and looking at them in their very simplest format, 1s
    >> and 0s.
    >>
    >> What I have noticed from my searches so far is things like pack(),
    >> unpack(), binmode() and some other stuff, but not really what I'm looking
    >> for, AFAIK.
    >>
    >> If anyone could even just point me in the right direction, maybe just the
    >> proper keywords to use in google to find some articles and tutorials on
    >> this kind of task in Perl I would much appreciate it.
    >>
    >> Thanks a lot,
    >> Lenny Challis

    >
    > Pack/unpack easily translate to hexadecimal, and conversion to binary from
    > there is trivial.
    >
    > But Perl has the full complement of bit twiddling operators anyway
    > (similar
    > to C).
    >
    > gtoomey
    Leonard Challis, Jan 17, 2005
    #4
  5. Leonard Challis

    Guest

    "Leonard Challis" <> wrote:
    > What I am saying is this: Is there any way of opening a file, and instead
    > of getting the text in side it, you actually see the 1s and0s or the
    > hexnumbers?


    The text is identical to the 1 and 0s. I highly doubt that you are able
    to stare at a CPU and "see" the insides of the Perl and perl programs. So
    what you see is not what is inside perl, but rather what you told Perl to
    print to your monitor. If you want to see 1s and 0s, then tell Perl to
    print them as 1s and 0s.

    > This would then allow me to use the bitwise operators etc to manipulate
    > it.


    Why don't you read the docs on those bitwise operators?

    > If there isn't, what would anyone suggest trying? For instance, could i
    > convert the ascii data i receive in to hex or binary. I'm guessing it
    > isn't just a simple case of converting it back right?
    >
    > Thanks for any tips or hints, even a book to read :)


    I think you've already been given those tips and hints. Please, go
    followup on them. After you read all of perlop and all of perlfunc, if
    you don't understand, come back and ask specific questions about the
    operators and functions you don't understand.

    Xho

    --
    -------------------- http://NewsReader.Com/ --------------------
    Usenet Newsgroup Service $9.95/Month 30GB
    , Jan 17, 2005
    #5
  6. Leonard Challis

    Anno Siegel Guest

    Leonard Challis <> wrote in comp.lang.perl.misc:
    > What I am saying is this: Is there any way of opening a file, and instead of
    > getting the text in side it, you actually see the 1s and0s or the
    > hexnumbers?


    The text *is* the ones and zeroes, it's just a matter of interpretation.
    To access individual bits, use vec() (perldoc -f vec).

    > This would then allow me to use the bitwise operators etc to manipulate it.


    What is stopping you?

    Here is an example: Compare the strings "a" and "A" bitwise and show the
    bit number(s) where they differ:

    my ( $lower, $upper) = ( 'a', 'A');
    for ( 0 .. 7 ) { # eight bits
    print "bits $_ differ\n" if vec( $lower, $_, 1) != vec( $upper, $_, 1);
    }

    For ASCII code, that prints "bits 5 differ". Check with an ASCII table.

    Anno
    Anno Siegel, Jan 17, 2005
    #6
  7. "Leonard Challis" <> wrote in
    news:csgpvs$q6d$:

    [ Please do not top-post. If you do not know what that means, and even if
    you do, please read the posting guidelines for this group. ]

    > What I am saying is this: Is there any way of opening a file, and
    > instead of getting the text in side it, you actually see the 1s and0s
    > or the hexnumbers?
    >
    > This would then allow me to use the bitwise operators etc to
    > manipulate it.


    You are misguided.

    See

    perldoc perlop

    Sinan.
    A. Sinan Unur, Jan 17, 2005
    #7
  8. You don'tseem to get me straight...

    > "Leonard Challis" <> wrote:
    >> What I am saying is this: Is there any way of opening a file, and instead
    >> of getting the text in side it, you actually see the 1s and0s or the
    >> hexnumbers?


    > The text is identical to the 1 and 0s. I highly doubt that you are able
    > to stare at a CPU and "see" the insides of the Perl and perl programs. So
    > what you see is not what is inside perl, but rather what you told Perl to
    > print to your monitor. If you want to see 1s and 0s, then tell Perl to
    > print them as 1s and 0s.


    I don't want to look at perl's code, I just want to see the file I open. Say
    for instance I have an image and I open it in a HEX editor, I see the binary
    or hex as well as the text for that file. I was wondering if there was any
    way of opening the file and inputting it in this format instead of doing a
    lot of converting. I know you can convert stuff, google has plenty of stuff.

    >> This would then allow me to use the bitwise operators etc to manipulate
    >> it.

    >
    > Why don't you read the docs on those bitwise operators?


    I know, i was just stating thats what I was planning on doing so you had
    more information. I have been reading up on them.

    >
    >> If there isn't, what would anyone suggest trying? For instance, could i
    >> convert the ascii data i receive in to hex or binary. I'm guessing it
    >> isn't just a simple case of converting it back right?
    >>
    >> Thanks for any tips or hints, even a book to read :)

    >
    > I think you've already been given those tips and hints. Please, go
    > followup on them. After you read all of perlop and all of perlfunc, if
    > you don't understand, come back and ask specific questions about the
    > operators and functions you don't understand.


    I have been toldabout conversion to hex/binary and about bitwise operators..
    NOT about opening a file and inputting in this format straight away. If you
    don'thave an answer I don't mind, don't moan.

    Lenny Challis
    Leonard Challis, Jan 17, 2005
    #8
  9. Awesome.

    This is exactly what I have been wondering, whether converting from the
    ASCII representation would give me what it is in Binary/HEXetc, and I have
    been reading about vec() too, it's pretty cool!

    Thanks alot for your example,
    Lenny Challis

    "Anno Siegel" <-berlin.de> wrote in message
    news:csgsih$t3v$-Berlin.DE...
    > Leonard Challis <> wrote in comp.lang.perl.misc:
    >> What I am saying is this: Is there any way of opening a file, and instead
    >> of
    >> getting the text in side it, you actually see the 1s and0s or the
    >> hexnumbers?

    >
    > The text *is* the ones and zeroes, it's just a matter of interpretation.
    > To access individual bits, use vec() (perldoc -f vec).
    >
    >> This would then allow me to use the bitwise operators etc to manipulate
    >> it.

    >
    > What is stopping you?
    >
    > Here is an example: Compare the strings "a" and "A" bitwise and show the
    > bit number(s) where they differ:
    >
    > my ( $lower, $upper) = ( 'a', 'A');
    > for ( 0 .. 7 ) { # eight bits
    > print "bits $_ differ\n" if vec( $lower, $_, 1) != vec( $upper, $_,
    > 1);
    > }
    >
    > For ASCII code, that prints "bits 5 differ". Check with an ASCII table.
    >
    > Anno
    Leonard Challis, Jan 17, 2005
    #9
  10. "Leonard Challis" <> wrote in
    news:csgvju$413$:

    >> "Leonard Challis" <> wrote:
    >>> What I am saying is this: Is there any way of opening a file, and
    >>> instead of getting the text in side it, you actually see the 1s
    >>> and0s or the hexnumbers?


    ....

    > I have been toldabout conversion to hex/binary and about bitwise
    > operators.. NOT about opening a file and inputting in this format
    > straight away. If you don'thave an answer I don't mind, don't moan.


    WTF are you talking about? There is no excuse for the problem
    description above unless you are six years old.

    #! /usr/bin/perl

    use strict;
    use warnings;

    use File::Slurp;

    my $file = shift or die "Provide an input file name.\n";
    my $contents = read_file($file, binmode => ':raw');

    use constant CHUNK_SIZE => 0x10;

    my $pos = 0;
    my $last = length $contents;

    while($pos + CHUNK_SIZE < $last) {
    print hexdump_line(substr $contents, $pos, CHUNK_SIZE);
    $pos += CHUNK_SIZE;
    }

    print hexdump_line(substr $contents, $pos, $last - $pos) if($pos <
    $last);

    sub hexdump_line {
    my ($row) = @_;
    my @bytes = split '', $row;
    my $output;
    $output .= sprintf '%2.2X', ord($_) for @bytes;
    $row =~ s/[^\x20-\x7e]/\./g;
    $output .= "\t$row\n";
    }
    __END__

    C:\Home> 050117-b.pl 050117-b.pl
    2321202F7573722F62696E2F7065726C #! /usr/bin/perl
    0D0A0D0A757365207374726963743B0D ....use strict;.
    0A757365207761726E696E67733B0D0A .use warnings;..
    0D0A7573652046696C653A3A536C7572 ..use File::Slur
    703B0D0A0D0A6D79202466696C65203D p;....my $file =

    Sinan
    A. Sinan Unur, Jan 17, 2005
    #10
  11. Leonard Challis

    Guest

    Leonard Challis wrote:
    >
    > I have spent a few hours looking on Google, Perl.com,
    > CPAN etc to try find some information on messing about
    > with low leveldata in Perl. I am talking about opening
    > files and looking at them in their very simplest format,
    > 1s and 0s.



    I do a lot of work with binary files. Sometimes I need to find the
    value of a specific bit in a binary file. Here is how to convert a
    binary file into its binary 0s and 1s:

    # Unix:
    perl -l -0777 -ne 'print unpack("B*", $_)' binary_file
    # DOS (using ActiveState Perl 5.8 or later):
    perl -l -Mopen=IO,:raw -0777 -ne "print unpack('B*', $_)" binary_file

    (Note: Replace the 'B*' packstring with 'H*' to see the hexadecimal
    equivalent.)

    Hopefully this will help you, Leonard.

    -- Jean-Luc
    , Jan 17, 2005
    #11
  12. Leonard Challis

    Anno Siegel Guest

    Leonard Challis <> wrote in comp.lang.perl.misc:

    > Awesome.


    Looks like you finally got it.

    [snip TOFU]

    Anno
    Anno Siegel, Jan 17, 2005
    #12
  13. Leonard Challis <> wrote:
    > You don'tseem to get me straight...



    Who is the "you" that you refer to?

    The only attribution you've provided is yourself.

    Are you talking to yourself?

    Please provide an attribution when you quote someone, so that folks
    can tell who said what.


    > don't moan.



    Consider following your own advice.


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Jan 18, 2005
    #13
  14. "Leonard Challis" <> wrote in message
    news:csjh2s$fc6$...
    >I am referring to most people in this group who aren't newbies. If you do
    >plenty of searching first and then ask a question you will still get
    >flamed. People seem to enjoy flaming each other in this group. It's quite
    >sad


    > really. I fully agree with people just posting "I cant do it how do i do

    ^disagree (typo)
    > it!" questions, but when people show what they have searched already and
    > why they are still struggling do they deserve being flamed still?
    >
    > Lenny Challis
    >
    > "Tad McClellan" <> wrote in message
    > news:...
    >> Leonard Challis <> wrote:
    >>> You don'tseem to get me straight...

    >>
    >>
    >> Who is the "you" that you refer to?
    >>
    >> The only attribution you've provided is yourself.
    >>
    >> Are you talking to yourself?
    >>
    >> Please provide an attribution when you quote someone, so that folks
    >> can tell who said what.
    >>
    >>
    >>> don't moan.

    >>
    >>
    >> Consider following your own advice.
    >>
    >>
    >> --
    >> Tad McClellan SGML consulting
    >> Perl programming
    >> Fort Worth, Texas

    >
    >
    Leonard Challis, Jan 18, 2005
    #14
  15. "Leonard Challis" <> writes:
    > I am referring to most people in this group who aren't newbies. If you do
    > plenty of searching first and then ask a question you will still get flamed.
    > People seem to enjoy flaming each other in this group. It's quite sad
    > really. I fully agree with people just posting "I cant do it how do i do
    > it!" questions, but when people show what they have searched already and why
    > they are still struggling do they deserve being flamed still?


    The text you quote (*) mostly mentions attributions, and their absence
    in your article. You wrote "You don't seem to get me straight", with
    no indication on who or what you mean.

    I don't know what "most people in this group who aren't newbies" like
    to do. Many of them probably read this group hoping to learn something
    now and then, and don't bother to post anything. Some jump in and post
    something now and then when they think it might be useful and not
    mentioned already. Some point out style and other errors, including
    failure to follow the posting guidelines, as a matter of routine.

    (*) In the wrong place. It's better and common practice to write your
    comments after the text you're referring to. The opposite is called
    "top-posting".
    Arndt Jonasson, Jan 19, 2005
    #15
  16. Arndt Jonasson wrote:
    > "Leonard Challis" <> writes:
    >> I am referring to most people in this group who aren't newbies. If
    >> you do plenty of searching first and then ask a question you will
    >> still get flamed. People seem to enjoy flaming each other in this
    >> group. It's quite sad really. I fully agree with people just posting
    >> "I cant do it how do i do it!" questions, but when people show what
    >> they have searched already and why they are still struggling do they
    >> deserve being flamed still?

    >
    > The text you quote (*) mostly mentions attributions, and their absence
    > in your article. You wrote "You don't seem to get me straight", with
    > no indication on who or what you mean.


    I don't know about you but it seemed plainly obvious to me that he was
    referring to the name attributed /below/ that line:

    (An excerpt.)

    You don'tseem to get me straight...

    > "Leonard Challis" <> wrote:
    >> What I am saying is this: Is there any way of opening a file, and
    >> instead
    >> of getting the text in side it, you actually see the 1s and0s or the
    >> hexnumbers?


    (End of excerpt.)

    While he wasn't using "standard quoting" practice, is that hard to see
    who he was referring to? (Nothing wrong with wanting to correct his
    quoting technique, but I don't see why some people had to act like it
    was so hard to tell...)
    Alfred Z. Newmane, Jan 20, 2005
    #16
  17. "Alfred Z. Newmane" <> wrote in message
    news:...

    > While he wasn't using "standard quoting" practice, is that hard to see who
    > he was referring to? (Nothing wrong with wanting to correct his quoting
    > technique, but I don't see why some people had to act like it was so hard
    > to tell...)


    This is the point I was trying to put forward, but anyhow let's drop the
    issue it doesn't matter.

    Would you mind pointing me in the direction of the newsgroup guidelines and
    quoting techniques and anything else you feel I ought to know. Thanks for
    your time,
    Lenny Challis
    Leonard Challis, Jan 20, 2005
    #17
  18. "Alfred Z. Newmane" <> writes:
    > Arndt Jonasson wrote:
    > > "Leonard Challis" <> writes:
    > >> I am referring to most people in this group who aren't newbies. If
    > >> you do plenty of searching first and then ask a question you will
    > >> still get flamed. People seem to enjoy flaming each other in this
    > >> group. It's quite sad really. I fully agree with people just posting
    > >> "I cant do it how do i do it!" questions, but when people show what
    > >> they have searched already and why they are still struggling do they
    > >> deserve being flamed still?

    > >
    > > The text you quote (*) mostly mentions attributions, and their absence
    > > in your article. You wrote "You don't seem to get me straight", with
    > > no indication on who or what you mean.

    >
    > I don't know about you but it seemed plainly obvious to me that he was
    > referring to the name attributed /below/ that line:
    >
    > (An excerpt.)
    >
    > You don'tseem to get me straight...
    >
    > > "Leonard Challis" <> wrote:
    > >> What I am saying is this: Is there any way of opening a file, and
    > >> instead
    > >> of getting the text in side it, you actually see the 1s and0s or the
    > >> hexnumbers?

    >
    > (End of excerpt.)


    Yes, but that's his _own_ name. But I think most points have been made
    by now, and maybe taken too. Let's get back to Perl.
    Arndt Jonasson, Jan 20, 2005
    #18
  19. Leonard Challis <> wrote:

    > Would you mind pointing me in the direction of the newsgroup guidelines



    They are posted here twice each week.

    http://mail.augustmail.com/~tadmc/clpmisc.shtml


    > and
    > quoting techniques



    There are pointers in the Guidelines to those.


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
    Tad McClellan, Jan 20, 2005
    #19
  20. Leonard Challis, Jan 20, 2005
    #20
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.

Share This Page