Binary string packing/unpacking issues

Discussion in 'Ruby' started by Lucas L., Jun 14, 2008.

  1. Lucas L.

    Lucas L. Guest

    Hi,
    I have a packed string of raw binary pixel data that I want to
    manipulate. However, I'm facing a few problems.
    Firstly, the only way I've managed to unpack the data so far is with
    "pixels.unpack('b*')". It works great, but comes out in one massive
    string.
    Is there a way to:
    a)put this into an array with 8 bits in each element, eg. ['00000000',
    '01010101'...]
    b)unpack small sections of a few bites
    The other problem is that I don't know of an efficient way to manipulate
    this data without having to unpack\pack massive strings each time.

    Are there any solutions to these?

    Thanks!
    --
    Posted via http://www.ruby-forum.com/.
    Lucas L., Jun 14, 2008
    #1
    1. Advertising

  2. Lucas L.

    Dave Bass Guest

    This code:

    "Pixels".each_byte { |b| printf("%08b\n", b) }

    produces:

    01010000
    01101001
    01111000
    01100101
    01101100
    01110011

    Any help?
    --
    Posted via http://www.ruby-forum.com/.
    Dave Bass, Jun 14, 2008
    #2
    1. Advertising

  3. Lucas L.

    Lucas L. Guest

    Dave Bass wrote:
    > "Pixels".each_byte { |b| printf("%08b\n", b) }


    That's good for the first problem, thanks.
    Now for the more difficult one of efficiently manipulating small
    portions of the packed binary data.
    --
    Posted via http://www.ruby-forum.com/.
    Lucas L., Jun 14, 2008
    #3
  4. On 14.06.2008 13:56, Lucas L. wrote:
    > Dave Bass wrote:
    >> "Pixels".each_byte { |b| printf("%08b\n", b) }

    >
    > That's good for the first problem, thanks.
    > Now for the more difficult one of efficiently manipulating small
    > portions of the packed binary data.


    With Ruby 1.8 you can easily manipulate individual bytes directly:

    irb(main):006:0> s="abc"
    => "abc"
    irb(main):007:0> s[2]
    => 99
    irb(main):008:0> s[2].to_s 2
    => "1100011"
    irb(main):009:0> s[2] |= 4
    => 103
    irb(main):010:0> s
    => "abg"
    irb(main):011:0> s[2].to_s 2
    => "1100111"

    I don't have a 1.9 handy so I can't tell you how to do it there. But
    the basic lesson should be, that you can leave your data in a String and
    manipulate it directly there.

    Kind regards

    robert
    Robert Klemme, Jun 14, 2008
    #4
  5. Lucas L.

    Lucas L. Guest

    Robert Klemme wrote:
    > With Ruby 1.8 you can easily manipulate individual bytes directly:
    >
    > irb(main):006:0> s="abc"
    > => "abc"
    > irb(main):007:0> s[2]
    > => 99
    > irb(main):008:0> s[2].to_s 2
    > => "1100011"
    > irb(main):009:0> s[2] |= 4
    > => 103
    > irb(main):010:0> s
    > => "abg"
    > irb(main):011:0> s[2].to_s 2
    > => "1100111"


    How can I replace portions of a packed binary string this way?
    I know with a normal string it is easy:
    irb(main):001:0> s = "abcdef"
    => "abcdef"
    irb(main):002:0> s[0..2] = "cba"
    => "cba"
    irb(main):003:0> s
    => "cbadef"

    But I can't figure out how to do it with binary.

    Thanks
    --
    Posted via http://www.ruby-forum.com/.
    Lucas L., Jun 15, 2008
    #5
  6. Lucas L.

    Todd Benson Guest

    On Sun, Jun 15, 2008 at 12:16 AM, Lucas L. <> wrote:

    > How can I replace portions of a packed binary string this way?
    > I know with a normal string it is easy:
    > irb(main):001:0> s = "abcdef"
    > => "abcdef"
    > irb(main):002:0> s[0..2] = "cba"
    > => "cba"
    > irb(main):003:0> s
    > => "cbadef"
    >
    > But I can't figure out how to do it with binary.


    Look closely at Robert's irb line number 9.

    A string is just bytes in a row with special characteristics/methods.

    Todd
    Todd Benson, Jun 15, 2008
    #6
  7. Lucas L.

    Lucas L. Guest

    Todd Benson wrote:
    > On Sun, Jun 15, 2008 at 12:16 AM, Lucas L. <> wrote:
    >
    >> How can I replace portions of a packed binary string this way?
    >> I know with a normal string it is easy:
    >> irb(main):001:0> s = "abcdef"
    >> => "abcdef"
    >> irb(main):002:0> s[0..2] = "cba"
    >> => "cba"
    >> irb(main):003:0> s
    >> => "cbadef"
    >>
    >> But I can't figure out how to do it with binary.

    >
    > Look closely at Robert's irb line number 9.
    >
    > A string is just bytes in a row with special characteristics/methods.
    >
    > Todd


    You'll have to bear with me on this, I struggle with this sort of this.
    I don't really know what using | achieves.
    And a string may be bytes normally, but mine is a sequence of bits (well
    is it in my flawed understanding). I'm using Gtk::pixbuf.pixels, if that
    helps at all.

    Sorry for the stupidity.
    --
    Posted via http://www.ruby-forum.com/.
    Lucas L., Jun 15, 2008
    #7
  8. On 15.06.2008 11:08, Lucas L. wrote:
    > Todd Benson wrote:
    >> On Sun, Jun 15, 2008 at 12:16 AM, Lucas L. <> wrote:
    >>
    >>> How can I replace portions of a packed binary string this way?
    >>> I know with a normal string it is easy:
    >>> irb(main):001:0> s = "abcdef"
    >>> => "abcdef"
    >>> irb(main):002:0> s[0..2] = "cba"
    >>> => "cba"
    >>> irb(main):003:0> s
    >>> => "cbadef"
    >>>
    >>> But I can't figure out how to do it with binary.

    >> Look closely at Robert's irb line number 9.
    >>
    >> A string is just bytes in a row with special characteristics/methods.

    >
    > You'll have to bear with me on this, I struggle with this sort of this.
    > I don't really know what using | achieves.


    It's the bitwise OR operator.

    irb(main):001:0> 1 | 2
    => 3

    > And a string may be bytes normally, but mine is a sequence of bits (well
    > is it in my flawed understanding). I'm using Gtk::pixbuf.pixels, if that
    > helps at all.


    I do not know what Gtk::pixbuf.pixels returns. But if it is a String
    you can manipulate it like was have show before. Can you post what "p
    your_pixbuf.pixels" and "p your_pixbuf.pixels.class" print?

    Kind regards

    robert
    Robert Klemme, Jun 15, 2008
    #8
  9. Lucas L.

    Lucas L. Guest

    Robert Klemme wrote:
    > It's the bitwise OR operator.

    I don't really know why he uses it though.

    Can you post what "p
    > your_pixbuf.pixels" and "p your_pixbuf.pixels.class" print?


    Now we're getting somewhere. I was using puts to print pixels, and all I
    got was a bunch of question marks. p returns a massive String
    (pixels.class shows String). The string itself is uncompressed pixel
    data, a byte per channel of RGBA. The output is a massive string of
    "\000\000\000\000\000\000\000\000..." with "\377" where a pixel is
    drawn, which I'm assumming is 0xFF in the alpha channel (why is \377
    0xFF?).

    However, when I tried
    >pixels[pixels.length-1] = '\377'
    >p pixels

    There was no change.
    So confused:(

    Thanks!
    --
    Posted via http://www.ruby-forum.com/.
    Lucas L., Jun 15, 2008
    #9
  10. Lucas L. wrote:
    > why is \377
    > 0xFF?


    Because those numbers are in octal and 0377 == 0xFF

    HTH,
    Sebastian
    --
    Jabber:
    ICQ: 205544826
    Sebastian Hungerecker, Jun 15, 2008
    #10
  11. Lucas L.

    Paul Irofti Guest

    On 2008-06-15, Lucas L. <> wrote:
    > So confused:(
    >


    That's because you don't seem to grasp the fundamentals, but eitherway
    you're trying to do more `advanced' stuff based on them.
    Relax a little bit, put your project on hold and start reading on the
    way data is stored, how a bit and a byte are defined, and some Boolean
    Algebra 101 won't hurt you.

    Afterwards you'll not only be more productive, but you'll be able to see
    things more clearly and make design choices a lot smarter.

    --

    SDF Public Access UNIX System - http://sdf.lonestar.org
    Paul Irofti, Jun 15, 2008
    #11
  12. On 15.06.2008 12:00, Lucas L. wrote:
    > Robert Klemme wrote:
    >> It's the bitwise OR operator.

    > I don't really know why he uses it though.
    >
    > Can you post what "p
    >> your_pixbuf.pixels" and "p your_pixbuf.pixels.class" print?

    >
    > Now we're getting somewhere. I was using puts to print pixels, and all I
    > got was a bunch of question marks. p returns a massive String
    > (pixels.class shows String). The string itself is uncompressed pixel
    > data, a byte per channel of RGBA. The output is a massive string of
    > "\000\000\000\000\000\000\000\000..." with "\377" where a pixel is
    > drawn, which I'm assumming is 0xFF in the alpha channel (why is \377
    > 0xFF?).
    >
    > However, when I tried
    >> pixels[pixels.length-1] = '\377'
    >> p pixels

    > There was no change.


    irb#1(main):006:0> "\377"
    => "\377"
    irb#1(main):007:0> "\377".length
    => 1
    irb#1(main):008:0> '\377'
    => "\\377"
    irb#1(main):009:0> '\377'.length
    => 4
    irb#1(main):010:0>

    Notice something?

    I agree with Paul, you have trouble with the basics. If you try things
    out on a smaller scale in IRB you'll probably see more easily how
    everything works.

    Kind regards

    robert
    Robert Klemme, Jun 15, 2008
    #12
  13. Lucas L.

    ara.t.howard Guest

    On Jun 14, 2008, at 4:53 AM, Lucas L. wrote:

    > Hi,
    > I have a packed string of raw binary pixel data that I want to
    > manipulate. However, I'm facing a few problems.
    > Firstly, the only way I've managed to unpack the data so far is with
    > "pixels.unpack('b*')". It works great, but comes out in one massive
    > string.
    > Is there a way to:
    > a)put this into an array with 8 bits in each element, eg. ['00000000',
    > '01010101'...]
    > b)unpack small sections of a few bites
    > The other problem is that I don't know of an efficient way to
    > manipulate
    > this data without having to unpack\pack massive strings each time.
    >
    > Are there any solutions to these?


    use narray

    a rather complicated example

    http://blade.nagaokaut.ac.jp/cgi-bin/scat.rb/ruby/ruby-talk/155483

    the basic starting point is

    width = 800
    height = 600

    na = NArray.to_na raw_pixel_data, NArray::BYTE, width, height

    then to, for example, increase every pixel value by 1

    na += 1

    see

    http://narray.rubyforge.org/SPEC.en

    for more.

    gem install narray

    too, of course.


    regards.

    a @ http://codeforpeople.com/
    --
    we can deny everything, except that we have the possibility of being
    better. simply reflect on that.
    h.h. the 14th dalai lama
    ara.t.howard, Jun 16, 2008
    #13
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Victor S. Miller

    Packing and unpacking bitvectors

    Victor S. Miller, Oct 9, 2003, in forum: Python
    Replies:
    0
    Views:
    288
    Victor S. Miller
    Oct 9, 2003
  2. Eelco
    Replies:
    121
    Views:
    1,690
    Eelco
    Jan 3, 2012
  3. Aaron D. Gifford
    Replies:
    3
    Views:
    174
    Aaron D. Gifford
    Apr 7, 2011
  4. Julie

    Packing/unpacking

    Julie, Sep 22, 2003, in forum: Perl Misc
    Replies:
    2
    Views:
    88
    Chris Mattern
    Sep 22, 2003
  5. Shashank R Khanvilkar

    Packing/Unpacking bit fields from a byte.

    Shashank R Khanvilkar, Sep 25, 2005, in forum: Perl Misc
    Replies:
    6
    Views:
    405
    Anno Siegel
    Sep 26, 2005
Loading...

Share This Page