Odd Perl bitwise-AND & MySQL problem?

Discussion in 'Perl Misc' started by dohnut, Oct 20, 2003.

  1. dohnut

    dohnut Guest

    Here's one for some bored problem solver :)

    I ran across this earlier today and fixed it, but don't exactly know
    why. (that usually only happens in C :)

    I'm using Perl version 5.8.0 btw.

    Ok, let's start here: I end up with an array that comes from a
    fetchrow_array() call to MySQL. I return the array to a function.

    Ok, then the function does this (roughly):

    sub func()
    {
    my @dat = shift;

    my $val = $dat[3] & $dat[4] & $dat[5] & 0x7fff;

    print $val;
    }

    In the MySQL database, element 0 = int, 1 = varchar(32), 2 =
    varchar(255), 3 = int, 4 = int, 5 = int, 6 = int, etc..

    Ok, so I call the function. Let's assume $dat[3] = 57 (0x39), $dat[4]
    = 2045 (0x7fd), $dat[5] = 2047 (0x7ff).

    Guess what I get for output? --> "00" (I should get 57 (0x39) btw)

    Not just a zero "0" but _2_ zeros. So, clearly it thinks something
    stringy is there. But why? I guess that is my question.

    Now, if I populate @dat myself (i.e. don't get the values from MySQL
    calls) it works fine.

    To my knowledge only numbers are in the 3 variables and I've tested
    this. If I print the values individually, they print correctly. I
    also do a 'length' on them and I get 2, 4, and 4 respectively -- which
    is correct.

    How did I fix it?

    my $val = int($dat[3]) & int($dat[4]) & int($dat[5]) & 0x7fff;

    I added int()s around each value. So apparently Perl thought there
    was something before the numbers in the variables?? Nothing should be
    there but numbers, the values come from table elements that are ints
    and are read right into a Perl array. Aside from a bug, I can't see
    how this could get corrupted or how I could be missing something.

    I've done quite a bit of MySQL/Perl programming and I've never had
    this happen before, in fact, this specific problem doesn't happen all
    the time even in this code. Only with when returning data from
    specific rows (in my case), though it is consistent and reproducable
    -- it's not flaky. I can note that $dat[2] (which comes from a
    varchar(255)) contains some text and some "/r/n" characters. The ones
    that don't fail don't contain "/r/n" characters in $dat[2]. I don't
    know if this is related, I only have 3 rows of data (at the moment)
    and 1 causes the problem.

    I'm thinking this is a problem in the Perl MySQL DBI module? I
    realize I can do more troubleshooting to narrow it down a bit better
    and I probably will tonight, but just wanted to get it out there and
    see what people thought. Maybe this has been seen before?
     
    dohnut, Oct 20, 2003
    #1
    1. Advertising

  2. dohnut

    Jay Tilton Guest

    (dohnut) wrote:

    : Ok, let's start here: I end up with an array that comes from a
    : fetchrow_array() call to MySQL. I return the array to a function.
    :
    : Ok, then the function does this (roughly):
    :
    : sub func()
    : {
    : my @dat = shift;
    : my $val = $dat[3] & $dat[4] & $dat[5] & 0x7fff;
    : print $val;
    : }

    Good thing you said "roughly." That sub is pretty messed up.

    : In the MySQL database, element 0 = int, 1 = varchar(32), 2 =
    : varchar(255), 3 = int, 4 = int, 5 = int, 6 = int, etc..
    :
    : Ok, so I call the function. Let's assume $dat[3] = 57 (0x39), $dat[4]
    : = 2045 (0x7fd), $dat[5] = 2047 (0x7ff).
    :
    : Guess what I get for output? --> "00" (I should get 57 (0x39) btw)
    :
    : Not just a zero "0" but _2_ zeros. So, clearly it thinks something
    : stringy is there. But why? I guess that is my question.

    Because the database query is returning the values as strings, even
    though they might be stored in the database as integers.

    : Now, if I populate @dat myself (i.e. don't get the values from MySQL
    : calls) it works fine.

    Then you're assigning numeric values, e.g.

    @dat[ 3, 4, 5 ] = ( 57, 2045, 2047 );

    Try it again with string values assigned and see what happens.

    @dat[ 3, 4, 5 ] = ( '57', '2045', '2047' );

    While most operators will force their arguments into the numeric or
    string context that is appropriate to the operator, the bitwise
    operators will change their own context to fit the arguments.

    : To my knowledge only numbers are in the 3 variables and I've tested
    : this. If I print the values individually, they print correctly. I
    : also do a 'length' on them and I get 2, 4, and 4 respectively -- which
    : is correct.

    Neither print() nor length() will reveal wether the value is stored as a
    string or as a number.

    : How did I fix it?
    :
    : my $val = int($dat[3]) & int($dat[4]) & int($dat[5]) & 0x7fff;

    That's a little overkill. To coerce the bitwise operators into working
    in numeric context, perlop recommends using a simple 0+, e.g.

    my $val = 0 + $dat[3] & $dat[4] & $dat[5];

    : So apparently Perl thought there
    : was something before the numbers in the variables??

    Nope. Perl just thinks your numbers are strings, and your program
    happens to use operaters where that distinction makes a difference.
     
    Jay Tilton, Oct 21, 2003
    #2
    1. Advertising

  3. dohnut

    dohnut Guest

    Here's some working code that I created to show the problem.

    First the MySQL database dump
    -------------------------------------------------------------
    CREATE DATABASE /*!32312 IF NOT EXISTS*/ pbug;

    USE pbug;

    CREATE TABLE hosts (
    id int(11) NOT NULL auto_increment,
    ip int(4) unsigned default NULL,
    password varchar(32) default NULL,
    description varchar(255) default '',
    clone int(4) unsigned default '0',
    conn int(4) unsigned default '0',
    status int(4) unsigned default '0',
    tdate timestamp(14) NOT NULL,
    alarms int(11) unsigned default '0',
    monitored int(11) unsigned default '0',
    installed int(11) unsigned default '0',
    soft_ver_major int(4) default '0',
    soft_ver_minor int(4) default '0',
    soft_ver_incr int(4) default '0',
    hard_ver_major int(4) default '0',
    hard_ver_minor int(4) default '0',
    hard_ver_incr int(4) default '0',
    case_temp int(4) default '-1',
    cpu1_temp int(4) default '-1',
    cpu2_temp int(4) default '-1',
    ext_temp int(4) default '-1',
    PRIMARY KEY (id),
    UNIQUE KEY ip (ip)
    ) TYPE=MyISAM;


    INSERT INTO hosts VALUES (1,168430090,'crystal','Demo
    XXXXXXXX',0,129,0,20031020210116,0,61,7,0,0,0,0,0,0,29,-1,-1,-1);
    INSERT INTO hosts VALUES (2,185273099,'password','Test XXXXXXXX
    #1\r\n',0,129,0,20031020210117,57,2045,2047,0,0,0,0,0,0,31,-1,-1,-1);
    INSERT INTO hosts VALUES (3,202116108,'password','Another
    XXXXXXXX\r\n\r\n',0,129,0,20031020210118,3,7,63,0,0,0,0,0,0,30,-1,-1,-1);
    --------------------------------------------

    Put this into a file and add it to your mysql server.
    type: mysql -u <username> -p<password> < pbug.sql

    Ok, and here's my little perl module that demonstrates the error (or
    my stupidity).
    ------------------------------------------------
    #!/usr/bin/perl -w

    use strict;
    use warnings;
    use DBI;


    foreach (1..3)
    {
    my @dat = &GetHostStatus( $_ );

    print "$_: ";
    #printf "0x%x & 0x%x & 0x%x & 0x7fff = ", $dat[3], $dat[4],
    $dat[5];
    print $dat[3] & $dat[4] & $dat[5] & 0x7fff;
    print "\n";
    }

    1;

    ########################################################################

    sub GetHostStatus()
    {
    my $id = shift || return ();

    my $db = DBI->connect( 'dbi:mysql:pbug','bla','bla' ) || die(
    DBI->errstr );

    my $query = "SELECT ip, password, description, alarms, monitored,
    installed, case_temp, cpu1_temp, cpu2_temp, ext_temp FROM hosts WHERE
    id = $id";

    my $sth = $db->prepare( $query ) || die( DBI->errstr );

    $sth->execute() || die( DBI->errstr );

    my @row = $sth->fetchrow_array();

    $sth->finish;
    $db->disconnect;

    return @row;
    }
    --------------------------------------------------

    When I run this on my system, here is my output:

    1: 0
    2: 0
    3: 2

    Clearly incorrect. Now, when I uncomment that printf line:

    1: 0x0 & 0x3d & 0x7 & 0x7fff = 0
    2: 0x39 & 0x7fd & 0x7ff & 0x7fff = 57
    3: 0x3 & 0x7 & 0x3f & 0x7fff = 3

    Then it works.. Hmm...

    I've tested this on 2 separate systems now with the same results, I'd
    be interested to see if any of you guys can duplicate it also? Or
    tell me where I screwed up, that would be better (and more likely).

    Regardless, I'd appreciate any feedback.

    - Chris
     
    dohnut, Oct 21, 2003
    #3
  4. dohnut

    Sam Holden Guest

    On 20 Oct 2003 20:16:05 -0700, dohnut <> wrote:
    > Here's some working code that I created to show the problem.
    >
    > First the MySQL database dump

    [snip the dump]
    > ------------------------------------------------
    > #!/usr/bin/perl -w
    >
    > use strict;
    > use warnings;
    > use DBI;
    >
    >
    > foreach (1..3)
    > {
    > my @dat = &GetHostStatus( $_ );


    Why are you using the & there?

    >
    > print "$_: ";
    > #printf "0x%x & 0x%x & 0x%x & 0x7fff = ", $dat[3], $dat[4],
    > $dat[5];
    > print $dat[3] & $dat[4] & $dat[5] & 0x7fff;
    > print "\n";
    > }


    [snip the GetHostStatus code]

    > --------------------------------------------------
    >
    > When I run this on my system, here is my output:
    >
    > 1: 0
    > 2: 0
    > 3: 2
    >
    > Clearly incorrect. Now, when I uncomment that printf line:
    >
    > 1: 0x0 & 0x3d & 0x7 & 0x7fff = 0
    > 2: 0x39 & 0x7fd & 0x7ff & 0x7fff = 57
    > 3: 0x3 & 0x7 & 0x3f & 0x7fff = 3
    >
    > Then it works.. Hmm...


    The values are strings. Perl defines a bitwise & operator for
    strings and hence uses it. Doing the printf causes perl to generate
    a numeric version of the value, so then the & operator operates
    on the numbers.

    The database stuff is all irrelevant (aside from the fact it is returning
    strings):

    # start with numbers
    my ($x,$y,$z) = (0x39,0x7fd,0x77f);
    print $x & $y & $z & 0x7fff, "\n";

    # create string variables containing the numbers as strings.
    my ($sx, $sy, $sz) = ("$x", "$y", "$z");
    print $sx & $sy & $sz & 0x7fff, "\n";

    # create numeric 'version's of the strings...
    # adding 0 seems clearer than the printf side effect :)
    $_ += 0 for ($sx, $sy, $sz);
    print $sx & $sy & $sz & 0x7fff, "\n";

    Things like this is why the knowledgable people here always argue against
    posters who quote variables for no reason (replace the '("$x", "$y", "$z")'
    in that code with '($z, $y, $z)' and the bahaviour will change).

    --
    Sam Holden
     
    Sam Holden, Oct 21, 2003
    #4
  5. dohnut

    dohnut Guest

    > While most operators will force their arguments into the numeric or
    > string context that is appropriate to the operator, the bitwise
    > operators will change their own context to fit the arguments.


    Ahh, you know I don't think I knew that. Funny this hasn't affected
    me earlier, though I guess I don't do a lot of bit operations on data
    from databases.

    I guess it was just drilled into my head that perl doesn't have data
    types, so the thought never crossed my mind that it would care that
    the data is actually a string and not an integer. And it doesn't --
    except for bit operations. :p

    Well, thanks, that saved me a lot of fruitless debugging. Obviously
    my follow-up I just made can be disregarded, though, at least the code
    actually 'works' in that one (or doesn't, depending on how you look at
    it). ;)

    -Chris
     
    dohnut, Oct 21, 2003
    #5
  6. dohnut

    dohnut Guest

    Sam,

    Thanks for the feedback. I think we've got it solved. I really should
    just start reading these groups instead of just showing up when I have
    problems, I may learn something. :)

    >> my @dat = &GetHostStatus( $_ );

    >
    > Why are you using the & there?


    Ok, I'll admit it, I'm an armchair perl programmer. I'm a long-time C
    programmer professionally, but I use perl quite a bit for web scripting and
    what not. So, with the disclaimer out of the way, may I ask, "What did I
    mess up this time?" ;) I'm assuming my implementation is not ideal, but
    here's why I do/did it. 1) It's how I learned -- subroutines begin with an
    &. 2) It helps me visually identify subroutines (ok, that's kind of weak).
    I know I've seen it left off, but I never explored the how and why. So,
    tell me.. why? :)

    --
    Using M2, Opera's revolutionary e-mail client: http://www.opera.com/m2/
     
    dohnut, Oct 21, 2003
    #6
  7. dohnut

    Sam Holden Guest

    On Tue, 21 Oct 2003 04:39:53 GMT, dohnut <> wrote:
    > Sam,
    >
    > Thanks for the feedback. I think we've got it solved. I really should
    > just start reading these groups instead of just showing up when I have
    > problems, I may learn something. :)
    >
    >>> my @dat = &GetHostStatus( $_ );

    >>
    >> Why are you using the & there?

    >
    > Ok, I'll admit it, I'm an armchair perl programmer. I'm a long-time C
    > programmer professionally, but I use perl quite a bit for web scripting and
    > what not. So, with the disclaimer out of the way, may I ask, "What did I
    > mess up this time?" ;) I'm assuming my implementation is not ideal, but
    > here's why I do/did it. 1) It's how I learned -- subroutines begin with an
    > &. 2) It helps me visually identify subroutines (ok, that's kind of weak).
    > I know I've seen it left off, but I never explored the how and why. So,
    > tell me.. why? :)


    See the documentation in 'perldoc perlsub'.

    or the FAQ:

    "What's the difference between calling a function as &foo
    and foo()?"

    Findable with:

    'perldoc -q "&"'

    Which also find the answer to your previous question, which I didn't
    realise was a FAQ. If I had I would have provided a pointer in my
    original reply, mea culpa...

    In a nutshell, it disables prototypes. You didn't have any
    prototypes so that doesn't matter so much, but turning off a
    mechanism for finding errors for no real benefit is silly, in the
    general case.

    --
    Sam Holden
     
    Sam Holden, Oct 21, 2003
    #7
  8. [ I can't figure out how to repair the attributions.
    Please quote followups properly in the future.
    ]


    dohnut <> wrote:

    >>> my @dat = &GetHostStatus( $_ );

    >>
    >> Why are you using the & there?

    >


    > So, with the disclaimer out of the way, may I ask, "What did I
    > mess up this time?" ;)



    Using subroutines without reading the docs for subroutines.


    > I know I've seen it left off, but I never explored the how and why. So,
    > tell me.. why? :)



    Subroutines in Perl are documented in:

    perldoc perlsub

    ...
    &NAME(LIST); # Circumvent prototypes.
    ...


    --
    Tad McClellan SGML consulting
    Perl programming
    Fort Worth, Texas
     
    Tad McClellan, Oct 21, 2003
    #8
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. dohnut
    Replies:
    0
    Views:
    547
    dohnut
    Oct 20, 2003
  2. dohnut
    Replies:
    1
    Views:
    585
    Sam Holden
    Oct 21, 2003
  3. dohnut
    Replies:
    0
    Views:
    583
    dohnut
    Oct 21, 2003
  4. Serve Laurijssen

    odd/even bitwise and

    Serve Laurijssen, Apr 6, 2004, in forum: C Programming
    Replies:
    27
    Views:
    1,225
    Michael Wojcik
    Apr 13, 2004
  5. Michael Speer

    Odd behavior with odd code

    Michael Speer, Feb 16, 2007, in forum: C Programming
    Replies:
    33
    Views:
    1,105
    Richard Heathfield
    Feb 18, 2007
Loading...

Share This Page