Is bytecode machine (in)dependent?

Discussion in 'Python' started by Robert McLay, Oct 28, 2005.

  1. Robert McLay

    Robert McLay Guest

    I'm trying to understand bytecodes generated on different machines.
    I understand that the bytecodes can change between version. But since
    I'm told that .pyc files are version dependent but not machine
    dependent, I'm wondering why the bytecodes are machine dependent.

    my friend and I created this simple example to explore the problem.
    The example code is:

    #!/usr/bin/env python
    #
    import sys, os

    def main():
    x = 1.234
    print x

    if ( __name__ == '__main__'):
    main()


    Using sib.py from Vendorid 1.0 generates different bytecodes under linux
    and sgi. At its core sib.py is using:

    # Compile the code using prefix and marshall it.
    compiled_code = compile(source_code, prefix, 'exec')
    marshalled_code = marshal.dumps(compiled_code)

    to get the bytecodes.

    So why are the byte codes different? Is it that the intel-linux is
    little endian and the SGI is big endian and the numerical constant
    (1.234) is stored different depending on the endian-ness?


    This was generated under intel-linux using python 2.4.2




    /*==========================================================================*/
    /* Frozen main script for test */
    /* Generated from test.py */
    /* This is generated code; Do not modify it! */
    /*--------------------------------------------------------------------------*/
    unsigned char M___main__[] =
    {
    99,0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,
    0,115,55,0,0,0,100,0,0,107,0,0,90,0,0,100,
    0,0,107,1,0,90,1,0,100,1,0,132,0,0,90,2,
    0,101,3,0,100,2,0,106,2,0,111,11,0,1,101,2,
    0,131,0,0,1,110,1,0,1,100,0,0,83,40,3,0,
    0,0,78,99,0,0,0,0,1,0,0,0,1,0,0,0,
    67,0,0,0,115,15,0,0,0,100,1,0,125,0,0,124,
    0,0,71,72,100,0,0,83,40,2,0,0,0,78,102,5,
    49,46,50,51,52,40,1,0,0,0,116,1,0,0,0,120,
    40,1,0,0,0,82,0,0,0,0,40,0,0,0,0,40,
    0,0,0,0,116,23,0,0,0,47,104,111,109,101,47,118, /* This line */
    112,97,114,114,47,115,105,98,47,116,101,115,116,46,112,121,
    116,4,0,0,0,109,97,105,110,5,0,0,0,115,4,0,
    0,0,0,1,6,1,116,8,0,0,0,95,95,109,97,105,
    110,95,95,40,4,0,0,0,116,3,0,0,0,115,121,115,
    116,2,0,0,0,111,115,82,2,0,0,0,116,8,0,0,
    0,95,95,110,97,109,101,95,95,40,3,0,0,0,82,4,
    0,0,0,82,2,0,0,0,82,5,0,0,0,40,0,0,
    0,0,40,0,0,0,0,82,1,0,0,0,116,1,0,0,
    0,63,3,0,0,0,115,6,0,0,0,18,2,9,4,13,
    1,
    }; x

    And under SGI (python 2.4.2) it created :

    /*==========================================================================*/
    /* Frozen main script for test */
    /* Generated from test.py */
    /* This is generated code; Do not modify it! */
    /*--------------------------------------------------------------------------*/
    unsigned char M___main__[] =
    {
    99,0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,
    0,115,55,0,0,0,100,0,0,107,0,0,90,0,0,100,
    0,0,107,1,0,90,1,0,100,1,0,132,0,0,90,2,
    0,101,3,0,100,2,0,106,2,0,111,11,0,1,101,2,
    0,131,0,0,1,110,1,0,1,100,0,0,83,40,3,0,
    0,0,78,99,0,0,0,0,1,0,0,0,1,0,0,0,
    67,0,0,0,115,15,0,0,0,100,1,0,125,0,0,124,
    0,0,71,72,100,0,0,83,40,2,0,0,0,78,102,5,
    49,46,50,51,52,40,1,0,0,0,116,1,0,0,0,120,
    40,1,0,0,0,82,0,0,0,0,40,0,0,0,0,40,
    0,0,0,0,116,23,0,0,0,47,87,111,114,107,47,118, /* This line */
    112,97,114,114,47,115,105,98,47,116,101,115,116,46,112,121,
    116,4,0,0,0,109,97,105,110,5,0,0,0,115,4,0,
    0,0,0,1,6,1,116,8,0,0,0,95,95,109,97,105,
    110,95,95,40,4,0,0,0,116,3,0,0,0,115,121,115,
    116,2,0,0,0,111,115,82,2,0,0,0,116,8,0,0,
    0,95,95,110,97,109,101,95,95,40,3,0,0,0,82,4,
    0,0,0,82,2,0,0,0,82,5,0,0,0,40,0,0,
    0,0,40,0,0,0,0,82,1,0,0,0,116,1,0,0,
    0,63,3,0,0,0,115,6,0,0,0,18,2,9,4,13,
    1,
    };


    The difference between the two is very slight:

    18c18
    < 0,0,0,0,116,23,0,0,0,47,104,111,109,101,47,118,
    ---
    > 0,0,0,0,116,23,0,0,0,47,87,111,114,107,47,118,
     
    Robert McLay, Oct 28, 2005
    #1
    1. Advertising

  2. Robert McLay

    Tim Peters Guest

    [Robert McLay]
    > I'm trying to understand bytecodes generated on different machines.
    > I understand that the bytecodes can change between version. But since
    > I'm told that .pyc files are version dependent but not machine
    > dependent, I'm wondering why the bytecodes are machine dependent.


    They aren't -- at least not particularly <wink>.

    > my friend and I created this simple example to explore the problem.
    > The example code is:
    >
    > #!/usr/bin/env python
    > #
    > import sys, os
    >
    > def main():
    > x = 1.234
    > print x
    >
    > if ( __name__ == '__main__'):
    > main()
    >
    >
    > Using sib.py from Vendorid 1.0 generates different bytecodes under linux
    > and sgi. At its core sib.py is using:
    >
    > # Compile the code using prefix and marshall it.
    > compiled_code = compile(source_code, prefix, 'exec')
    > marshalled_code = marshal.dumps(compiled_code)
    >
    > to get the bytecodes.


    Why do you believe that `prefix` had the same value in both runs? The
    output suggests it did not, but can't guess more than that from here
    since I don't know where `prefix` came from.

    > So why are the byte codes different? Is it that the intel-linux is
    > little endian and the SGI is big endian


    No; marshal format has fixed endianness.

    > and the numerical constant (1.234) is stored different depending on the
    > endian-ness?


    Python defers to the platform C library for string->float conversions,
    and it's *possible* that different platforms could convert "1.234" to
    a C double in slightly different ways. There's no evidence of that
    here, though.

    > This was generated under intel-linux using python 2.4.2
    >
    >
    >
    >
    > /*==========================================================================*/
    > /* Frozen main script for test */
    > /* Generated from test.py */
    > /* This is generated code; Do not modify it! */
    > /*--------------------------------------------------------------------------*/
    > unsigned char M___main__[] =
    > {
    > 99,0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,
    > 0,115,55,0,0,0,100,0,0,107,0,0,90,0,0,100,
    > 0,0,107,1,0,90,1,0,100,1,0,132,0,0,90,2,
    > 0,101,3,0,100,2,0,106,2,0,111,11,0,1,101,2,
    > 0,131,0,0,1,110,1,0,1,100,0,0,83,40,3,0,
    > 0,0,78,99,0,0,0,0,1,0,0,0,1,0,0,0,
    > 67,0,0,0,115,15,0,0,0,100,1,0,125,0,0,124,
    > 0,0,71,72,100,0,0,83,40,2,0,0,0,78,102,5,
    > 49,46,50,51,52,40,1,0,0,0,116,1,0,0,0,120,
    > 40,1,0,0,0,82,0,0,0,0,40,0,0,0,0,40,
    > 0,0,0,0,116,23,0,0,0,47,104,111,109,101,47,118, /* This line */
    > 112,97,114,114,47,115,105,98,47,116,101,115,116,46,112,121,
    > 116,4,0,0,0,109,97,105,110,5,0,0,0,115,4,0,
    > 0,0,0,1,6,1,116,8,0,0,0,95,95,109,97,105,
    > 110,95,95,40,4,0,0,0,116,3,0,0,0,115,121,115,
    > 116,2,0,0,0,111,115,82,2,0,0,0,116,8,0,0,
    > 0,95,95,110,97,109,101,95,95,40,3,0,0,0,82,4,
    > 0,0,0,82,2,0,0,0,82,5,0,0,0,40,0,0,
    > 0,0,40,0,0,0,0,82,1,0,0,0,116,1,0,0,
    > 0,63,3,0,0,0,115,6,0,0,0,18,2,9,4,13,
    > 1,
    > }; x
    >
    > And under SGI (python 2.4.2) it created :
    >
    > /*==========================================================================*/
    > /* Frozen main script for test */
    > /* Generated from test.py */
    > /* This is generated code; Do not modify it! */
    > /*--------------------------------------------------------------------------*/
    > unsigned char M___main__[] =
    > {
    > 99,0,0,0,0,0,0,0,0,2,0,0,0,64,0,0,
    > 0,115,55,0,0,0,100,0,0,107,0,0,90,0,0,100,
    > 0,0,107,1,0,90,1,0,100,1,0,132,0,0,90,2,
    > 0,101,3,0,100,2,0,106,2,0,111,11,0,1,101,2,
    > 0,131,0,0,1,110,1,0,1,100,0,0,83,40,3,0,
    > 0,0,78,99,0,0,0,0,1,0,0,0,1,0,0,0,
    > 67,0,0,0,115,15,0,0,0,100,1,0,125,0,0,124,
    > 0,0,71,72,100,0,0,83,40,2,0,0,0,78,102,5,
    > 49,46,50,51,52,40,1,0,0,0,116,1,0,0,0,120,
    > 40,1,0,0,0,82,0,0,0,0,40,0,0,0,0,40,
    > 0,0,0,0,116,23,0,0,0,47,87,111,114,107,47,118, /* This line */
    > 112,97,114,114,47,115,105,98,47,116,101,115,116,46,112,121,
    > 116,4,0,0,0,109,97,105,110,5,0,0,0,115,4,0,
    > 0,0,0,1,6,1,116,8,0,0,0,95,95,109,97,105,
    > 110,95,95,40,4,0,0,0,116,3,0,0,0,115,121,115,
    > 116,2,0,0,0,111,115,82,2,0,0,0,116,8,0,0,
    > 0,95,95,110,97,109,101,95,95,40,3,0,0,0,82,4,
    > 0,0,0,82,2,0,0,0,82,5,0,0,0,40,0,0,
    > 0,0,40,0,0,0,0,82,1,0,0,0,116,1,0,0,
    > 0,63,3,0,0,0,115,6,0,0,0,18,2,9,4,13,
    > 1,
    > };
    >
    >
    > The difference between the two is very slight:
    >
    > 18c18
    > < 0,0,0,0,116,23,0,0,0,47,104,111,109,101,47,118,
    > ---
    > > 0,0,0,0,116,23,0,0,0,47,87,111,114,107,47,118,


    Stare at this:

    >>> ''.join(map(chr, [47,104,111,109,101,47,118]))

    '/home/v'
    >>> ''.join(map(chr, [47,87,111,114,107,47,118]))

    '/Work/v'

    It _suggests_ that `prefix` contained the substring "/home/" on one
    box but ""/Work/" on the other.
     
    Tim Peters, Oct 28, 2005
    #2
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Replies:
    4
    Views:
    5,844
    Michael Borgwardt
    Dec 10, 2004
  2. gk
    Replies:
    13
    Views:
    2,331
    Dijon Yu
    Sep 22, 2006
  3. puzzlecracker
    Replies:
    1
    Views:
    549
    James Kanze
    Aug 7, 2008
  4. Rouslan Korneychuk

    a basic bytecode to machine code compiler

    Rouslan Korneychuk, Mar 31, 2011, in forum: Python
    Replies:
    10
    Views:
    450
    Robert Kern
    Apr 3, 2011
  5. Rouslan Korneychuk

    basic bytecode to machine code compiler (part 2)

    Rouslan Korneychuk, May 18, 2011, in forum: Python
    Replies:
    0
    Views:
    226
    Rouslan Korneychuk
    May 18, 2011
Loading...

Share This Page