How to bypass Windows 'cooking' the I/O? (One more time, please)

Discussion in 'Python' started by norseman, Jul 4, 2008.

  1. norseman

    norseman Guest

    I know I saw the answer recently, as in since February '08, but I can't
    re-find it. :( I tried the mail archives and such and my own
    collections but the piece I saw still eludes me.


    Problem: (sos=same old s...) Microsoft insists the world work it's way
    even when the Microsoft way was proven wrong decades ago. In this case
    it's (still) 'cooking' the writes even with 'rwb' and O_RDWR|O_BINARY in
    (proper respective) use.

    Specific: python created and inspected binary file ends:
    00460: 0D 1A (this is correct)

    after a write
    os.lseek(target, -1, 2)
    os.write(target,record)
    the expected result would be:
    00460: 0D 20 .....data bytes.... 1A

    BUT I get:
    00460: 20 .... data bytes... 1A

    It is one byte off!!! And the 0D has to be there. Signifies the end of
    the header.


    Same python program runs as expected in Linux. Maybe because that's
    where it was written?! :)


    What I seek is the way to slap Microsoft up side the head and make it
    work correctly. OK, well, at least in this situation.


    Note: Things like this justify Python implementers bypassing OS calls
    (data fetch, data write) and using the BIOS direct. Remember, the CPU
    understands bit patterns only. It has no comprehension of 'text',
    'program', 'number', 'pointer', blah blah blah.... All that is totally
    beyond it's understanding. A given bit pattern means 'do that'. The CPU
    is 100% binary. Memory, storage and the rest is just bits-on, bits-off.
    Patterns. Proper binary I/O is mandatory for the machine to function.


    Anyway - if whoever mentioned the flags and such to 'over ride'
    Microsoft's BS would re-send that piece I would be very appreciative.


    Steve
    norseman, Jul 4, 2008
    #1
    1. Advertising

  2. norseman

    Tim Roberts Guest

    norseman <> wrote:
    >
    >Problem: (sos=same old s...) Microsoft insists the world work it's way
    >even when the Microsoft way was proven wrong decades ago. In this case
    >it's (still) 'cooking' the writes even with 'rwb' and O_RDWR|O_BINARY in
    >(proper respective) use.


    No, it doesn't. Where did you get the idea that 'rwb' is a valid fopen
    mode? It's not. If you need to read and write an existing file, you want
    "r+b".


    C:\tmp>python
    Python 2.4.4 (#71, Oct 18 2006, 08:34:43) [MSC v.1310 32 bit (Intel)] on
    win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> f = open('x.bat','r+b')
    >>> s = f.read()
    >>> s

    'sed -e "s/[ \\t]*$//" -e "/^$/d" %1\rhow about that\r\n'
    >>> f.seek(-1,2)
    >>> f.write('xxx\r\n')
    >>> f.close()
    >>> f = open('x.bat','rb')
    >>> t = f.read()
    >>> t

    'sed -e "s/[ \\t]*$//" -e "/^$/d" %1\rhow about that\rxxx\r\n'
    >>>


    >Same python program runs as expected in Linux. Maybe because that's
    >where it was written?! :)


    Perhaps your Linux C runtime library accepts the fopen mode 'rwb', but if
    it does, it's a non-standard extension.

    >What I seek is the way to slap Microsoft up side the head and make it
    >work correctly. OK, well, at least in this situation.


    It works correctly if you use it correctly.
    --
    Tim Roberts,
    Providenza & Boekelheide, Inc.
    Tim Roberts, Jul 4, 2008
    #2
    1. Advertising

  3. norseman wrote:
    > In this case it's [MS Windows] (still) 'cooking' the writes even
    > with 'rwb' and O_RDWR|O_BINARY in (proper respective) use.
    >
    > Specific: python created and inspected binary file ends:
    > 00460: 0D 1A (this is correct)


    I'm not actually sure what the 0x1a is supposed to do there. The 0x0d is
    a '\r' which looks like a MacOS line ending, but I also seem to remember
    0x1a being use as EOF signal in some old system's text files. Anyway, you
    mentioned some "binary file" (aren't they all?) so this might not matter
    after all.

    > after a write
    > os.lseek(target, -1, 2)


    That should be os.SEEK_END and not 2. In any case, it moves the write
    position to the last byte (-1 offset to the end of the file).

    > os.write(target,record)
    > the expected result would be:
    > 00460: 0D 20 .....data bytes.... 1A


    I'm sorry, but I don't see that. write() will not insert the bytes but first
    overwrite till EOF and then append further output, i.e. the 0x1a at the end
    is only present if it was present in the record.

    > BUT I get:
    > 00460: 20 .... data bytes... 1A
    >
    > It is one byte off!!! And the 0D has to be there. Signifies the end of
    > the header.


    Okay, this looks strange, both the offset and the fact that it seems to
    insert bytes. How about providing a minimal example?

    > Anyway - if whoever mentioned the flags and such to 'over ride'
    > Microsoft's BS would re-send that piece I would be very appreciative.


    You are not one with the Tao, which is why you programs don't run. ;)

    Uli

    --
    Sator Laser GmbH
    Geschäftsführer: Thorsten Föcking, Amtsgericht Hamburg HR B62 932
    Ulrich Eckhardt, Jul 7, 2008
    #3
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. norseman
    Replies:
    2
    Views:
    384
    norseman
    Jul 7, 2008
  2. Replies:
    6
    Views:
    94
    John W. Kennedy
    Oct 8, 2006
  3. Xeno Campanoli
    Replies:
    1
    Views:
    73
    Martin Boese
    Jul 10, 2008
  4. Smitty

    'cooking' regex's

    Smitty, Nov 22, 2005, in forum: Perl Misc
    Replies:
    15
    Views:
    181
    Euclid Uranium
    Nov 22, 2005
  5. Steven D'Aprano
    Replies:
    0
    Views:
    75
    Steven D'Aprano
    Dec 23, 2013
Loading...

Share This Page