module file length limitations on windows?

Discussion in 'Python' started by Lonnie Princehouse, Dec 16, 2004.

  1. I've run into some eccentric behavior... It appears that one of my
    modules is being cut off at exactly 2^14 characters when I try to
    import it. Has anyone else encountered this? I can't find any mention
    of such a bug, and stranger yet, other modules that exceed 16384
    characters seem to work just fine.

    In particular, suppose that my module foo.py contains the following as
    its last line:

    thing = "goodbye world"

    Now, suppose that the length of the file is 16383 characters. It works
    just fine:

    Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
    win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import foo
    >>>


    But if I make the string longer, it explodes:

    thing = "goodbye world spam spam spam spam spam"

    Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
    win32
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import foo

    Traceback (most recent call last):
    File "<stdin>", line 1, in ?
    File "foo.py", line 583
    thing = "goodbye world sp
    ^
    SyntaxError: EOL while scanning single-quoted string


    What in the world is going on here?!

    This happens with Python 2.4 and 2.3.4 on win2k (under vmware), but it
    does _not_ happen with 2.3.4 on Linux. Very strange! Could vmware be
    the problem?

    I have also tried replacing my unix newlines with DOS \r\n with the
    exact same result.

    I don't want to spend much time on this, since the workaround of
    splitting the code into smaller files works just fine, but wow.. weird.
     
    Lonnie Princehouse, Dec 16, 2004
    #1
    1. Advertising

  2. Lonnie Princehouse

    Steve Holden Guest

    Lonnie Princehouse wrote:

    > I've run into some eccentric behavior... It appears that one of my
    > modules is being cut off at exactly 2^14 characters when I try to
    > import it. Has anyone else encountered this? I can't find any mention
    > of such a bug, and stranger yet, other modules that exceed 16384
    > characters seem to work just fine.
    >
    > In particular, suppose that my module foo.py contains the following as
    > its last line:
    >
    > thing = "goodbye world"
    >
    > Now, suppose that the length of the file is 16383 characters. It works
    > just fine:
    >
    > Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
    > win32
    > Type "help", "copyright", "credits" or "license" for more information.
    >
    >>>>import foo
    >>>>

    >
    >
    > But if I make the string longer, it explodes:
    >
    > thing = "goodbye world spam spam spam spam spam"
    >
    > Python 2.4 (#60, Nov 30 2004, 11:49:19) [MSC v.1310 32 bit (Intel)] on
    > win32
    > Type "help", "copyright", "credits" or "license" for more information.
    >
    >>>>import foo

    >
    > Traceback (most recent call last):
    > File "<stdin>", line 1, in ?
    > File "foo.py", line 583
    > thing = "goodbye world sp
    > ^
    > SyntaxError: EOL while scanning single-quoted string
    >
    >
    > What in the world is going on here?!
    >
    > This happens with Python 2.4 and 2.3.4 on win2k (under vmware), but it
    > does _not_ happen with 2.3.4 on Linux. Very strange! Could vmware be
    > the problem?
    >
    > I have also tried replacing my unix newlines with DOS \r\n with the
    > exact same result.
    >
    > I don't want to spend much time on this, since the workaround of
    > splitting the code into smaller files works just fine, but wow.. weird.
    >

    As a further data point, it doesn't appear to happen under Cygwin with
    2.4 either:

    sholden@dellboy ~
    $ wc module.py
    8177 2 8191 module.py

    sholden@dellboy ~
    $ python
    Python 2.4 (#1, Dec 4 2004, 20:10:33)
    [GCC 3.3.3 (cygwin special)] on cygwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import module

    hello
    >>>


    sholden@dellboy ~
    $ vi module.py

    sholden@dellboy ~
    $ wc module.py
    8177 5 8206 module.py

    sholden@dellboy ~
    $ python
    Python 2.4 (#1, Dec 4 2004, 20:10:33)
    [GCC 3.3.3 (cygwin special)] on cygwin
    Type "help", "copyright", "credits" or "license" for more information.
    >>> import module

    hello spam spam spam

    Is it possible you've somehow inserted non-printing characters? Seems
    bizarrely improbable to me that 2^14 would be a significant boundary to
    the interpreter.

    regards
    Steve
    --
    Steve Holden http://www.holdenweb.com/
    Python Web Programming http://pydish.holdenweb.com/
    Holden Web LLC +1 703 861 4237 +1 800 494 3119
     
    Steve Holden, Dec 16, 2004
    #2
    1. Advertising

  3. No non-printing characters.

    However, I just tried copying the file (from a windows cmd prompt), and
    the copy was cut off at the same point the interpreter is getting to.
    When I edit the file with vim, though, the whole thing comes through.
    I think this is a pretty strong indication that this monkey-business is
    happening either in Windows or in vmware, and has nothing to do with
    Python.

    Confirmed. I rebooted the (virtual) machine and the problem is gone.
    Weird.
     
    Lonnie Princehouse, Dec 16, 2004
    #3
  4. ElementTree.write() question

    [If there is a separate list for elementtree, please someone
    clue me ... I didn't see one.]

    Fredrik or other xml / elementtree gurus:

    I see from the source that ElementTree.write() writes

    <?xml version="1.0"? encoding=[whatever]>

    at the beginning of the xml output if an encoding
    other than utf-8 or us-ascii is selected. Shouldn't
    it also write that if utf-8 or us-ascii is being
    used? Or at least the

    <?xml version="1.0"?>

    ?

    Thanks,
    Steve
     
    Stephen Waterbury, Dec 16, 2004
    #4
  5. Re: ElementTree.write() question

    Stephen Waterbury wrote:

    > [If there is a separate list for elementtree, please someone
    > clue me ... I didn't see one.]


    the xml-sig is preferred, I think.

    > Fredrik or other xml / elementtree gurus:
    >
    > I see from the source that ElementTree.write() writes
    >
    > <?xml version="1.0"? encoding=[whatever]>
    >
    > at the beginning of the xml output if an encoding
    > other than utf-8 or us-ascii is selected. Shouldn't
    > it also write that if utf-8 or us-ascii is being
    > used? Or at least the
    >
    > <?xml version="1.0"?>


    this is mostly for historical reasons; early versions only supported UTF-8 output,
    and never generated encoding directives, so people ended up writing:

    print >>myfile, "<document>"
    for elem in list_of_elements:
    ElementTree(elem).write(myfile)
    print >>myfile, "</document>"

    version 1.3 will probably include an option to always write the header.

    (note that the XML header isn't needed for UTF-8 and ASCII; if it's not there,
    a proper XML parser will treat the stream as UTF-8, and ASCII is a compatible
    subset of UTF-8).

    </F>
     
    Fredrik Lundh, Dec 17, 2004
    #5
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Mitchua
    Replies:
    5
    Views:
    2,781
    Eric J. Roode
    Jul 17, 2003
  2. =?Utf-8?B?SG96aQ==?=
    Replies:
    1
    Views:
    6,989
    Ken Cox [Microsoft MVP]
    Jun 2, 2004
  3. Robert Strickland

    File Upload and File Size Limitations

    Robert Strickland, Oct 29, 2004, in forum: ASP .Net
    Replies:
    6
    Views:
    4,687
    Lau Lei Cheong
    Nov 1, 2004
  4. Angus Parvis
    Replies:
    4
    Views:
    1,114
    janne
    Nov 23, 2004
  5. Corey_G

    limitations of forking on windows

    Corey_G, Jun 28, 2004, in forum: Perl Misc
    Replies:
    47
    Views:
    374
    MichiganBob
    Jul 3, 2004
Loading...

Share This Page