getdefaultencoding - how to change this?

Discussion in 'Python' started by Helmut Jarausch, Jan 20, 2011.

  1. Hi,
    I've searched the net but didn't find the information I need.
    Using Python-2.7.1, I know, I can't modify defaultencoding at run time.
    Python even ignores
    export PYTHONIOENCODING=ISO8859-1

    locale.getdefaultlocale()[1]
    returns
    'ISO8859-1'

    still sys.stdout is using the ascii codec.
    How can I recompile Python (itself) to change this to iso8859-1 ?
    (My favourite editor cannot be told to use unicode.)

    Many thanks for a hint,
    Helmut.
     
    Helmut Jarausch, Jan 20, 2011
    #1
    1. Advertising

  2. On Thu, 20 Jan 2011 14:31:09 +0000, Helmut Jarausch wrote:

    > Hi,
    > I've searched the net but didn't find the information I need. Using
    > Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
    > even ignores
    > export PYTHONIOENCODING=ISO8859-1
    >
    > locale.getdefaultlocale()[1]
    > returns
    > 'ISO8859-1'
    >
    > still sys.stdout is using the ascii codec. How can I recompile Python
    > (itself) to change this to iso8859-1 ? (My favourite editor cannot be
    > told to use unicode.)
    >


    Sorry for answering myself. One last trial did succeed.
    My locale as root differed from my locale as non-root user.
    After changing the root locale and recompiling Python, it works now.





    --
     
    Helmut Jarausch, Jan 20, 2011
    #2
    1. Advertising

  3. On Jan 20, 2011, at 10:39 AM, Helmut Jarausch wrote:

    > On Thu, 20 Jan 2011 14:31:09 +0000, Helmut Jarausch wrote:
    >
    >> Hi,
    >> I've searched the net but didn't find the information I need. Using
    >> Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
    >> even ignores
    >> export PYTHONIOENCODING=ISO8859-1
    >>
    >> locale.getdefaultlocale()[1]
    >> returns
    >> 'ISO8859-1'
    >>
    >> still sys.stdout is using the ascii codec. How can I recompile Python
    >> (itself) to change this to iso8859-1 ? (My favourite editor cannot be
    >> told to use unicode.)
    >>

    >
    > Sorry for answering myself. One last trial did succeed.
    > My locale as root differed from my locale as non-root user.
    > After changing the root locale and recompiling Python, it works now.


    I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.


    Cheers
    Philip
     
    Philip Semanchuk, Jan 20, 2011
    #3
  4. Helmut Jarausch

    Robert Kern Guest

    On 1/20/11 8:31 AM, Helmut Jarausch wrote:
    > Hi,
    > I've searched the net but didn't find the information I need.
    > Using Python-2.7.1, I know, I can't modify defaultencoding at run time.


    You're not supposed to. It must remain 'ascii'. Otherwise, you will break dict
    lookups among other things. Can you be specific about why you think you want to
    change this? What are you trying to achieve? It looks like you are conflating a
    variety of different behaviors below.

    > Python even ignores
    > export PYTHONIOENCODING=ISO8859-1
    >
    > locale.getdefaultlocale()[1]
    > returns
    > 'ISO8859-1'


    This does not introspect the same thing as sys.getdefaultencoding().

    > still sys.stdout is using the ascii codec.


    This reflects your terminal settings, not sys.getdefaultencoding(). I'm not sure
    why the PYTHONIOENCODING variable isn't affecting that. It works fine for me.

    > How can I recompile Python (itself) to change this to iso8859-1 ?
    > (My favourite editor cannot be told to use unicode.)


    You wouldn't want to change sys.getdefaultencoding() for this. That parameter
    affects how str and unicode objects are converted between each other without an
    explicit .encode()/.decode() call, not what encoding Python assumes for the
    parsing of the code.

    Instead, you want to use an encoding declaration in each file:

    http://docs.python.org/reference/lexical_analysis.html#encoding-declarations

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Jan 20, 2011
    #4
  5. Helmut Jarausch

    Robert Kern Guest

    On 1/20/11 9:47 AM, Philip Semanchuk wrote:

    > I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.


    Never do that. It breaks dicts and makes your code non-portable.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Jan 20, 2011
    #5
  6. On Jan 20, 2011, at 11:47 AM, Robert Kern wrote:

    > On 1/20/11 9:47 AM, Philip Semanchuk wrote:
    >
    >> I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.

    >
    > Never do that. It breaks dicts and makes your code non-portable.


    I've never been tempted for the very non-portability reason you cite. I didn't know that it would break dicts, though. Thanks for the education.

    bye
    Philip
     
    Philip Semanchuk, Jan 20, 2011
    #6
  7. On Thu, 20 Jan 2011 10:46:37 -0600, Robert Kern wrote:

    > On 1/20/11 8:31 AM, Helmut Jarausch wrote:
    >> Hi,
    >> I've searched the net but didn't find the information I need. Using
    >> Python-2.7.1, I know, I can't modify defaultencoding at run time.

    >
    > You're not supposed to. It must remain 'ascii'. Otherwise, you will
    > break dict lookups among other things. Can you be specific about why you
    > think you want to change this? What are you trying to achieve? It looks
    > like you are conflating a variety of different behaviors below.
    >
    >> Python even ignores
    >> export PYTHONIOENCODING=ISO8859-1
    >>
    >> locale.getdefaultlocale()[1]
    >> returns
    >> 'ISO8859-1'

    >
    > This does not introspect the same thing as sys.getdefaultencoding().
    >
    >> still sys.stdout is using the ascii codec.

    >
    > This reflects your terminal settings, not sys.getdefaultencoding(). I'm
    > not sure why the PYTHONIOENCODING variable isn't affecting that. It
    > works fine for me.
    >
    >> How can I recompile Python (itself) to change this to iso8859-1 ? (My
    >> favourite editor cannot be told to use unicode.)

    >
    > You wouldn't want to change sys.getdefaultencoding() for this. That
    > parameter affects how str and unicode objects are converted between each
    > other without an explicit .encode()/.decode() call, not what encoding
    > Python assumes for the parsing of the code.
    >
    > Instead, you want to use an encoding declaration in each file:
    >
    > http://docs.python.org/reference/lexical_analysis.html#encoding-

    declarations

    Thanks Robert,
    probably I wasn't too clear about my issue.
    I couldn't "print" any non-ascii character to my console although
    my console has an en_US.iso88591 locale.

    I didn't modfiy anything in Python's source code.
    I noticed that my root account still had an ASCII locale.
    Now I have changed it such that the root account has the same locale
    as non-root accounts. Recompiling Python under such a root account
    has rectified things. Now I can print non-ascii characters if they are
    properly encoded.

    Thanks,
    Helmut.




    --
     
    Helmut Jarausch, Jan 20, 2011
    #7
  8. On 20 Jan 2011 17:20:14 GMT
    Helmut Jarausch <> wrote:
    > Thanks Robert,
    > probably I wasn't too clear about my issue.
    > I couldn't "print" any non-ascii character to my console although
    > my console has an en_US.iso88591 locale.


    Well, if you want a correct answer, you should paste actual code as
    well as its result on your system, rather than start by asking how to
    change getdefaultencoding (which you shouldn't do as explained by
    others).

    > I didn't modfiy anything in Python's source code.
    > I noticed that my root account still had an ASCII locale.
    > Now I have changed it such that the root account has the same locale
    > as non-root accounts. Recompiling Python under such a root account
    > has rectified things.


    Recompiling Python doesn't change its behaviour wrt. locales and
    encodings. All this is done at runtime.

    > Now I can print non-ascii characters if they are
    > properly encoded.


    You can *always* print characters if they are properly encoded. What
    you are asking is for Python to guess and do the encoding by itself,
    which is a different matter (and a poorly supported one under 2.x;
    Python 3 behaves much better in that regard).

    Regards

    Antoine.
     
    Antoine Pitrou, Jan 20, 2011
    #8
  9. Helmut Jarausch

    John Pinner Guest

    John Pinner, Jan 21, 2011
    #9
  10. Helmut Jarausch

    John Pinner Guest

    To answer the OP's original question:

    On Jan 20, 2:31 pm, Helmut Jarausch <> wrote:
    > Hi,
    > I've searched the net but didn't find the information I need.
    > Using Python-2.7.1, I know, I can't modify defaultencoding at run time.


    I think you can. There is a function setdefaultencoding in the sys
    module, but at startup when site.py runs the function gets deleted
    after it has been used, because as others have said it is a *bad* idea
    to change the default encoding (although I *think* changing it to utf8
    should cause no harm).

    so if you reload sys, setdefaultencoding() becomes available again,
    and you can use it, with all the caveats mentioned:

    import sys
    reload(sys)
    sys.setdefaultencoding( 'utf-8' )

    This works on a Unicode build of Python only.

    As has been said, you can change the default encoding in site.py, but
    this means that it gets changed for everyone/every Python program on
    the system, using setdefaultencoding() as above only changes it for
    the running program, and hopefully the change will have been made by
    someone who knows what they are doing and is aware of the possible
    consequences.

    > Python even ignores
    > export PYTHONIOENCODING=ISO8859-1
    >
    > locale.getdefaultlocale()[1]
    > returns
    > 'ISO8859-1'
    >
    > still sys.stdout is using the ascii codec.
    > How can I recompile Python (itself) to change this to iso8859-1 ?


    You don't need to, nor should you.

    > (My favourite editor cannot be told to use unicode.)


    Maybe you need a new editor. scite works well with Python, and is
    cross-platform.

    John
    --
     
    John Pinner, Jan 21, 2011
    #10
  11. Helmut Jarausch

    Robert Kern Guest

    On 1/21/11 5:43 AM, John Pinner wrote:
    > On Jan 20, 4:46 pm, Robert Kern<> wrote:
    >
    > <snips>
    >
    >>
    >> Instead, you want to use an encoding declaration in each file:
    >>
    >> http://docs.python.org/reference/lexical_analysis.html#encoding-decla...

    >
    > All that this does is tell the interpreter how the source file is
    > encoded, it does not affect default encodings etc.


    Yes! In the part of the OP's message that you snipped "(My favourite editor
    cannot be told to use unicode.)", that seemed to be part of his actual problem,
    not the default encoding.

    --
    Robert Kern

    "I have come to believe that the whole world is an enigma, a harmless enigma
    that is made terrible by our own mad attempt to interpret it as though it had
    an underlying truth."
    -- Umberto Eco
     
    Robert Kern, Jan 21, 2011
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Q. John Chen
    Replies:
    0
    Views:
    359
    Q. John Chen
    Nov 15, 2006
  2. Q. John Chen
    Replies:
    0
    Views:
    319
    Q. John Chen
    Nov 15, 2006
  3. Suganya
    Replies:
    0
    Views:
    475
    Suganya
    Apr 29, 2008
  4. Hemant

    Change Framework 3.5 to 2.0 change

    Hemant, Nov 13, 2009, in forum: ASP .Net
    Replies:
    5
    Views:
    1,738
    Digvijay Singh
    Nov 16, 2009
  5. tom c
    Replies:
    1
    Views:
    218
    PeterKellner
    Jun 8, 2006
Loading...

Share This Page