getdefaultencoding - how to change this?

[Helmut Jarausch]

Hi,
I've searched the net but didn't find the information I need.
Using Python-2.7.1, I know, I can't modify defaultencoding at run time.
Python even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

still sys.stdout is using the ascii codec.
How can I recompile Python (itself) to change this to iso8859-1 ?
(My favourite editor cannot be told to use unicode.)

Many thanks for a hint,
Helmut.

 

[Helmut Jarausch]

Hi,
I've searched the net but didn't find the information I need. Using
Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

still sys.stdout is using the ascii codec. How can I recompile Python
(itself) to change this to iso8859-1 ? (My favourite editor cannot be
told to use unicode.)

Sorry for answering myself. One last trial did succeed.
My locale as root differed from my locale as non-root user.
After changing the root locale and recompiling Python, it works now.





--

 

[Philip Semanchuk]

Hi,
I've searched the net but didn't find the information I need. Using
Python-2.7.1, I know, I can't modify defaultencoding at run time. Python
even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

still sys.stdout is using the ascii codec. How can I recompile Python
(itself) to change this to iso8859-1 ? (My favourite editor cannot be
told to use unicode.)

Sorry for answering myself. One last trial did succeed.
My locale as root differed from my locale as non-root user.
After changing the root locale and recompiling Python, it works now.

I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.


Cheers
Philip

 

[Robert Kern]

Hi,
I've searched the net but didn't find the information I need.
Using Python-2.7.1, I know, I can't modify defaultencoding at run time.

You're not supposed to. It must remain 'ascii'. Otherwise, you will break dict
lookups among other things. Can you be specific about why you think you want to
change this? What are you trying to achieve? It looks like you are conflating a
variety of different behaviors below.

Python even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

This does not introspect the same thing as sys.getdefaultencoding().

still sys.stdout is using the ascii codec.

This reflects your terminal settings, not sys.getdefaultencoding(). I'm not sure
why the PYTHONIOENCODING variable isn't affecting that. It works fine for me.

How can I recompile Python (itself) to change this to iso8859-1 ?
(My favourite editor cannot be told to use unicode.)

You wouldn't want to change sys.getdefaultencoding() for this. That parameter
affects how str and unicode objects are converted between each other without an
explicit .encode()/.decode() call, not what encoding Python assumes for the
parsing of the code.

Instead, you want to use an encoding declaration in each file:

http://docs.python.org/reference/lexical_analysis.html#encoding-declarations

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Robert Kern]

I'm glad that worked for you. Alternatively, it seems like you can set the default encoding in site.py which sounds easier than recompiling Python.

Never do that. It breaks dicts and makes your code non-portable.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco

 

[Philip Semanchuk]

Never do that. It breaks dicts and makes your code non-portable.

I've never been tempted for the very non-portability reason you cite. I didn't know that it would break dicts, though. Thanks for the education.

bye
Philip

 

[Helmut Jarausch]

Hi,
I've searched the net but didn't find the information I need. Using
Python-2.7.1, I know, I can't modify defaultencoding at run time.

You're not supposed to. It must remain 'ascii'. Otherwise, you will
break dict lookups among other things. Can you be specific about why you
think you want to change this? What are you trying to achieve? It looks
like you are conflating a variety of different behaviors below.

Python even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

This does not introspect the same thing as sys.getdefaultencoding().

still sys.stdout is using the ascii codec.

This reflects your terminal settings, not sys.getdefaultencoding(). I'm
not sure why the PYTHONIOENCODING variable isn't affecting that. It
works fine for me.

How can I recompile Python (itself) to change this to iso8859-1 ? (My
favourite editor cannot be told to use unicode.)

You wouldn't want to change sys.getdefaultencoding() for this. That
parameter affects how str and unicode objects are converted between each
other without an explicit .encode()/.decode() call, not what encoding
Python assumes for the parsing of the code.

Instead, you want to use an encoding declaration in each file:

http://docs.python.org/reference/lexical_analysis.html#encoding-

declarations

Thanks Robert,
probably I wasn't too clear about my issue.
I couldn't "print" any non-ascii character to my console although
my console has an en_US.iso88591 locale.

I didn't modfiy anything in Python's source code.
I noticed that my root account still had an ASCII locale.
Now I have changed it such that the root account has the same locale
as non-root accounts. Recompiling Python under such a root account
has rectified things. Now I can print non-ascii characters if they are
properly encoded.

Thanks,
Helmut.




--

 

[Antoine Pitrou]

Thanks Robert,
probably I wasn't too clear about my issue.
I couldn't "print" any non-ascii character to my console although
my console has an en_US.iso88591 locale.

Well, if you want a correct answer, you should paste actual code as
well as its result on your system, rather than start by asking how to
change getdefaultencoding (which you shouldn't do as explained by
others).

I didn't modfiy anything in Python's source code.
I noticed that my root account still had an ASCII locale.
Now I have changed it such that the root account has the same locale
as non-root accounts. Recompiling Python under such a root account
has rectified things.

Recompiling Python doesn't change its behaviour wrt. locales and
encodings. All this is done at runtime.

Now I can print non-ascii characters if they are
properly encoded.

You can *always* print characters if they are properly encoded. What
you are asking is for Python to guess and do the encoding by itself,
which is a different matter (and a poorly supported one under 2.x;
Python 3 behaves much better in that regard).

Regards

Antoine.

 

[John Pinner]

Instead, you want to use an encoding declaration in each file:

http://docs.python.org/reference/lexical_analysis.html#encoding-decla...

All that this does is tell the interpreter how the source file is
encoded, it does not affect default encodings etc.

John
--

 

[John Pinner]

To answer the OP's original question:

Hi,
I've searched the net but didn't find the information I need.
Using Python-2.7.1, I know, I can't modify defaultencoding at run time.

I think you can. There is a function setdefaultencoding in the sys
module, but at startup when site.py runs the function gets deleted
after it has been used, because as others have said it is a *bad* idea
to change the default encoding (although I *think* changing it to utf8
should cause no harm).

so if you reload sys, setdefaultencoding() becomes available again,
and you can use it, with all the caveats mentioned:

import sys
reload(sys)
sys.setdefaultencoding( 'utf-8' )

This works on a Unicode build of Python only.

As has been said, you can change the default encoding in site.py, but
this means that it gets changed for everyone/every Python program on
the system, using setdefaultencoding() as above only changes it for
the running program, and hopefully the change will have been made by
someone who knows what they are doing and is aware of the possible
consequences.

Python even ignores
export PYTHONIOENCODING=ISO8859-1

locale.getdefaultlocale()[1]
returns
'ISO8859-1'

still sys.stdout is using the ascii codec.
How can I recompile Python (itself) to change this to iso8859-1 ?

You don't need to, nor should you.

(My favourite editor cannot be told to use unicode.)

Maybe you need a new editor. scite works well with Python, and is
cross-platform.

John
--

 

[Robert Kern]

All that this does is tell the interpreter how the source file is
encoded, it does not affect default encodings etc.

Yes! In the part of the OP's message that you snipped "(My favourite editor
cannot be told to use unicode.)", that seemed to be part of his actual problem,
not the default encoding.

--
Robert Kern

"I have come to believe that the whole world is an enigma, a harmless enigma
that is made terrible by our own mad attempt to interpret it as though it had
an underlying truth."
-- Umberto Eco