enabling universal newline

[Peter Kleiweg]

In Python 3.1 and 3.2

At start-up, the value of sys.stdin.newlines is None, which
means, universal newline should be enabled. But it isn't.

So I do this:

sys.stdin = io.TextIOWrapper(sys.stdin.detach(), newline=None)

Now, sys.stdin.newlines is still None, but universal newline is
enabled.

Why is this?

 

[Steven D'Aprano]

In Python 3.1 and 3.2

At start-up, the value of sys.stdin.newlines is None, which means,
universal newline should be enabled. But it isn't.

What makes you think it is not enabled?

sys.stdin.newlines shows you the newlines actually seen. Until you put
text including newlines through stdin, it will remain None.

http://docs.python.org/2/library/stdtypes.html#file.newlines
http://docs.python.org/3/library/io.html#io.TextIOBase.newlines

For example, I have Python built with universal newlines, but
stdin.newlines remains None:

py> f = open('test.txt')
py> f.newlines
py> f.readlines()
['a\n', 'b\n', 'c\n', 'd\n']
py> f.newlines
('\r', '\n', '\r\n')
py> sys.stdin.newlines is None
True

 

[Peter Otten]

What makes you think it is not enabled?

$ python3 -c 'open("tmp.txt", "wb").write(b"a\nb\r\nc\rd")'

This is the output with universal newlines:

$ python3 -c 'print(open("tmp.txt").readlines())'
['a\n', 'b\n', 'c\n', 'd']

But this is what you get from stdin:

$ cat tmp.txt | python3 -c 'import sys; print(sys.stdin.readlines())'
['a\n', 'b\r\n', 'c\rd']

With Peter Kleiweg's fix:

$ cat tmp.txt | python3 -c 'import sys, io; print(io.TextIOWrapper(sys.stdin.detach(), newline=None).readlines())'
['a\n', 'b\n', 'c\n', 'd']

I think it's reasonable to make the latter the default.

 

[Peter Kleiweg]

Steven D'Aprano schreef op de 2e dag van de slachtmaand van het jaar 2012:

What makes you think it is not enabled?

Script 1:

#!/usr/bin/env python3.1
import sys
print(sys.stdin.readlines())

Output:

~ test.py < text
['a\rbc\rdef\r']


Script 2:

#!/usr/bin/env python3.1
import io, sys
sys.stdin = io.TextIOWrapper(sys.stdin.detach(), newline=None)
print(sys.stdin.readlines())

Output:

~ test.py < text
['a\n', 'bc\n', 'def\n']