windows line termination

[Chris Morris]

Why does Ruby translate \n -> \r\n automagically when writing to/from
file? Convenience, I assume, so I don't have to worry with this. But
I've run into some wackiness regarding this.

I'm uploading a file from a ruby script to a ruby cgi script. cgi.rb
when reading multipart assumes that \r\n will exist as terminators in
the contents. My client.rb reads in a file, Ruby translates the \r\n in
the file to \n -- I then have to retranslate this back to \r\n prior to
uploading to the cgi script, so the cgi will recognize the line
terminators properly. Then my cgi writes the contents of the uploaded
file to a local file, still translating \n -> \r\n so \r\n in the
uploaded content becomes \r\r\n when written to disk.

Argh. I don't know what I'm asking for, but I needed to rant. Is
there a better way? Or is this just a weird case because of the specific
line terminator cgi.rb multiform reader code is using?

D:\temp>more endlinetest.rb
s = "hey\r\n"
File.open('endlinetest.txt', 'w+') do |f| f.print s end
s_read = ''
File.open('endlinetest.txt', 'r') do |f| s_read = f.read end
p s_read

D:\temp>endlinetest.rb
"hey\r\n"

D:\temp>w2k_dump.exe endlinetest.txt

// w2k_dump.exe
// SBS Windows 2000 Hex Dump Utility V1.00
// 08-27-2000 Sven B. Schreiber
// (e-mail address removed)

File: endlinetest.txt
Size: 0x00000006 (6)

Address | 00 01 02 03-04 05 06 07 : 08 09 0A 0B-0C 0D 0E 0F |
0123456789ABCDEF
---------|-------------------------:-------------------------|-----------------
00000000 | 68 65 79 0D-0D 0A : - | hey...

 

[ts]

try it with the `b' switch

C> D:\temp>more endlinetest.rb
C> s = "hey\r\n"
C> File.open('endlinetest.txt', 'w+') do |f| f.print s end

File.open('endlinetest.txt', 'wb+') do |f| f.print s end

C> s_read = ''
C> File.open('endlinetest.txt', 'r') do |f| s_read = f.read end

File.open('endlinetest.txt', 'rb') do |f| s_read = f.read end

C> p s_read


Guy Decoux

 

[Chris Morris]

try it with the `b' switch

Dang it - forgot about that *again*. Thanks for the reminder

Still don't know if that quells my rant. I assume that the default \n is
used in memory at all times, and Ruby will handle the appropriate
translation for *nix/Windows/Mac, freeing me from worrying over that?
And the case I stumbled upon today was a leaky abstraction?

 

[Xavier Noria]

Still don't know if that quells my rant. I assume that the default \n
is used in memory at all times, and Ruby will handle the appropriate
translation for *nix/Windows/Mac, freeing me from worrying over that?
And the case I stumbled upon today was a leaky abstraction?

When you read/write a text file Ruby (as C or Perl) lets you see a
newline as "\n", a single character. Unless you do I/O in binmode that
happens transparently behind the scenes, and in some platforms as Unix
nothing needs to be done indeed.

That way one can easily write text-oriented portable code, print "\n"
will do the right thing anywhere.

The problem you faced has to do with the HTTP specs. HTTP headers, for
instance, have to be separated by CRLF, that is \015\012 because that's
required. So if you wrote an HTTP server written in Ruby with code like

socket.print("Location: ${location}\n") # broken

that server wouldn't be portable. In such cases you have to manually
ensure that the CRLF travels well, untouched. And that usually means
you put the handle in binmode (which turns off that transalation) and
print exactly what's needed.

The newlines for the body of an HTTP message with text (i.e., a normal
HTML page) need not be CRLF, but in the kind of data you were uploading
that's required (in my understanding of the RFC).

-- fxn

 

[Chris Morris]

Xavier Noria wrote:

Thanks, Xavier -- guess I just needed the confirmation to make sure I
wasn't missing something