marshalled data in 1.9

[Roger Pack]

Is there the possibility this would fail in 1.9?

big = eval(File.read("out_inspect.small"))
File.open("out.marshal", "w") do |f|

f.write(Marshal.dump(big))

end
Marshal.load(File.open('out.marshal', 'r'))


When I do this with large structures (on windows) I get messages like:

bad.rb:7:in `load': dump format error for symbol(0x6c) (ArgumentError)


irb(main):001:0> Encoding.default_external
=> #<Encoding:IBM437>
irb(main):002:0> Encoding.default_internal
=> nil

But I had assumed since I was reading and writing in the same mode it
would work all right. Was I wrong?
-r

 

[Rob Biedenharn]

Is there the possibility this would fail in 1.9?

big = eval(File.read("out_inspect.small"))
File.open("out.marshal", "w") do |f|

f.write(Marshal.dump(big))

end
Marshal.load(File.open('out.marshal', 'r'))


When I do this with large structures (on windows) I get messages like:

bad.rb:7:in `load': dump format error for symbol(0x6c) (ArgumentError)


irb(main):001:0> Encoding.default_external
=> #<Encoding:IBM437>
irb(main):002:0> Encoding.default_internal
=> nil

But I had assumed since I was reading and writing in the same mode it
would work all right. Was I wrong?
-r

You almost certainly want the 'rb' and 'wb' modes on Windows to read
and write in binary, rather than text, mode.

-Rob

Rob Biedenharn
http://agileconsultingllc.com
(e-mail address removed)
http://gaslightsoftware.com
(e-mail address removed)

 

[Roger Pack]

You almost certainly want the 'rb' and 'wb' modes on Windows to read
and write in binary, rather than text, mode.

Hmm. The problem may occur when I read the file in--because I'm not
reading it in Binary mode, I'm actually reading it in as ascii + some
encoding (Encoding.default_external), which is IBM437

I'm still not entirely sure why something like this *shouldn't* round
trip appropriately though.

 

[Brian Candler]

Hmm. The problem may occur when I read the file in--because I'm not
reading it in Binary mode, I'm actually reading it in as ascii + some
encoding (Encoding.default_external), which is IBM437

I'm still not entirely sure why something like this *shouldn't* round
trip appropriately though.

If you're under Windows, ruby/C will translate \r\n to \n on read and
vice versa on write, unless you open the file in binary mode.

 

[Roger Pack]

If you're under Windows, ruby/C will translate \r\n to \n on read and
vice versa on write, unless you open the file in binary mode.

Yes, but if I read and write both in ASCII mode, should it not be
expected to round trip? I'm a bit confused...

-r

 

[Bill Kelly]

Yes, but if I read and write both in ASCII mode, should it not be
expected to round trip? I'm a bit confused...

irb(main):001:0> [RUBY_VERSION, RUBY_PLATFORM]
=> ["1.9.2", "i386-mswin32_100"]
irb(main):002:0> File.open("zz", "w") {|io| io.write "foo\r\nbar\nbaz\n"}
=> 16
irb(main):003:0> File.read("zz")
=> "foo\n\nbar\nbaz\n"


Notice the "\r\n" came back as "\n\n".


Regards,

Bill

 

[Roger Pack]

irb(main):002:0> File.open("zz", "w") {|io| io.write
"foo\r\nbar\nbaz\n"}
=> 16
irb(main):003:0> File.read("zz")
=> "foo\n\nbar\nbaz\n"


Notice the "\r\n" came back as "\n\n".

This feels like a bug to me...

 

[Bill Kelly]

This feels like a bug to me...

If I could pick one statement to summarize my feeling
about developing on Windows, that might be it. ;P

But anyway...

It's not strictly a ruby issue. I recall encountering CRLF
text mode vs. binary mode issues in DOS with Borland C
in the mid 1980's.

But, back to the ruby example above... I don't see
how one could expect the data to survive a round trip.

Note that if we force ruby to deal with a single
character at a time, the results are consistent with
the above:

irb(main):004:0> File.open("zz2", "w") {|io| "foo\r\nbar\nbaz\n".each_char {|c| io.write c; io.flush}}
=> "foo\r\nbar\nbaz\n"

irb(main):005:0> File.open("zz2", "r") {|io| x=c=""; x << c while (c = io.getc); x}
=> "foo\n\nbar\nbaz\n"



So... it's not clear to me how round-trip semantics
could be implemented given the need to consider each
character individually?


Regards,

Bill

 

[Luis Lavena]

If I could pick one statement to summarize my feeling
about developing on Windows, that might be it.  ;P

But anyway...

It's not strictly a ruby issue.  I recall encountering CRLF
text mode vs. binary mode issues in DOS with Borland C
in the mid 1980's.

But, back to the ruby example above... I don't see
how one could expect the data to survive a round trip.

Note that if we force ruby to deal with a single
character at a time, the results are consistent with
the above:

irb(main):004:0> File.open("zz2", "w") {|io| "foo\r\nbar\nbaz\n".each_char {|c| io.write c; io.flush}}
=> "foo\r\nbar\nbaz\n"

irb(main):005:0> File.open("zz2", "r") {|io| x=c=""; x << c while (c = io.getc); x}
=> "foo\n\nbar\nbaz\n"

So... it's not clear to me how round-trip semantics
could be implemented given the need to consider each
character individually?

If you want to specify the new lines yourself, you need to use binary
mode.

Text mode read and write under Windows will do weird things, but is
defined as the "spec" of Ruby behavior.