\r disappear even in binary mode

[noone]

Hi,

I wrote the following perl script test.pl which generates another perl
script:

binmode(STDOUT);
print "binmode(STDOUT);print \'\r\n\';"

When I run the generated perl script, it only prints the \n:

$ perl test.pl | perl - | hexdump -c
0000000 \n
0000001

Is it a bug or a feature? How to avoid it?

I use perl v5.8.0 built for i386-linux-thread-multi.

Thanks,

David.

 

[pkent]

I wrote the following perl script test.pl which generates another perl
script:

binmode(STDOUT);
print "binmode(STDOUT);print \'\r\n\';"

When I run the generated perl script, it only prints the \n:

$ perl test.pl | perl - | hexdump -c
0000000 \n
0000001

Is it a bug or a feature? How to avoid it?

It's doing the right thing. Literal CRs are skipped when perl reads the
source under some conditions, including this condition:

http://www.perldoc.com/perl5.005_03/pod/perldelta.html#More-generous-trea
tment-of-carriage-returns

( http://shrunk.net/?c=9801d4dd )


If you want the perl-program-that-was-generated to produce a CR on _its_
STDOUT you need:

#!/usr/local/bin/perl -w
use strict;
binmode(STDOUT);
print 'binmode(STDOUT);print "\r\n";';

and if I do:
perl x.pl | perl - | hexdump

I get:

00000000 : 0D 0A

which is what you want, I think.

P

 

[noone]

Hi P,

If you want the perl-program-that-was-generated to produce a CR on _its_
STDOUT you need:

#!/usr/local/bin/perl -w
use strict;
binmode(STDOUT);
print 'binmode(STDOUT);print "\r\n";';

and if I do:
perl x.pl | perl - | hexdump

I get:

00000000 : 0D 0A

which is what you want, I think.

Thank you for your suggestion but I need a single-quoted string ('\r\n').
Indeed, the program I am writing is more complicate than the toy example I gave.
It does sth like that:

print "binmode(STDOUT);print \'",$buff,"\';"

And $buff might contain non backslashed double quotes or other special
characters that would be interpreted in a double-quote string.

David.

 

[David K. Wall]

Thank you for your suggestion but I need a single-quoted string
('\r\n'). Indeed, the program I am writing is more complicate than the
toy example I gave. It does sth like that:

print "binmode(STDOUT);print \'",$buff,"\';"

And $buff might contain non backslashed double quotes or other special
characters that would be interpreted in a double-quote string.

I don't know what you're trying to do.... You might look in perlop under
quote and quote-like operators, and possibly at the quotemeta() function.

For example:

my $buff = '\r\n';
print qq{binmode(STDOUT); print '$buff';};

 

[noone]

I don't know what you're trying to do.... You might look in perlop under
quote and quote-like operators, and possibly at the quotemeta() function.

Thank you but I cannot find any answer there. It seems it is a @$#%!^&
feature of perl: literal carriage returns are ignored if they occur paired
with linefeeds, or get interpreted as whitespace if they stand alone!

David.

 

[Uri Guttman]

Thank you but I cannot find any answer there. It seems it is a @$#%!^&
feature of perl: literal carriage returns are ignored if they occur paired
with linefeeds, or get interpreted as whitespace if they stand alone!

why don't you explain your real problem instead of the solution you
want. i smell an xy problem here. you are struggling too hard with the
only solution you think you have but we don't know what the underlying
goal really is.

uri

 

[noone]

why don't you explain your real problem instead of the solution you
want. i smell an xy problem here. you are struggling too hard with the
only solution you think you have but we don't know what the underlying
goal really is.

In fact I found by myself another solution for my real problem.

However, I think that the feature introduced from perl 5.005
( http://shrunk.net/?c=9801d4dd ) which consists in ignoring literal
carriage returns is too drastic.

David.

 

[Uri Guttman]

In fact I found by myself another solution for my real problem.

so at least state it and your solution.

However, I think that the feature introduced from perl 5.005
( http://shrunk.net/?c=9801d4dd ) which consists in ignoring literal
carriage returns is too drastic.

it was prolly because of the moving of source files from winblows to
unix and back. it makes sense from that perspective. you must be the
first to complain about it and until i know your real problem, i don't
sympathise with your complaint.

uri

 

[David K. Wall]

In fact I found by myself another solution for my real problem.

However, I think that the feature introduced from perl 5.005
( http://shrunk.net/?c=9801d4dd ) which consists in ignoring literal
carriage returns is too drastic.

Would you mind telling us more about the problem and its solution? Someone
else might benefit from the solution to your problem. There have been plenty
of occasions when I've found an answer or a pointer to an answer in the
Google (formerly Dejanews) archives.

 

[Bart Lateur]

I wrote the following perl script test.pl which generates another perl
script:

binmode(STDOUT);
print "binmode(STDOUT);print \'\r\n\';"

When I run the generated perl script, it only prints the \n:

$ perl test.pl | perl - | hexdump -c
0000000 \n
0000001

Is it a bug or a feature? How to avoid it?

You forgot to show the generated script. (indented

binmode(STDOUT);print '
';

This is, considered in text mode, just plain (DOS) text.

If you want to keep the symbolic representation, swap the quote types:

binmode(STDOUT);
print 'binmode(STDOUT);print "\r\n";'

which looks like:

binmode(STDOUT);print "\r\n";
which now will print both the CR and the LF.

 

[noone]

it was prolly because of the moving of source files from winblows to
unix and back. it makes sense from that perspective. you must be the
first to complain about it and until i know your real problem, i don't
sympathise with your complaint.

Well, suppose you want to write a program that takes a binary file as
input and generates a perl script which output this binary. The simplest
generated perl script would be sth like:

print '...'

where ... is the content of the binary file. But if the binary file
contains \r then you are in trouble because perl will ignore them when you
execute the generated perl script!

Of course it is easy to generate more complicate scripts that get around
this problem but...

Now do you sympathise? ^_^

David.

 

[David K. Wall]

Well, suppose you want to write a program that takes a binary file as
input and generates a perl script which output this binary.

Then I'd probably use MIME::Base64 in some fashion.

Now do you sympathise? ^_^

Mmmm.... no? The basic problem is already solved. This is just a variation.

Come to think of it, I was digging through old files a few days ago and found
an old Basic program (a text file) that produced a uudecode.com file intended
to run under MS-DOS 3. I found it useful years ago (10+). That program used a
DATA section to store a representation of the binary that it decoded and
wrote to a file, just like a Perl program would do. (If I recall correctly,
Perl copied the DATA section idea from Basic as an idea too useful to pass up
-- but I could be mistaken)

 

[Anno Siegel]

Come to think of it, I was digging through old files a few days ago and found
an old Basic program (a text file) that produced a uudecode.com file intended
to run under MS-DOS 3. I found it useful years ago (10+). That program used a
DATA section to store a representation of the binary that it decoded and
wrote to a file, just like a Perl program would do. (If I recall correctly,
Perl copied the DATA section idea from Basic as an idea too useful to pass up
-- but I could be mistaken)

Well, Basic only did what dozens of JCLs (job control language) of the day
did -- make parts of the input stream available to the program as data,
usually as a file. JCLs were proto-scripting languages, without much
flow control, but a lot of system interfacing, including file control,
was done on that level.

Basic was unusual in that it handled some of that on the language level,
and Perl followed Basic that way.

Anno

 

[noone]

Then I'd probably use MIME::Base64 in some fashion.

But I have no reason to use any encoding such as Base64. It is only
useful if you send data by mail or in newsgroups. Moreover it would
increase the size of my file. I think that in a single-quoted string not
only \' and \\ should be interpreted but also \r. It would be logical as
it cannot appear litterally.

 

[R. KRause]

Thank you for your suggestion but I need a single-quoted string ('\r\n').
Indeed, the program I am writing is more complicate than the toy example I gave.
It does sth like that:

print "binmode(STDOUT);print \'",$buff,"\';"

And $buff might contain non backslashed double quotes or other special
characters that would be interpreted in a double-quote string.

It sounds like you want to get a binary stream of potentially unsafe
characters into printable form before sending it to the next perl
processor. I tossed this together, so I think it will do the trick:

my( $buf ) = "\$hi\@my.org\r\n"; # TEST BUFFER=> [email protected][CR][LF]

binmode( STDOUT );
$buf =~ s/([\$\@\000-\037\177])/( '\x' . unpack( 'H2', $1 ) )/ego;
print( 'binmode( STDOUT ); print "' . $buf . '";' );

The result with hexdump is...

00000000 24 68 69 40 6d 79 2e 6f 72 67 0d 0a |[email protected]..|
0000000c

Hope this helps,

--Randall

 

[Ben Morrow]

Thank you for your suggestion but I need a single-quoted string
('\r\n'). Indeed, the program I am writing is more complicate
than the toy example I gave. It does sth like that:

print "binmode(STDOUT);print \'",$buff,"\';"

And $buff might contain non backslashed double quotes or other special
characters that would be interpreted in a double-quote string.

It sounds like you want to get a binary stream of potentially unsafe
characters into printable form before sending it to the next perl
processor. I tossed this together, so I think it will do the trick:

my( $buf ) = "\$hi\@my.org\r\n"; # TEST BUFFER=> [email protected][CR][LF]

binmode( STDOUT );
$buf =~ s/([\$\@\000-\037\177])/( '\x' . unpack( 'H2', $1 ) )/ego;
print( 'binmode( STDOUT ); print "' . $buf . '";' );

Howsabout:

use Data:ump qw/dump/; # from CPAN: Data:umper doesn't work

my $buf = qq/"\$hi\@my.org\r\n"/;
binmode STDOUT;
print 'binmode STDOUT; print ' . (dump $buf) . ';';

I was going to simply suggest quotemeat[1], but that doesn't quote
control chars. Should it?

[1] a typo which amused me...

Ben

 

[R. KRause]

I was going to simply suggest quotemeat[1], but that doesn't quote
control chars. Should it?

[1] a typo which amused me...

Hmm, not sure. But I don't think there is the expectation one would
put invisible/control characters as string literals in a Perl script.
It would certainly make for one very non-portable program, that's for
sure

PS. I see I forgot to add " and \ in my original regexp char set.

--Randall

 

[David K. Wall]

But I have no reason to use any encoding such as Base64.

<Sigh> Perl programs are plain text. If you want your Perl program to
contain data representing a binary file, you're going to have to encode the
data somehow, so you might as well use something standard and convenient.
Otherwise you're going to be inventing your own method and wondering what
to do with bare carriage returns when instead you could write the program
fairly quickly by using a standard encoding scheme and a DATA section.

It is only
useful if you send data by mail or in newsgroups.

Seems like you've found another use for it.

Moreover it would
increase the size of my file.

That happens when you represent binary data as plain text.

Try this little program. It's bare-bones, but it does take an input file
(which can be binary data) and produces as output a Perl program that will
recreate the original file.

The important parts were cut and pasted from the MIME::Base64 docs.


use strict;
use warnings;
use MIME::Base64 qw(encode_base64);
use File::Basename;

die "Usage: $0 filename" if $ARGV[0] eq '';
open FILE, $ARGV[0] or die "Error: $!";
my $filename = basename $ARGV[0];

print <<END_PROG;
use MIME::Base64 qw(decode_base64);
open OUT, '>', "$filename" or die "Error: $!";
binmode OUT;
while (<DATA>) {
print OUT decode_base64 \$_;
}
__DATA__
END_PROG


my $buf;
while (read(FILE, $buf, 60*57)) {
print encode_base64($buf);
}