Arg, annoying problem, seems buffer related. (Kinda Long)

[Rhugga]

I have a script that uses ssh to connect to remove hosts and execute a
remote command, and then reads this data line by line. My problem is
that the first 20 lines or so get gargbled with exatra space, but then
after 20 it is fine. I have triple-checked the input data for
non-printing characters and etc.. and am confident my input data is not
the issue.

Here is my relevant code:
foreach my $server (@TSERVERS) {
open(RMT_CMD, "$ssh $server $remote_cmd |") || warn(localtime()
.. ": WARN: Unable to open connection to $server, skipping.\n");

while(<RMT_CMD>) {
my $arg = $_;
my @toks;
chomp($arg);

# Here I simply print $_
print "$_";

# Here I print $arg (which has been chomped)
print "$arg\n";

@toks = split(/ /, $arg); # Works 100%, the line parses
into tokens correctly

}

close(RMT_CMD);
}

My output:
HH0126 *NULL* 1 14 1131797824 1132071366 1135095366 1131873918
654115296 2076 2076 2 6 3 8 1024 0 2559153 0 0 0 0 0 0 0 adcbkp18



HH0127 *NULL* 1 14 1131812825 1131985268 1135009268 0 460649632
1548 1548 2 5 0 2 1024 0 1803578 0 0 0 0 0 0 0 adcbkp18
HH0127 *NULL* 1 14 1131812825
1131985268 1135009268 0 460649632 1548 1548 2 5 0 2 1024 0 1803578 0 0
0 0 0 0 0 adcbkp18
HH0127 *NULL* 1 14 1131812825 1131985268 1135009268 0 460649632 1548
1548 2 5 0 2 1024 0 1803578 0 0 0 0 0 0 0 adcbkp18
HH0154 *NULL* 1 14 1131727568 1131812648 1134836648 0 161241056 490 490
2 5 0 2 1024 0 631145 0 0 0 0 0 0 0 adcbkp18
HH0154 *NULL* 1 14 1131727568 1131812648 1134836648 0 161241056 490 490
2 5 0 2 1024 0 631145 0 0 0 0 0 0 0 adcbkp18

As you can see, the first few lines or so do not print correctly
(actually it's like the first 50 or so, then followed by about 200 that
print correctly), but then it displays fine. I also noticed that on the
lines that don't print correctly, the amount of white space preceeding
each line is idenitcal to the length of the previous line.

I even tried doing this before a print $arg:
$arg =~ s/^\s+//g;
$arg =~ s/\s+/\s/g;
$arg =~ s/\s+$//g;

If I create a data file that contains each line of data with only a
newline and read it thru STDIN, same thing. I'm confident my source
data is not the problem. It contains 1 line per line of data, and one
and only 1 newline. (and no other non-printing characters) I tested for
non-printing characters by doing 'cat -v -e <filename> | more' and
examining each line.

Here is a sample data file:
HH0002 *NULL* 1 14 1124390273 1133071439 1133676239 0 430258848 91 2 0
4 0 0 1024 0 1680923 0 0 0 0 0 0 0 adcbkp18
HH0003 *NULL* 1 14 1132678024 1132678024 1135702024 0 358432 1 1 2 5 0
2 1024 0 1405 0 0 0 0 0 0 0 adcbkp18
HH0009 *NULL* 1 14 1131639723 1131725822 1134749822 0 4613376 3 3 2 5 0
2 1024 0 18030 0 0 0 0 0 0 0 adcbkp18
HH0010 *NULL* 1 14 1130861467 1130948172 1133972172 0 57313984 310 310
2 5 0 2 1024 0 224715 0 0 0 0 0 0 0 adcbkp18
HH0011 *NULL* 1 14 1130861411 1131083769 1134107769 0 603713632 2512
2512 2 5 0 10 1024 0 2364142 0 0 0 0 0 0 0 adcbkp18
HH0012 *NULL* 1 14 1130861587 1130861587 1133885587 0 102432 1 1 2 5 0
2 1024 0 405 0 0 0 0 0 0 0 adcbkp18

Using this code, I should be able to duplicate my input file (or input
stream):
while (<STDIN>)
{
my $arg = $_;
chomp($arg);
print "$arg\n";
}

I am not seeing this behavior. Anyone have any ideas on where to look?

Thx,
rhugga

 

[usenet]

Before I really read through the post (yes, kinda long), I want to make
sure you've tried the obvious:

select(STDOUT); $| = 1; # unbuffer output

 

[Rhugga]

Yea, I tried setting that and it didn't have any effect. This problem
baffles me, as well as 3 other people that have looked at it.

Another odd thing is this:

../media_tracker | grep "^[A-Za-z]" | wc -l

So, this says give me a count of the number of lines that start with a
letter. The result I get from this command is the total number of
lines, including the ones that seem to be preceeded with white space.

I have now compeltely eliminated by data file but merely just printing
some text:
my @TSERVERS = "adcbkp18";
foreach my $server (@TSERVERS) {
open(RMT_CMD, "$ssh $server $rmt_cmd |") || warn(localtime() .
": WARN: Unable to open connection to $server, skipping.\n");

while(<RMT_CMD>) {
print "DEBUG\n";
}

close(RMT_CMD);
}

And now the very baffling output:
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG

DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG
DEBUG

ugh. This one eludes me.

=rhugga

 

[Rhugga]

I found my problem.

I was using 'ssh -q -t' to connect. Using ssh w/o any switches solved
the issue. Of course now I need to find a work-around for the problem
that caused me to start using '-q -t' in the first place.

Thanks,rhugga

 

[Tad McClellan]

while(<RMT_CMD>) {
my $arg = $_;

Why bother putting it into $_ if you really want it in $arg?

If you want it in in $arg, then put it in $arg:

while ( my $arg = <RMT_CMD> ) {

 

[Rhugga]

I just have it like that for debugging. Normally I just do this:

while(<RMT_CMD>) {
chomp;
my (@toks) = split;
}

What I posted was a dummied up version with anal debugging.

-CC