Net::SSH2 scp_put not working!

K

Krishna Chaitanya

Hi,

Am trying to use scp_put on an Net::SSH2 object but it fails thus:

====================
Listing 1.pl
---------------

#!/usr/bin/perl

use warnings;
use strict;

use Net::SSH2;

my $ssh2 = Net::SSH2->new();

$ssh2->connect('10.0.1.174') or die;

if ($ssh2->auth_password('inmdev','inmdev')) {
print "Authorization successful\n";
my $chan2 = $ssh2->channel();
$chan2->shell();
$chan2->blocking(0);
print $chan2 "uname -a\n";
print "LINE : $_" while <$chan2>;
$chan2->close();
$ssh2->scp_put("/home/perl_progs/2.pl") or warn "Could not scp
the file 2.pl ";
} else {
print "Authorization failure\n";
}


====================

Output is :

Authorization successful
LINE : Linux IMITS174.localdomain 2.6.9-42.ELsmp #1 SMP Wed Jul 12
23:27:17 EDT 2006 i686 i686 i386 GNU/Linux
Could not scp the file 2.pl at 1.pl line 20.

I've seen source of Net/SSH2.pm on this Linux box and the sub scp_put
has these lines:

my $chan = $self->_scp_put($remote, $mode, @stat[7, 8, 9]);
return unless $chan;
$chan->blocking(1);

There is no such function as _scp_put in this module or elsewhere on
my Linux box (I found out by find with -exec grep). What do I do in
this case? I tried running this program through perl -d and it failed
at this line.

Pls. help. Many thanks in advance.

-Chaitanya
 
G

Gunnar Hjalmarsson

Krishna said:
$ssh2->scp_put("/home/perl_progs/2.pl") or warn "Could not scp
the file 2.pl ";

Have you tried to make use of the error method?

$ssh2->scp_put("/home/perl_progs/2.pl") or
warn "Could not scp the file 2.pl", $ssh2->error;
 
Z

zentara

Hi,

Am trying to use scp_put on an Net::SSH2 object but it fails thus:

====================
Listing 1.pl
---------------

#!/usr/bin/perl

use warnings;
use strict;

use Net::SSH2;

my $ssh2 = Net::SSH2->new();

$ssh2->connect('10.0.1.174') or die;

if ($ssh2->auth_password('inmdev','inmdev')) {
print "Authorization successful\n";
my $chan2 = $ssh2->channel();
$chan2->shell();
$chan2->blocking(0);
print $chan2 "uname -a\n";
print "LINE : $_" while <$chan2>;
$chan2->close();
$ssh2->scp_put("/home/perl_progs/2.pl") or warn "Could not scp
the file 2.pl ";
} else {
print "Authorization failure\n";
}


====================

Output is :

Authorization successful
LINE : Linux IMITS174.localdomain 2.6.9-42.ELsmp #1 SMP Wed Jul 12
23:27:17 EDT 2006 i686 i686 i386 GNU/Linux
Could not scp the file 2.pl at 1.pl line 20.

I've seen source of Net/SSH2.pm on this Linux box and the sub scp_put
has these lines:

my $chan = $self->_scp_put($remote, $mode, @stat[7, 8, 9]);
return unless $chan;
$chan->blocking(1);

There is no such function as _scp_put in this module or elsewhere on
my Linux box (I found out by find with -exec grep). What do I do in
this case? I tried running this program through perl -d and it failed
at this line.

Pls. help. Many thanks in advance.

-Chaitanya

Sometimes the error messages only hint at the problem.
Your line:
$ssh2->scp_put("/home/perl_progs/2.pl") or warn "Could not scp
the file 2.pl ";
Error:
my $chan = $self->_scp_put($remote, $mode, @stat[7, 8, 9]);

I believe you may not have a remote filename listed?
It should look like
$ssh->scp_put('local_filename', 'full_remote_filename');
# complete full paths from the root /

These ssh modules can be very demanding about full file paths,
both local and remote.

zentara
 
K

Krishna Chaitanya

Hi,

This is from Net::SSH2 documentation of scp_put :

===============================

scp_put ( local [, remote ] )

Send a file with scp; remote path defaults to same as local. local may
be an IO object instead of a filename (but it must have a valid stat
method).

===============================

The code of this function in SSH2.pm is :

sub scp_put {
my ($self, $path, $remote) = @_;
$remote = basename $path if not defined $remote;

my $file = ref $path ? $path : IO::File->new($path, O_RDONLY);
$self->error($!, $!), return unless $file;
my @stat = $file->stat;
$self->error($!, $!), return unless @stat;

my $mode = $stat[2] & 0777; # mask off extras such as S_IFREG
my $chan = $self->_scp_put($remote, $mode, @stat[7, 8, 9]);
return unless $chan;
$chan->blocking(1);

# read and transmit blocks until we're finished
for (my ($size, $count) = ($stat[7]); $size > 0; $size -= $count)
{
my $buf;
my $block = ($size > 8192) ? 8192 : $size;
$count = $file->sysread($buf, $block);
$self->error($!, $!), return unless defined $count;
$self->error(0, "want $block, have $count"), return
unless $count == $block;
die 'sysread mismatch' unless length $buf == $count;
$self->error(0, "error writing $count bytes to channel"), return
unless $chan->write($buf) == $count;
}

# send/receive SCP acknowledgement
$chan->write("\0");
my $eof;
$chan->read($eof, 1);
return 1;
}

So I ran my program through perl -d and when it came to the line
having _scp_put, I printed out values of variables $path and $remote
and their values were "/home/perl_progs/2.pl" and "2.pl" ($remote is
filled up by basename as seen in code above). Now it's clear that both
the local and remote variables have values.....but shouldn't $remote
have some mention of the remote "DIRECTORY" too?

Also, where does _scp_put function exist? It looks from naming
convention like a method from a base class. I even tried doing an "nm"
on libssh2.* in /usr/lib (where I installed libssh2).

Thanks,
Chaitanya
 
Z

zentara

Hi,

This is from Net::SSH2 documentation of scp_put :

===============================

scp_put ( local [, remote ] )


So I ran my program through perl -d and when it came to the line
having _scp_put, I printed out values of variables $path and $remote
and their values were "/home/perl_progs/2.pl" and "2.pl" ($remote is
filled up by basename as seen in code above). Now it's clear that both
the local and remote variables have values.....but shouldn't $remote
have some mention of the remote "DIRECTORY" too?

Yeah, thats my point...it NEEDS the directory too. It wants fully
qualified pathnames to \ on the remote machine. I don't think it
gaurantees that you intend the transfer with an automatic pathname
created by some regex than sends the localfilename to the cwd
on the remote machine. It is more secure than that...it demands
you know the full path on the remote computer.
Also, where does _scp_put function exist? It looks from naming
convention like a method from a base class. I even tried doing an "nm"
on libssh2.* in /usr/lib (where I installed libssh2).

Thanks,
Chaitanya
All that stuff is hidden in the XS stuff that accesses the libssh2 c
libs. You rarely need to question those libs and their access, they are
widely scruntized.
As a Perl scripter, you don't need to know the inner workings....that is
for the security people to decide. Suffice it to say, that SSH code is
not the easiest stuff to comprehend, at least for me anyways. :)

zentara
 
K

Krishna Chaitanya

Hi Ted, sorry...I'm new to groups, I'll keep in mind not to bump any
posts...

Hi Zentara,

If it NEEDS the directory, then why would it document the remote path
as optional (conventionally...anything in square brackets...)? Also, I
had pasted the code of scp_put function for your review.....and in it,
the $remote variable is filled up by doing a basename on $file ....
which means it intends to only grab the file's basename.....why this
issue then? Can anyone answer please?

-Chaitanya
 
G

Gunnar Hjalmarsson

Krishna said:
Can anyone answer please?

You ignored my suggestion to have Net::SSH2 tell you *why* scp_put fails
(without digging in the source code).
 
K

Krishna Chaitanya

Thanks, Gunnar. Apologies for not doing it earlier.....it wasn't
deliberate, I missed your earlier post completely.

When I printed $ssh2->error, it said LIBSSH2_ERROR_EAGAIN (-37).
Looked up "man 3 libssh2_scp_send_ex" and it says this:

"LIBSSH2_ERROR_EAGAIN - Marked for non-blocking I/O but the call would
block."

Combining that with the documentation of Net::SSH2::blocking ("Note
that if blocking is disabled, methods that create channels may fail,
e.g. channel, SFTP, scp_*."), I figured out that $chan2->blocking(0)
was the mistake. Now I've re-written the code as:

====================
Listing 1.pl
---------------

#!/usr/bin/perl

use warnings;
use strict;

use Net::SSH2;

my $ssh2 = Net::SSH2->new();

$ssh2->connect('10.0.1.174') or die;

if ($ssh2->auth_password('inmdev','inmdev')) {
print "Authorization successful\n";
my $chan2 = $ssh2->channel();
$chan2->shell();
$ssh2->blocking(0);
print $chan2 "uname -a\n";
print "LINE : $_" while <$chan2>;
$chan2->close();
$ssh2->blocking(1);
$ssh2->scp_put("/home/perl_progs/2.pl","/tmp/2.pl") or warn
"Could not scp
the file 2.pl ";
} else {

print "Authorization failure\n";

}

====================

Just added $ssh2->blocking(1) before calling scp_put and it worked
successfully !!!

Thanks everyone! I am summarising this:

"If you intend to use channel, SFTP, scp_* (methods that create
channels), do not disable blocking ... i.e. do NOT say $ssh->blocking
(0) before executing those methods. Also, always print out $ssh->error
for descriptive info of error and combine it with the man pages of
libssh2"
 
K

Krishna Chaitanya

Hi Ben,

Thnx for your insight. It's true that I don't have anything else to do
while the I/O goes on...so you're right, I should just stick to
blocking I/O. But when I'm doing that, the results in repeated runs of
my program are unpredictable. Sometimes, it returns soon after
successfully copying the file, sometimes it just "hangs". This
particular program will be developed to use on a large network
(predictably 100s or more of Linux boxes) to copy some files to each
Linux box and it'd be ideal to see a predictable nature of response
with every run.
 
K

Krishna Chaitanya

Hmm...I've never done anything of this sort, so pls. bear with my
ignorance. Let's say 10.0.1.174 is the remote machine and my machine
where I'm working is 10.0.75.202. I've used tcpdump in this way:

Opened 2 parallel PuTTY sessions to 10.0.1.174 and executed these :

Putty session A - "tcpdump -vvv src host 10.0.75.202"
Putty session B - "tcpdump -vvv dst host 10.0.75.202"

Session A shows 13 packets and Session B shows 12.....frankly I have
no clue of what's going on here.....I've searched google on how to
interpret tcpdump headers but it's not helping much to decipher why
blocking I/O is causing the problem here.... :(
 
T

Tim Greer

Krishna said:
Hmm...I've never done anything of this sort, so pls. bear with my
ignorance. Let's say 10.0.1.174 is the remote machine and my machine
where I'm working is 10.0.75.202. I've used tcpdump in this way:

Opened 2 parallel PuTTY sessions to 10.0.1.174 and executed these :

Putty session A - "tcpdump -vvv src host 10.0.75.202"
Putty session B - "tcpdump -vvv dst host 10.0.75.202"

Session A shows 13 packets and Session B shows 12.....frankly I have
no clue of what's going on here.....I've searched google on how to
interpret tcpdump headers but it's not helping much to decipher why
blocking I/O is causing the problem here.... :(

I'm curious, out of all of the testing with the script, have you
thoroughly tested SSH/scp over the network/systems in such a way that
it would indicate a problem with the connections or transfer process
before you start getting into the gory details of the script? Have you
considered using SSH keys rather than the login? How many files are
you moving, how often, and how large are they potentially?
 
K

Krishna Chaitanya

Hi Tim,

SSH/SCP work very smoothly at least in my test bed (don't have access
yet to the customer's network). At least, they work smoothly when I
type the commands at the prompt. But they don't when I use synchronous
I/O as discussed above.

SSH key exchange is not permitted by the customer in his network. I am
expected to host my program on a Linux box and execute it to copy a
set of files (about 300 MB in total) from there to each machine (list
of IP addresses and their passwords is provided up front). After
copying those files, I will have to execute certain programs on each
machine and log the output to a central directory. With Net::SSH2,
since it is smart enough (?) to expect and react to SSH challenges, I
had a feeling I wouldn't really need SSH key exchange.
 
Z

zentara

Hi Tim,

SSH/SCP work very smoothly at least in my test bed (don't have access
yet to the customer's network). At least, they work smoothly when I
type the commands at the prompt. But they don't when I use synchronous
I/O as discussed above.

SSH key exchange is not permitted by the customer in his network. I am
expected to host my program on a Linux box and execute it to copy a
set of files (about 300 MB in total) from there to each machine (list
of IP addresses and their passwords is provided up front). After
copying those files, I will have to execute certain programs on each
machine and log the output to a central directory. With Net::SSH2,
since it is smart enough (?) to expect and react to SSH challenges, I
had a feeling I wouldn't really need SSH key exchange.

Have you seen http://perlmonks.org?node_id=635521

zentara
 
K

Krishna Chaitanya

Have you seen  http://perlmonks.org?node_id=635521 

Yes, I'd seen it earlier. I hope blocking(0) means asynchronous or non-
blocking I/O. It poses a problem for me : (a) I've nothing else to do
while waiting for async I/O to complete and why is the program hanging
anyway just because blocking(1) is used? (b) blocking(0) causes
scp_put to fail since this function requires blocking I/O.

In closing, it'd be great to know why blocking I/O poses a problem for
issuing commands and gathering output.....it's not like the network is
super-busy or something. Traffic's pretty relaxed here. Also, the
scp_put command sends the files real fast if blocking(1) is used. Any
help?
 
K

Krishna Chaitanya

I understood what you said. Maybe I'm not able to communicate my
concern in the right manner...my program structure is this:

#!/usr/bin/perl

use warnings;
use strict;

use Net::SSH2;

my $ssh2 = Net::SSH2->new();

$ssh2->debug(1);

$ssh2->connect('10.0.1.174') or die;

if ($ssh2->auth_password('inmdev','inmdev')) {
print "Authorization successful\n";
my $chan2 = $ssh2->channel();
$chan2->blocking(0);
$chan2->shell();
print $chan2 "uname -a\n";
print "LINE : $_" while <$chan2>;
$chan2->close();
$chan2->blocking(1);
$ssh2->scp_put("/root/perl_progs/1.pl","/tmp/1.pl") or warn
"Could not scp the file ", $ssh2->error;
} else {
print "Authorization failure\n";
}

If I don't put the "$chan2->blocking(0);" or if I put "$chan2->blocking
(1);", it just hangs after the "Authorization successful" message.
Every single time. Any reason for this to happen? Something wrong in
the code above?

If you think blocking I/O is the way to go, it's OK with me provided
the hangs don't occur.....I've been at this problem now since many
days, as you can see. :-(

The basic problem I'm trying to solve is to be able to run commands on
remote linux machines through ssh 2 and copy some files to remote
linux machines. Do you have any suggestion for the program structure?
Can you pls. point me in the right direction if I approached the
problem wrongly? Thanks a lot, Ben, for your time and thought.
 
K

Krishna Chaitanya

        print $chan2 "uname -a\n";
Do you get any output here? I would expect that if you've set
nonblocking mode and it's trying to block that both the print and the <>
would immediately fail with EAGAIN as well, and you'd get no output. Is
this not the case?

No, it successfully prints the uname. No EGAIN errors here if I set
blocking(0). And this succeeds every time on my setup.
I don't know. Can you turn on some sort of debug trace, or use tcpdump,
to see what's actually happening? You will of course need to know more
about the ssh protocol that I do to make sense of it... :)

Since you're only executing a single command on your Channel, have you
tried using ->exec instead of ->shell? Messing around with talking to a
remote shell seems worth avoiding if you can.

Ben

I'd posted in 1 of the posts above my findings from a tcpdump
session.....after which Tim Greer had advised to go back to the basics
instead of examining the gory details of the tcp output. :) So it
kinda got stopped at that. I'm actually quite new to these protocols.
And regarding exec, in this example it will suit me because I'm
running a single command.....in the production program, there will be
many. That's why I had used shell.
 
Z

zentara

I understood what you said. Maybe I'm not able to communicate my
concern in the right manner...my program structure is this:

If you havn't yet, look at
http://perlmonks.org/?node=621761

Also, I can't find the nodes, but I also seem to remember problems
with Net::SSH2 and Perl 5.10 (if that is what you are using)
So googling for "Net::SSH2 Perl 5.10 patch" might be helpful.

zentara
 
K

Krishna Chaitanya

If you havn't yet, look athttp://perlmonks.org/?node=621761

Yep, I'd read this many days back...that's what gave me the idea of
blocking(0) in the first place :D
I wonder what u235sentinel did (in the above-mentioned posting on
perlmonks) when (s)he succeeded in hitting a couple hundred of boxes
over the network using Net::SSH2. I can't even hit one....without
using blocking(0) I mean.
Also, I can't find the nodes, but I also seem to remember problems
with Net::SSH2 and Perl 5.10 (if that is what you are using)
So googling for "Net::SSH2  Perl 5.10  patch" might be helpful.

The customer has Perl 5.8.8 so that's going to be my version for a
while. Thanks!
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,483
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top