Net::SSH2 scp_put not working!

Discussion in 'Perl Misc' started by Krishna Chaitanya, Feb 17, 2009.

  1. Hi,

    Am trying to use scp_put on an Net::SSH2 object but it fails thus:

    ====================
    Listing 1.pl
    ---------------

    #!/usr/bin/perl

    use warnings;
    use strict;

    use Net::SSH2;

    my $ssh2 = Net::SSH2->new();

    $ssh2->connect('10.0.1.174') or die;

    if ($ssh2->auth_password('inmdev','inmdev')) {
    print "Authorization successful\n";
    my $chan2 = $ssh2->channel();
    $chan2->shell();
    $chan2->blocking(0);
    print $chan2 "uname -a\n";
    print "LINE : $_" while <$chan2>;
    $chan2->close();
    $ssh2->scp_put("/home/perl_progs/2.pl") or warn "Could not scp
    the file 2.pl ";
    } else {
    print "Authorization failure\n";
    }


    ====================

    Output is :

    Authorization successful
    LINE : Linux IMITS174.localdomain 2.6.9-42.ELsmp #1 SMP Wed Jul 12
    23:27:17 EDT 2006 i686 i686 i386 GNU/Linux
    Could not scp the file 2.pl at 1.pl line 20.

    I've seen source of Net/SSH2.pm on this Linux box and the sub scp_put
    has these lines:

    my $chan = $self->_scp_put($remote, $mode, @stat[7, 8, 9]);
    return unless $chan;
    $chan->blocking(1);

    There is no such function as _scp_put in this module or elsewhere on
    my Linux box (I found out by find with -exec grep). What do I do in
    this case? I tried running this program through perl -d and it failed
    at this line.

    Pls. help. Many thanks in advance.

    -Chaitanya
     
    Krishna Chaitanya, Feb 17, 2009
    #1
    1. Advertisements

  2. Have you tried to make use of the error method?

    $ssh2->scp_put("/home/perl_progs/2.pl") or
    warn "Could not scp the file 2.pl", $ssh2->error;
     
    Gunnar Hjalmarsson, Feb 17, 2009
    #2
    1. Advertisements

  3. Krishna Chaitanya

    zentara Guest

    Sometimes the error messages only hint at the problem.
    Your line:
    I believe you may not have a remote filename listed?
    It should look like
    $ssh->scp_put('local_filename', 'full_remote_filename');
    # complete full paths from the root /

    These ssh modules can be very demanding about full file paths,
    both local and remote.

    zentara
     
    zentara, Feb 17, 2009
    #3
  4. Hi,

    This is from Net::SSH2 documentation of scp_put :

    ===============================

    scp_put ( local [, remote ] )

    Send a file with scp; remote path defaults to same as local. local may
    be an IO object instead of a filename (but it must have a valid stat
    method).

    ===============================

    The code of this function in SSH2.pm is :

    sub scp_put {
    my ($self, $path, $remote) = @_;
    $remote = basename $path if not defined $remote;

    my $file = ref $path ? $path : IO::File->new($path, O_RDONLY);
    $self->error($!, $!), return unless $file;
    my @stat = $file->stat;
    $self->error($!, $!), return unless @stat;

    my $mode = $stat[2] & 0777; # mask off extras such as S_IFREG
    my $chan = $self->_scp_put($remote, $mode, @stat[7, 8, 9]);
    return unless $chan;
    $chan->blocking(1);

    # read and transmit blocks until we're finished
    for (my ($size, $count) = ($stat[7]); $size > 0; $size -= $count)
    {
    my $buf;
    my $block = ($size > 8192) ? 8192 : $size;
    $count = $file->sysread($buf, $block);
    $self->error($!, $!), return unless defined $count;
    $self->error(0, "want $block, have $count"), return
    unless $count == $block;
    die 'sysread mismatch' unless length $buf == $count;
    $self->error(0, "error writing $count bytes to channel"), return
    unless $chan->write($buf) == $count;
    }

    # send/receive SCP acknowledgement
    $chan->write("\0");
    my $eof;
    $chan->read($eof, 1);
    return 1;
    }

    So I ran my program through perl -d and when it came to the line
    having _scp_put, I printed out values of variables $path and $remote
    and their values were "/home/perl_progs/2.pl" and "2.pl" ($remote is
    filled up by basename as seen in code above). Now it's clear that both
    the local and remote variables have values.....but shouldn't $remote
    have some mention of the remote "DIRECTORY" too?

    Also, where does _scp_put function exist? It looks from naming
    convention like a method from a base class. I even tried doing an "nm"
    on libssh2.* in /usr/lib (where I installed libssh2).

    Thanks,
    Chaitanya
     
    Krishna Chaitanya, Feb 17, 2009
    #4
  5. Bumping this up...
     
    Krishna Chaitanya, Feb 18, 2009
    #5
  6. Krishna Chaitanya

    zentara Guest

    Yeah, thats my point...it NEEDS the directory too. It wants fully
    qualified pathnames to \ on the remote machine. I don't think it
    gaurantees that you intend the transfer with an automatic pathname
    created by some regex than sends the localfilename to the cwd
    on the remote machine. It is more secure than that...it demands
    you know the full path on the remote computer.
    All that stuff is hidden in the XS stuff that accesses the libssh2 c
    libs. You rarely need to question those libs and their access, they are
    widely scruntized.
    As a Perl scripter, you don't need to know the inner workings....that is
    for the security people to decide. Suffice it to say, that SSH code is
    not the easiest stuff to comprehend, at least for me anyways. :)

    zentara
     
    zentara, Feb 18, 2009
    #6

  7. Scoring this OP down...

    (Usenet is not a BBS)
     
    Tad J McClellan, Feb 18, 2009
    #7
  8. Hi Ted, sorry...I'm new to groups, I'll keep in mind not to bump any
    posts...

    Hi Zentara,

    If it NEEDS the directory, then why would it document the remote path
    as optional (conventionally...anything in square brackets...)? Also, I
    had pasted the code of scp_put function for your review.....and in it,
    the $remote variable is filled up by doing a basename on $file ....
    which means it intends to only grab the file's basename.....why this
    issue then? Can anyone answer please?

    -Chaitanya
     
    Krishna Chaitanya, Feb 18, 2009
    #8
  9. You ignored my suggestion to have Net::SSH2 tell you *why* scp_put fails
    (without digging in the source code).
     
    Gunnar Hjalmarsson, Feb 18, 2009
    #9
  10. Thanks, Gunnar. Apologies for not doing it earlier.....it wasn't
    deliberate, I missed your earlier post completely.

    When I printed $ssh2->error, it said LIBSSH2_ERROR_EAGAIN (-37).
    Looked up "man 3 libssh2_scp_send_ex" and it says this:

    "LIBSSH2_ERROR_EAGAIN - Marked for non-blocking I/O but the call would
    block."

    Combining that with the documentation of Net::SSH2::blocking ("Note
    that if blocking is disabled, methods that create channels may fail,
    e.g. channel, SFTP, scp_*."), I figured out that $chan2->blocking(0)
    was the mistake. Now I've re-written the code as:

    ====================
    Listing 1.pl
    ---------------

    #!/usr/bin/perl

    use warnings;
    use strict;

    use Net::SSH2;

    my $ssh2 = Net::SSH2->new();

    $ssh2->connect('10.0.1.174') or die;

    if ($ssh2->auth_password('inmdev','inmdev')) {
    print "Authorization successful\n";
    my $chan2 = $ssh2->channel();
    $chan2->shell();
    $ssh2->blocking(0);
    print $chan2 "uname -a\n";
    print "LINE : $_" while <$chan2>;
    $chan2->close();
    $ssh2->blocking(1);
    $ssh2->scp_put("/home/perl_progs/2.pl","/tmp/2.pl") or warn
    "Could not scp
    the file 2.pl ";
    } else {

    print "Authorization failure\n";

    }

    ====================

    Just added $ssh2->blocking(1) before calling scp_put and it worked
    successfully !!!

    Thanks everyone! I am summarising this:

    "If you intend to use channel, SFTP, scp_* (methods that create
    channels), do not disable blocking ... i.e. do NOT say $ssh->blocking
    (0) before executing those methods. Also, always print out $ssh->error
    for descriptive info of error and combine it with the man pages of
    libssh2"
     
    Krishna Chaitanya, Feb 18, 2009
    #10
  11. Hi Ben,

    Thnx for your insight. It's true that I don't have anything else to do
    while the I/O goes on...so you're right, I should just stick to
    blocking I/O. But when I'm doing that, the results in repeated runs of
    my program are unpredictable. Sometimes, it returns soon after
    successfully copying the file, sometimes it just "hangs". This
    particular program will be developed to use on a large network
    (predictably 100s or more of Linux boxes) to copy some files to each
    Linux box and it'd be ideal to see a predictable nature of response
    with every run.
     
    Krishna Chaitanya, Feb 19, 2009
    #11
  12. Hmm...I've never done anything of this sort, so pls. bear with my
    ignorance. Let's say 10.0.1.174 is the remote machine and my machine
    where I'm working is 10.0.75.202. I've used tcpdump in this way:

    Opened 2 parallel PuTTY sessions to 10.0.1.174 and executed these :

    Putty session A - "tcpdump -vvv src host 10.0.75.202"
    Putty session B - "tcpdump -vvv dst host 10.0.75.202"

    Session A shows 13 packets and Session B shows 12.....frankly I have
    no clue of what's going on here.....I've searched google on how to
    interpret tcpdump headers but it's not helping much to decipher why
    blocking I/O is causing the problem here.... :(
     
    Krishna Chaitanya, Feb 19, 2009
    #12
  13. Krishna Chaitanya

    Tim Greer Guest

    I'm curious, out of all of the testing with the script, have you
    thoroughly tested SSH/scp over the network/systems in such a way that
    it would indicate a problem with the connections or transfer process
    before you start getting into the gory details of the script? Have you
    considered using SSH keys rather than the login? How many files are
    you moving, how often, and how large are they potentially?
     
    Tim Greer, Feb 19, 2009
    #13
  14. Hi Tim,

    SSH/SCP work very smoothly at least in my test bed (don't have access
    yet to the customer's network). At least, they work smoothly when I
    type the commands at the prompt. But they don't when I use synchronous
    I/O as discussed above.

    SSH key exchange is not permitted by the customer in his network. I am
    expected to host my program on a Linux box and execute it to copy a
    set of files (about 300 MB in total) from there to each machine (list
    of IP addresses and their passwords is provided up front). After
    copying those files, I will have to execute certain programs on each
    machine and log the output to a central directory. With Net::SSH2,
    since it is smart enough (?) to expect and react to SSH challenges, I
    had a feeling I wouldn't really need SSH key exchange.
     
    Krishna Chaitanya, Feb 19, 2009
    #14
  15. Krishna Chaitanya

    zentara Guest

    Have you seen http://perlmonks.org?node_id=635521

    zentara
     
    zentara, Feb 21, 2009
    #15
  16. Have you seen  http://perlmonks.org?node_id=635521 

    Yes, I'd seen it earlier. I hope blocking(0) means asynchronous or non-
    blocking I/O. It poses a problem for me : (a) I've nothing else to do
    while waiting for async I/O to complete and why is the program hanging
    anyway just because blocking(1) is used? (b) blocking(0) causes
    scp_put to fail since this function requires blocking I/O.

    In closing, it'd be great to know why blocking I/O poses a problem for
    issuing commands and gathering output.....it's not like the network is
    super-busy or something. Traffic's pretty relaxed here. Also, the
    scp_put command sends the files real fast if blocking(1) is used. Any
    help?
     
    Krishna Chaitanya, Feb 26, 2009
    #16
  17. I understood what you said. Maybe I'm not able to communicate my
    concern in the right manner...my program structure is this:

    #!/usr/bin/perl

    use warnings;
    use strict;

    use Net::SSH2;

    my $ssh2 = Net::SSH2->new();

    $ssh2->debug(1);

    $ssh2->connect('10.0.1.174') or die;

    if ($ssh2->auth_password('inmdev','inmdev')) {
    print "Authorization successful\n";
    my $chan2 = $ssh2->channel();
    $chan2->blocking(0);
    $chan2->shell();
    print $chan2 "uname -a\n";
    print "LINE : $_" while <$chan2>;
    $chan2->close();
    $chan2->blocking(1);
    $ssh2->scp_put("/root/perl_progs/1.pl","/tmp/1.pl") or warn
    "Could not scp the file ", $ssh2->error;
    } else {
    print "Authorization failure\n";
    }

    If I don't put the "$chan2->blocking(0);" or if I put "$chan2->blocking
    (1);", it just hangs after the "Authorization successful" message.
    Every single time. Any reason for this to happen? Something wrong in
    the code above?

    If you think blocking I/O is the way to go, it's OK with me provided
    the hangs don't occur.....I've been at this problem now since many
    days, as you can see. :-(

    The basic problem I'm trying to solve is to be able to run commands on
    remote linux machines through ssh 2 and copy some files to remote
    linux machines. Do you have any suggestion for the program structure?
    Can you pls. point me in the right direction if I approached the
    problem wrongly? Thanks a lot, Ben, for your time and thought.
     
    Krishna Chaitanya, Feb 26, 2009
    #17
  18.         print $chan2 "uname -a\n";
    No, it successfully prints the uname. No EGAIN errors here if I set
    blocking(0). And this succeeds every time on my setup.
    I'd posted in 1 of the posts above my findings from a tcpdump
    session.....after which Tim Greer had advised to go back to the basics
    instead of examining the gory details of the tcp output. :) So it
    kinda got stopped at that. I'm actually quite new to these protocols.
    And regarding exec, in this example it will suit me because I'm
    running a single command.....in the production program, there will be
    many. That's why I had used shell.
     
    Krishna Chaitanya, Feb 26, 2009
    #18
  19. Krishna Chaitanya

    zentara Guest

    If you havn't yet, look at
    http://perlmonks.org/?node=621761

    Also, I can't find the nodes, but I also seem to remember problems
    with Net::SSH2 and Perl 5.10 (if that is what you are using)
    So googling for "Net::SSH2 Perl 5.10 patch" might be helpful.

    zentara
     
    zentara, Feb 26, 2009
    #19
  20. If you havn't yet, look athttp://perlmonks.org/?node=621761

    Yep, I'd read this many days back...that's what gave me the idea of
    blocking(0) in the first place :D
    I wonder what u235sentinel did (in the above-mentioned posting on
    perlmonks) when (s)he succeeded in hitting a couple hundred of boxes
    over the network using Net::SSH2. I can't even hit one....without
    using blocking(0) I mean.
    The customer has Perl 5.8.8 so that's going to be my version for a
    while. Thanks!
     
    Krishna Chaitanya, Feb 26, 2009
    #20
    1. Advertisements

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments (here). After that, you can post your question and our members will help you out.