Automate rsync w/ authentication

B

Bryan

I am trying to automate rsync to backup server A from server B. I
have set up a private/public key between the two servers so I don't
have to enter a password when using rsync. Running rsync manually
with the following command works fine:
rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
[email protected]:/home/bry/jquery.lookup /home/bry/tmp

But when I try to do it with python, the subprocess simply returns the
ssh -h output on stderr like I am passing some invalid syntax. What
is wrong in my translation of rsync's -e command from shell to
pythyon?

#! /usr/bin/python
from subprocess import Popen, PIPE
rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
try:
p = Popen(args, stdout=PIPE, stderr=PIPE)
print 'rsync running with pid %s' % p.pid
out, err = p.communicate()
print 'Errors: %s' % err
print 'Output: %s' % out
except Exception:
print 'Error running rsync'
 
C

Chris Rebert

I am trying to automate rsync to backup server A from server B.  I
have set up a private/public key between the two servers so I don't
have to enter a password when using rsync.  Running rsync manually
with the following command works fine:
rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
[email protected]:/home/bry/jquery.lookup /home/bry/tmp

But when I try to do it with python, the subprocess simply returns the
ssh -h output on stderr like I am passing some invalid syntax.  What
is wrong in my translation of rsync's -e command from shell to
pythyon?

#! /usr/bin/python
from subprocess import Popen, PIPE
rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]

Like many problems involving the subprocess module, I think you've
tokenized the arguments incorrectly. Try:

rshArg = '"/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-av', '--dry-run', '-e', rshArg, source, dest]

Note that the -e switch and its operand are separate arguments for the
purposes of POSIX shell tokenization.

Cheers,
Chris
 
G

Gary Duzan

rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]

I think you want -e and the ssh command to be separate args.
Something like:

rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest]

or:

rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ]
args = [rsyncExec, '-a', '-v', '--dry-run'] + rshArgs + [ source, dest]

Gary Duzan
Motorola H&NM
 
P

Piet van Oostrum

Chris Rebert said:
I am trying to automate rsync to backup server A from server B.  I
have set up a private/public key between the two servers so I don't
have to enter a password when using rsync.  Running rsync manually
with the following command works fine:
rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
[email protected]:/home/bry/jquery.lookup /home/bry/tmp

But when I try to do it with python, the subprocess simply returns the
ssh -h output on stderr like I am passing some invalid syntax.  What
is wrong in my translation of rsync's -e command from shell to
pythyon?

#! /usr/bin/python
from subprocess import Popen, PIPE
rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
CR> Like many problems involving the subprocess module, I think you've
CR> tokenized the arguments incorrectly. Try:
CR> rshArg = '"/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
CR> args = [rsyncExec, '-av', '--dry-run', '-e', rshArg, source, dest]
CR> Note that the -e switch and its operand are separate arguments for the
CR> purposes of POSIX shell tokenization.

I think you should have only one kind of quotes in rshArg:
rshArg = "/usr/bin/ssh -i /home/bry/keys/brybackup.key"

I haven't tried it, however, but this is just how Unix works.
 
C

Chris Rebert

I am trying to automate rsync to backup server A from server B.  I
have set up a private/public key between the two servers so I don't
have to enter a password when using rsync.  Running rsync manually
with the following command works fine:
rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
[email protected]:/home/bry/jquery.lookup /home/bry/tmp

But when I try to do it with python, the subprocess simply returns the
ssh -h output on stderr like I am passing some invalid syntax.  What
is wrong in my translation of rsync's -e command from shell to
pythyon?

#! /usr/bin/python
from subprocess import Popen, PIPE
rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
CR> Like many problems involving the subprocess module, I think you've
CR> tokenized the arguments incorrectly. Try:
CR> rshArg = '"/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
CR> args = [rsyncExec, '-av', '--dry-run', '-e', rshArg, source, dest]
CR> Note that the -e switch and its operand are separate arguments for the
CR> purposes of POSIX shell tokenization.

I think you should have only one kind of quotes in rshArg:
rshArg = "/usr/bin/ssh -i /home/bry/keys/brybackup.key"

I haven't tried it, however, but this is just how Unix works.

Ah, indeed, I think you're probably right. Just goes to show it's not
always easy to get exactly right.

Cheers,
Chris
 
B

Bryan

Chris Rebert <[email protected]> (CR) wrote:
I am trying to automate rsync to backup server A from server B.  I
have set up a private/public key between the two servers so I don't
have to enter a password when using rsync.  Running rsync manually
with the following command works fine:
rsync -av --dry-run -e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"
[email protected]:/home/bry/jquery.lookup /home/bry/tmp
But when I try to do it with python, the subprocess simply returns the
ssh -h output on stderr like I am passing some invalid syntax.  What
is wrong in my translation of rsync's -e command from shell to
pythyon?
#! /usr/bin/python
from subprocess import Popen, PIPE
rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]
CR> Like many problems involving the subprocess module, I think you've
CR> tokenized the arguments incorrectly. Try:
CR> rshArg = '"/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
CR> args = [rsyncExec, '-av', '--dry-run', '-e', rshArg, source, dest]
CR> Note that the -e switch and its operand are separate arguments for the
CR> purposes of POSIX shell tokenization.

I think you should have only one kind of quotes in rshArg:
rshArg = "/usr/bin/ssh -i /home/bry/keys/brybackup.key"

I haven't tried it, however, but this is just how Unix works.

I tried removing the internal quotes, and splitting '-e' from the
other arguments. I still get the same ssh -h output however:

rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source,
dest]
p = Popen(args, stdout=PIPE, stderr=PIPE)
 
P

Piet van Oostrum

Bryan said:
B> I tried removing the internal quotes, and splitting '-e' from the
B> other arguments. I still get the same ssh -h output however:
B> rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
B> args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source,
B> dest]
B> p = Popen(args, stdout=PIPE, stderr=PIPE)

For me it works. Of course I substituted local server names, file names
etc. I run this on Mac OS X 10.4.11. The server is some Linux system.

------------------------------------------------------------------------
#! /usr/bin/env python

from subprocess import Popen, PIPE
rsyncExec = '/usr/local/bin/rsync'

source = 'xxx.cs.uu.nl:/users/piet/yyyyyy'
dest = '/Users/piet/TEMP/test.rsync'

rshArg = '/usr/bin/ssh -i /Users/piet/.ssh/id_rsa'

args = [rsyncExec, '-a', '-v', '-e', rshArg, source, dest]

try:
p = Popen(args, stdout=PIPE, stderr=PIPE)
print 'rsync running with pid %s' % p.pid
out, err = p.communicate()
print 'Errors: %s' % err
print 'Output: %s' % out
except Exception:
print 'Error running rsync'
 
B

Bryan

Bryan   said:
rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]

   I think you want -e and the ssh command to be separate args.
Something like:

rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest]

or:

rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ]
args = [rsyncExec, '-a', '-v', '--dry-run'] +  rshArgs + [ source, dest]

                                        Gary Duzan
                                        Motorola H&NM

Separating the argument parts worked. Strangely though, I don't need
to do that for arguments such as --files-from='/path/to/file'. Also,
in this example code I had the rsync executable path pointing to the
ssh program, so no wonder I was getting the output of ssh!
 
P

Piet van Oostrum

Bryan said:
rsyncExec = '/usr/bin/ssh'
source = '[email protected]:/home/bry/jquery.lookup'
dest = '/home/bry/tmp'
rshArg = '-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key"'
args = [rsyncExec, '-a', '-v', '--dry-run', rshArg, source, dest]

   I think you want -e and the ssh command to be separate args.
Something like:

rshArg = '/usr/bin/ssh -i /home/bry/keys/brybackup.key'
args = [rsyncExec, '-a', '-v', '--dry-run', '-e', rshArg, source, dest]

or:

rshArgs = [ '-e', '/usr/bin/ssh -i /home/bry/keys/brybackup.key' ]
args = [rsyncExec, '-a', '-v', '--dry-run'] +  rshArgs + [ source, dest]

                                        Gary Duzan
                                        Motorola H&NM
B> Separating the argument parts worked. Strangely though, I don't need
B> to do that for arguments such as --files-from='/path/to/file'. Also,
B> in this example code I had the rsync executable path pointing to the
B> ssh program, so no wonder I was getting the output of ssh!

I should have seen that because I changed it in my own copy!!!

--files-from='/path/to/file *is* one argument, in contrast to
-e "/usr/bin/ssh -i /home/bry/keys/brybackup.key" which is two
arguments.
 

Ask a Question

Want to reply to this thread or ask your own question?

You'll need to choose a username for the site, which only take a couple of moments. After that, you can post your question and our members will help you out.

Ask a Question

Members online

No members online now.

Forum statistics

Threads
473,744
Messages
2,569,484
Members
44,903
Latest member
orderPeak8CBDGummies

Latest Threads

Top