getopt or optparse options/arguments wrapping?

[Rocky Zhou]

I wonder is there any way to make the wrapper program can wrap options
&& arguments for the the subprocess/command the wrapper will
execute? by getopt or optparse module?

This is something like the shell script like this:
optwrap=""
while [ $# -gt 0 ]; do
case $1 in
-a) do-something; shift
;;
-b) do-something; shift
;;
*) optwrap="$optwrap $1 $2"
# collect the option and argument
shift
;;
esac
shift
done

$command $optwrap
# execute the command with $optwrap as subprocess

I want that all the options and arguments which are feed to this
scripts command line will be used by executing the sub command.

I need this because I now have finished a fs_backup script written in
python, it will execute tar && find to complete full and
differentiating/incremental backup for the file system level files,
and their restoring. This script will do different things by the
command line arguments, and execute find/tar in different ways, but
for being more flexible, I want some options of tar/find can also been
specified from the command line directly, and the script just transmit
those to tar/find simply.

Is there anyone can give me some advices?

Thank you.

 

[Steven Bethard]

I wonder is there any way to make the wrapper program can wrap options
&& arguments for the the subprocess/command the wrapper will
execute? by getopt or optparse module? [snip]
I need this because I now have finished a fs_backup script written in
python, it will execute tar && find to complete full and
differentiating/incremental backup for the file system level files,
and their restoring. This script will do different things by the
command line arguments, and execute find/tar in different ways, but
for being more flexible, I want some options of tar/find can also been
specified from the command line directly, and the script just transmit
those to tar/find simply.

I'm not clear on exactly what it is you want. Are you trying to do
something like::

fs_backup --foo --bar x y z --tar-foo --tar-bar tx ty tz

where ``--foo --bar x y z`` gets handled by your command and ``--tar-foo
--tar-bar tx ty tz`` gets passed on through to the tar command?

STeVe

 

[Rocky Zhou]

Well, I think I must have more explanation on my script. The script's
usage is like this:
~# fs_backup
Lack of backup identity name
program usage: fs_backup [OPTIONS] $identity
OPTIONS:
-a|--append [t/x[L]:$path, append 1 dir or file to $identity
t/x: include/exclude list, as -T/-X of 'tar'
L:file, append a batch of dirs/files to $identity by LIST
-d|--delete $path, delete 1 dir or file from $identity
-t|--type $type, specify the backup type: full/diff/incr
-D|--delete-outdated [f/d], delete outdated files/empty-dirs when
restoring.
-o 'option=value'
-w|--wrap 'wrap of tar options'
-h|--help, print this help message


For example, I want to backup /var/www/html, with excluding /var/www/
html/syssite/home/cache, and name this backup as pages, I will do
this:
sh# fs_backup -a t:/var/www/html pages
sh# fs_backup -a x:/var/www/html/syssite/home/cache pages
# These t/x have the same meaning of tar's '-T/-X'
sh# fs_backup -t full pages
sh# fs_backup -t incr pages

These will update /var/fs_backup/.table, (/var/fs_backup is 'datadir',
you can configure it to anywhere you like by editing the configuration
file, /etc/new/fs_backup.config):
pages, full, 20070204-231749, pages/pages.full.20070204-231749
pages, incr, 20070205-101011, pages/pages.incr.20070205-101011
.......

May be I want add something to 'pages':
sh# fs_backup -a t:/usr/local/apache2/htdocs pages
sh# fs_backup -t full pages

Once I want recovery from the backup, I will do this:
sh# fs_rstore pages
This fs_rstore is just a symlink to fs_backup. All the files from the
last full backup will be restored.

The actually thing the script done is like this:
2007-03-18 20:32:20 fs_backup DEBUG find /mnt/file/data/ -cnewer /var/
fs_backup/work/.ts ! -type d -print >>/tmp/fs_backup.work.incr.
1174221140.0.list
2007-03-18 20:32:20 fs_backup DEBUG find /mnt/file/work/ -cnewer /var/
fs_backup/work/.ts ! -type d -print >>/tmp/fs_backup.work.incr.
1174221140.0.list
2007-03-18 20:32:20 fs_backup DEBUG tar -czp -P -f /var/fs_backup/work/
work.incr.20070318-203220.tgz -T /tmp/fs_backup.work.incr.
1174221140.0.list .identity .dirs .files

..dirs .files is just a snapshot of the current directories, which can
be used to delete-outdated files when restoring. Here I used absolute
path by using tar's -P parameter. When fs_rstore, it will do this:
command = "tar -xpz -P -f %s.tgz -T %s" % (archive, self.t_files)

maybe I want add some parameters of tar, for example, --strip-path=X,
so I hope the fs_rstore can do this:
sh# fs_rstore pages --strip-path=X
rather than:
sh# fs_rstore pages -w '--strip-path=X'

What should I do?

Thank you.

 

[Steven Bethard]

.dirs .files is just a snapshot of the current directories, which can
be used to delete-outdated files when restoring. Here I used absolute
path by using tar's -P parameter. When fs_rstore, it will do this:
command = "tar -xpz -P -f %s.tgz -T %s" % (archive, self.t_files)

maybe I want add some parameters of tar, for example, --strip-path=X,
so I hope the fs_rstore can do this:
sh# fs_rstore pages --strip-path=X
rather than:
sh# fs_rstore pages -w '--strip-path=X'

What should I do?

One possibility using argparse (http://argparse.python-hosting.com/)
would be to require the '--' pseudo-argument before all tar options.
Then you could do something like::

fs_rstore pages -- --strip-path=X --same-owner

That '--' pseudo-argument makes everything else after it into a
positional argument (even if they start with '-' or '--'), so you can
then just collect those positional arguments and add them to your
"command". Here's what that code might look like::

>>> parser = argparse.ArgumentParser(prog='fs_backup')
>>> parser.add_argument('-a', '--append', metavar='[t/x[L]:]PATH')
>>> parser.add_argument('-d', '--delete', metavar='PATH')
>>> parser.add_argument('-t', '--type', ... choices=['full', 'diff', 'incr'])
>>> parser.add_argument('identity')
>>> parser.add_argument('tar_options', nargs='*')
>>> parser.parse_args('-a foo pages -- --strip-path=X')
>>> args = parser.parse_args( ... '-a foo pages -- --strip-path=X --same-owner'.split())
>>> args.append 'foo'
>>> args.identity 'pages'
>>> args.tar_options ['--strip-path=X', '--same-owner']
>>> "tar %s -f %s.tgz" % (' '.join(args.tar_options), args.identity)

'tar --strip-path=X --same-owner -f pages.tgz'

Note that all the tar options got collected in 'args.tar_options'.

You could get rid of the need for '--' by adding each tar options to
your parser with an add_argument() call, but I suspect that's too
tedious. I've added a feature request to argparse to make this kind of
thing easier::
http://argparse.python-hosting.com/ticket/28
Not sure when I'll have a chance to look into this though.

STeVe

P.S. You should be able to get similar behavior out of optparse, though
you'll have to parse the 'args' list yourself.