getopt and options with multiple arguments

[pinkfloydhomer]

I want to be able to do something like:

myscript.py * -o outputfile

and then have the shell expand the * as usual, perhaps to hundreds of
filenames. But as far as I can see, getopt can only get one argument
with each option. In the above case, there isn't even an option string
before the *, but even if there was, I don't know how to get getopt to
give me all the expanded filenames in an option.

Help!

/David

 

[Simon Brunning]

I want to be able to do something like:

myscript.py * -o outputfile

and then have the shell expand the * as usual, perhaps to hundreds of
filenames. But as far as I can see, getopt can only get one argument
with each option. In the above case, there isn't even an option string
before the *, but even if there was, I don't know how to get getopt to
give me all the expanded filenames in an option.

Help!

You could use the glob module to expand the asterisk yourself.

 

[PoD]

I want to be able to do something like:

myscript.py * -o outputfile

and then have the shell expand the * as usual, perhaps to hundreds of
filenames. But as far as I can see, getopt can only get one argument
with each option. In the above case, there isn't even an option string
before the *, but even if there was, I don't know how to get getopt to
give me all the expanded filenames in an option.

Help!

/David

The convention is to put options first and then any other stuff such as
filenames:-
myscript.py -o outputfile *

In this case getopt will return the options as one list and the rest as
another list.

 

[Steven D'Aprano]

I want to be able to do something like:

myscript.py * -o outputfile

and then have the shell expand the * as usual, perhaps to hundreds of
filenames.

If you are calling this from most Linux and Unix shells, the shell will
already have expanded the * and myscript.py will never see the asterisk.

But as far as I can see, getopt can only get one argument
with each option.

The getopt module has two main functions you probably want to call.

getopt.getopt expects options like -o arg followed by any remaining args.

getopt.gnu_getopt allows mixed args and options.

But frankly, the easiest way of dealing with your problem is just to
change your user-interface. What is wrong with using

myscript.py -o outputfile *

instead?

In the above case, there isn't even an option string
before the *, but even if there was, I don't know how to get getopt to
give me all the expanded filenames in an option.

If you use a shell that doesn't expand the filenames, you can use the glob
module to expand it yourself.

 

[Tom Anderson]

I want to be able to do something like:

myscript.py * -o outputfile

and then have the shell expand the * as usual, perhaps to hundreds of
filenames. But as far as I can see, getopt can only get one argument
with each option. In the above case, there isn't even an option string
before the *, but even if there was, I don't know how to get getopt to
give me all the expanded filenames in an option.

I'm really surprised that getopt doesn't handle this properly by default
(so getopt.getopt mimics unices with crappy getopts - since when was that
a feature?), but as Steven pointed out, getopt.gnu_getopt will float your
boat.

I have an irrational superstitious fear of getopt, so this is what i use
(it returns a list of arguments, followed by a dict mapping flags to
values; it only handles long options, but uses a single dash for them, as
is, for some reason, the tradition in java, where i grew up):

def arguments(argv, expand=True):
argv = list(argv)
args = []
flags = {}
while (len(argv) > 0):
arg = argv.pop(0)
if (arg == "--"):
args.extend(argv)
break
elif (expand and arg.startswith("@")):
if (len(arg) > 1):
arg = arg[1:]
else:
arg = argv.pop(0)
argv[0:0] = list(stripped(file(arg)))
elif (arg.startswith("-") and (len(arg) > 1)):
arg = arg[1:]
if (":" in arg):
key, value = arg.split(":")
else:
key = arg
value = ""
flags[key] = value
else:
args.append(arg)
return args, flags

def stripped(f):
"""Return an iterator over the strings in the iterable f in which
strings are stripped of #-delimited comments and leading and
trailing whitespace, and blank strings are skipped.

"""
for line in f:
if ("#" in line): line = line[:line.index("#")]
line = line.strip()
if (line == ""): continue
yield line
raise StopIteration

As a bonus, you can say @foo or @ foo to mean "insert the lines contained
in file foo in the command line here", which is handy if, say, you have a
file containing a list of files to be processed, and you want to invoke a
script to process them, or if you want to put some standard flags in a
file and pull them in on the command line. Yes, you could use xargs for
this, but this is a bit easier. If you don't want this, delete the elif
block mentioning the @, and the stripped function. A slightly neater
implementation not involving list.pop also then becomes possible.

tom