From getoptlong to optparse

[Massimiliano Mirra - bard]

I have a command line program invoked like this:

$ prog --host localhost --port 8000 login --password secret
$ prog --host localhost --port 8000 list --open

....and so on. In other words:

$ command [general options] subcommand [specific options]

Like darcs or cvs.

So far I've been doing fine with getoptlong:

while ARGV.first =~ /^-/
opt, arg = opts.get
case opt
when "--host"
config["host"] = arg
when "--port"
config["port"] = arg.to_i
end
end

When the loop sees the first thing that does not begin with `-'
(option arguments don't count, they get eaten by opts.get), it assumes
it to be the subcommand, and leaves ARGV alone from that point on.
ARGV is later emptied by the subcommand specific options parsing.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Massimiliano

 

[Robert Klemme]

I have a command line program invoked like this:

$ prog --host localhost --port 8000 login --password secret
$ prog --host localhost --port 8000 list --open

...and so on. In other words:

$ command [general options] subcommand [specific options]

Like darcs or cvs.

So far I've been doing fine with getoptlong:

while ARGV.first =~ /^-/
opt, arg = opts.get
case opt
when "--host"
config["host"] = arg
when "--port"
config["port"] = arg.to_i
end
end

I don't think the code above is proper GetoptLong code. I don't remember
exactly but the occurrence of ARGV in the loop makes me suspicious.

When the loop sees the first thing that does not begin with `-'
(option arguments don't count, they get eaten by opts.get), it assumes
it to be the subcommand, and leaves ARGV alone from that point on.
ARGV is later emptied by the subcommand specific options parsing.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Command line option parsing libs normally do that for you as long as you
don't have any special requirements. If you need to stop parsing options
after a special option #terminate might do what you need. There's a quite
complete example at the optparse rdoc page:
http://www.ruby-doc.org/stdlib/libdoc/optparse/rdoc/classes/OptionParser.html

Reconsidering your situation this is what I'd do: iterative parse all
remaining options starting with the full set at the beginning. Evaluate
the first non option, this is your next command. Eat it and go to the
first step only now remembering that options are stored for this command.
Repeat until there are no more args left or the first non option is not a
command (i.e. file name, ignorable or whatever).

Kind regards

robert

 

[Thomas Leitner]

On Tue, 02 Nov 2004 20:33:15 GMT

|
| I have a command line program invoked like this:
|
| $ prog --host localhost --port 8000 login --password secret
| $ prog --host localhost --port 8000 list --open
|
| ...and so on. In other words:
|
| $ command [general options] subcommand [specific options]
|
| Like darcs or cvs.
|
| So far I've been doing fine with getoptlong:
|
| while ARGV.first =~ /^-/
| opt, arg = opts.get
| case opt
| when "--host"
| config["host"] = arg
| when "--port"
| config["port"] = arg.to_i
| end
| end
|
| When the loop sees the first thing that does not begin with `-'
| (option arguments don't count, they get eaten by opts.get), it assumes
| it to be the subcommand, and leaves ARGV alone from that point on.
| ARGV is later emptied by the subcommand specific options parsing.
|
| I'd like to migrate to optparse, though. But how can I stop argument
| parsing at a certain point in optparse?
|
| Massimiliano

You may want to take a look at cmdparse.rubyforge.org. It is based on optparse, but provides the semantics of subcommands as you called them. Your command line

$ command [general options] subcommand [specific options]

is a prime example for the use of it. Have a look at the example at this page http://cmdparse.rubyforge.org/rdoc/classes/CommandParser.html

*hth*,
Thomas Leitner

 

[Massimiliano Mirra - bard]

while ARGV.first =~ /^-/
opt, arg = opts.get
case opt
when "--host"
config["host"] = arg
when "--port"
config["port"] = arg.to_i
end
end

I don't think the code above is proper GetoptLong code. I don't remember
exactly but the occurrence of ARGV in the loop makes me suspicious.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Command line option parsing libs normally do that for you as long as you
don't have any special requirements. If you need to stop parsing options
after a special option #terminate might do what you need.

Not exactly. I need to stop parsing when a special command is seen.
"--host" and "--port" in the above example might appear in a different
order or not appear at all, so they're not reliable.

Reconsidering your situation this is what I'd do: iterative parse all
remaining options starting with the full set at the beginning.

Bypassing optparse?

Evaluate the first non option, this is your next command.

This needs to go hand in hand with option parsing. Consider this:

$ prog --opt1 arg1 --opt2 stuff --blah

Is `stuff' a subcommand or opt2's argument? That depends on opt2's
definition (which in GetoptLong means NO_ARGUMENT or
REQUIRED_ARGUMENT). Then again, if you're thinking that things get
muddy when opt2 has an optional argument, I do agree...

Eat it and go to the first step only now remembering that options
are stored for this command. Repeat until there are no more args
left or the first non option is not a command (i.e. file name,
ignorable or whatever).

How is this different from the ARGV loop above?

Massimiliano

 

[Massimiliano Mirra - bard]

You may want to take a look at cmdparse.rubyforge.org. It is based
on optparse, but provides the semantics of subcommands as you called
them. Your command line

Seems what I need. The only drawback is that it would be an external
dependency, so I'll still try to bend optparse to my will , but
failing that I'll go with cmdparse.

Thanks!
Massimiliano

 

[Robert Klemme]

while ARGV.first =~ /^-/
opt, arg = opts.get
case opt
when "--host"
config["host"] = arg
when "--port"
config["port"] = arg.to_i
end
end

I don't think the code above is proper GetoptLong code. I don't remember
exactly but the occurrence of ARGV in the loop makes me suspicious.

I'd like to migrate to optparse, though. But how can I stop argument
parsing at a certain point in optparse?

Command line option parsing libs normally do that for you as long as you
don't have any special requirements. If you need to stop parsing options
after a special option #terminate might do what you need.

Not exactly. I need to stop parsing when a special command is seen.
"--host" and "--port" in the above example might appear in a different
order or not appear at all, so they're not reliable.

Do you mean the problem is that you need to stop when a command is seen
*and* this command can occur anywhere, i.e. also behind an option with an
optional argument?

Bypassing optparse?

No, using it.

This needs to go hand in hand with option parsing. Consider this:

$ prog --opt1 arg1 --opt2 stuff --blah

Is `stuff' a subcommand or opt2's argument? That depends on opt2's
definition (which in GetoptLong means NO_ARGUMENT or
REQUIRED_ARGUMENT).

You have the same (or finer) level of control for option arguments in
optparse, too.

Then again, if you're thinking that things get
muddy when opt2 has an optional argument, I do agree...

Yes, they certainly get muddy at this point. They get as muddy as to be
undecidable if you have a command that is also a valid optional argument
of an option. You can't decide what it is other than defining a priority
for the resolution of this ambiguity.

I'm not as much an optparse expert to come up with a quick solution. If
there are only special values allowed for a certain option, this might
help if optparse has support for it. (I believe I remembered saw
something about enumeration arguments but I'm not 100% sure.)

How is this different from the ARGV loop above?

You have two nested loops, one of them is hidden in optparse.

Kind regards

robert

 

[Massimiliano Mirra - bard]

while ARGV.first =~ /^-/
opt, arg = opts.get
case opt
when "--host"
config["host"] = arg
when "--port"
config["port"] = arg.to_i
end
end

Not exactly. I need to stop parsing when a special command is seen.
"--host" and "--port" in the above example might appear in a different
order or not appear at all, so they're not reliable.

Do you mean the problem is that you need to stop when a command is seen
*and* this command can occur anywhere, i.e. also behind an option with an
optional argument?

Yes and no. Yes: this command can occur anywhere after one or more
options. No: I have no options with optional arguments so far, so I
haven't dealt with this case which I suspect to be more complex.

You have the same (or finer) level of control for option arguments in
optparse, too.

The kind of control I am trying to get was not given by GetoptLong per
se, but by the `while ARGV.first =~ /^-/', which breaks down parsing
in chunks where the subcommand can be detected.

Yes, they certainly get muddy at this point. They get as muddy as to be
undecidable if you have a command that is also a valid optional argument
of an option. You can't decide what it is other than defining a priority
for the resolution of this ambiguity.

Yes. Or just disallowing optional arguments for generic options, the
need hasn't arisen so far and I doubt it will at least for this application.

You have two nested loops, one of them is hidden in optparse.

Sorry but I still can't make a picture of what you are suggesting.
Would you mind sketching a couple of lines of pseudocode?

Massimiliano

 

[Robert Klemme]

while ARGV.first =~ /^-/
opt, arg = opts.get
case opt
when "--host"
config["host"] = arg
when "--port"
config["port"] = arg.to_i
end
end
Not exactly. I need to stop parsing when a special command is seen.
"--host" and "--port" in the above example might appear in a different
order or not appear at all, so they're not reliable.

Do you mean the problem is that you need to stop when a command is seen
*and* this command can occur anywhere, i.e. also behind an option with an
optional argument?

Yes and no. Yes: this command can occur anywhere after one or more
options. No: I have no options with optional arguments so far, so I
haven't dealt with this case which I suspect to be more complex.

Certainly.

The kind of control I am trying to get was not given by GetoptLong per
se, but by the `while ARGV.first =~ /^-/', which breaks down parsing
in chunks where the subcommand can be detected.


Yes. Or just disallowing optional arguments for generic options, the
need hasn't arisen so far and I doubt it will at least for this

application.

Yep, that works as well.

Sorry but I still can't make a picture of what you are suggesting.
Would you mind sketching a couple of lines of pseudocode?

Hm... I'll attache a modified version of the optparse example which
yields this output

10:55:10 [ruby]: ruby optparse-example-2.rb --verbose cmd1 -i foo
<OpenStruct encoding="utf8" transfer_type=:auto verbose=true library=[]
inplace=false>
<OpenStruct extension=".foo" encoding="utf8" transfer_type=:auto
verbose=false library=[] inplace=true>
10:55:38 [ruby]:

Of course, additional handling has to be done for file name arguments but
I hope you get the picture.

Kind regards

robert

 

[nobu.nokada]

Hi,

At Wed, 3 Nov 2004 05:33:48 +0900,
Massimiliano Mirra - bard wrote in [ruby-talk:118836]:

I have a command line program invoked like this:

$ prog --host localhost --port 8000 login --password secret
$ prog --host localhost --port 8000 list --open

....and so on. In other words:

$ command [general options] subcommand [specific options]

Like darcs or cvs.

I had included an example file cmd.rb (and its incidental file
cmd-ls.rb), in sole packaged optparse, but they are not bundled
now.


#! /usr/bin/ruby
require 'optparse'
require 'pp'

@host = @port = nil

# general options
opt = OptionParser.new do |opt|
opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
opt.define(" general options:")
opt.define('--host HOST', String) {|host| @host = host}
opt.define('--port PORT', Integer) {|port| @port = port}
end

# specific option initializers
specific = {
"login" => proc do |opt|
@passwd = nil
opt.define("--password PASSWORD", String) {|passwd| @passwd = passwd}
end,
"list" => proc do |opt|
@open = false
opt.define("--open") {|open| @open = true}
end,
}
missing = proc {|opt|
opt.define([" subcommand:", specific.keys.sort].join("\n\t"))
}

begin
cmd = nil
opt.order!(ARGV) {|cmd| opt.terminate}
opt = OptionParser.new(opt) do |opt|
unless cmd
abort "#{$0}: command missing\n#{missing[opt]}"
end
subopt = specific.fetch(cmd) do
abort "#{$0}: unknown command: #{cmd}\n#{missing[opt]}"
end
opt.define(" specific options for #{cmd}:")
subopt[opt]
end
opt.parse!(ARGV)
rescue OptionParser:arseError => e
abort "#{e}\n#{opt}"
end
pp self

 

[Ara.T.Howard]

On Wed, 10 Nov 2004 (e-mail address removed) wrote:

thanks nobu! i've been dying to know how to do this!

-a

#! /usr/bin/ruby
require 'optparse'
require 'pp'

@host = @port = nil

# general options
opt = OptionParser.new do |opt|
opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
opt.define(" general options:")
opt.define('--host HOST', String) {|host| @host = host}
opt.define('--port PORT', Integer) {|port| @port = port}
end

# specific option initializers
specific = {
"login" => proc do |opt|
@passwd = nil
opt.define("--password PASSWORD", String) {|passwd| @passwd = passwd}
end,
"list" => proc do |opt|
@open = false
opt.define("--open") {|open| @open = true}
end,
}
missing = proc {|opt|
opt.define([" subcommand:", specific.keys.sort].join("\n\t"))
}

begin
cmd = nil
opt.order!(ARGV) {|cmd| opt.terminate}
opt = OptionParser.new(opt) do |opt|
unless cmd
abort "#{$0}: command missing\n#{missing[opt]}"
end
subopt = specific.fetch(cmd) do
abort "#{$0}: unknown command: #{cmd}\n#{missing[opt]}"
end
opt.define(" specific options for #{cmd}:")
subopt[opt]
end
opt.parse!(ARGV)
rescue OptionParser:arseError => e
abort "#{e}\n#{opt}"
end
pp self

-a
--
===============================================================================
| EMAIL :: Ara [dot] T [dot] Howard [at] noaa [dot] gov
| PHONE :: 303.497.6469
| When you do something, you should burn yourself completely, like a good
| bonfire, leaving no trace of yourself. --Shunryu Suzuki
===============================================================================

 

[Robert Klemme]

Hi,

At Wed, 3 Nov 2004 05:33:48 +0900,
Massimiliano Mirra - bard wrote in [ruby-talk:118836]:

I have a command line program invoked like this:

$ prog --host localhost --port 8000 login --password secret
$ prog --host localhost --port 8000 list --open

....and so on. In other words:

$ command [general options] subcommand [specific options]

Like darcs or cvs.

I had included an example file cmd.rb (and its incidental file
cmd-ls.rb), in sole packaged optparse, but they are not bundled
now.


#! /usr/bin/ruby
require 'optparse'
require 'pp'

@host = @port = nil

# general options
opt = OptionParser.new do |opt|
opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
opt.define(" general options:")
opt.define('--host HOST', String) {|host| @host = host}
opt.define('--port PORT', Integer) {|port| @port = port}
end

# specific option initializers
specific = {
"login" => proc do |opt|
@passwd = nil
opt.define("--password PASSWORD", String) {|passwd| @passwd = passwd}
end,
"list" => proc do |opt|
@open = false
opt.define("--open") {|open| @open = true}
end,
}
missing = proc {|opt|
opt.define([" subcommand:", specific.keys.sort].join("\n\t"))
}

begin
cmd = nil
opt.order!(ARGV) {|cmd| opt.terminate}
opt = OptionParser.new(opt) do |opt|
unless cmd
abort "#{$0}: command missing\n#{missing[opt]}"
end
subopt = specific.fetch(cmd) do
abort "#{$0}: unknown command: #{cmd}\n#{missing[opt]}"
end
opt.define(" specific options for #{cmd}:")
subopt[opt]
end
opt.parse!(ARGV)
rescue OptionParser:arseError => e
abort "#{e}\n#{opt}"
end
pp self

Somehow I though Massimiliano will have multiple sub commands - apparently
that was wrong. Out of curiosity: What does the code above do in the
light of multiple sub commands? As far as I understand it, once a
subcommand is encountered options for this subcommand are defined and
would remain valid once another subcommand is seen. Is that right? Or is
this scheme incapable of dealing with multiple sub commands.

Kind regards

robert

 

[nobu.nokada]

Hi,

At Wed, 10 Nov 2004 23:23:31 +0900,
Robert Klemme wrote in [ruby-talk:119714]:

Somehow I though Massimiliano will have multiple sub commands - apparently
that was wrong. Out of curiosity: What does the code above do in the
light of multiple sub commands? As far as I understand it, once a
subcommand is encountered options for this subcommand are defined and
would remain valid once another subcommand is seen. Is that right? Or is
this scheme incapable of dealing with multiple sub commands.

Do you mean this?

prog.rb --host example.com --port 1024 login --pass secret list --open

Then loop subcommand parser until ARGV becomes empty using
#order! instead of #parse!.


--- prog.rb 2004-11-12 10:33:03.000000000 +0900
+++ mprog.rb 2004-11-12 10:47:16.000000000 +0900
@@ -6,5 +6,5 @@ require 'pp'

# general options
-opt = OptionParser.new do |opt|
+general = OptionParser.new do |opt|
opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
opt.define(" general options:")
@@ -30,6 +30,7 @@ missing = proc {|opt|
begin
cmd = nil
- opt.order!(ARGV) {|cmd| opt.terminate}
- opt = OptionParser.new(opt) do |opt|
+ (opt = general).order!(ARGV) {|cmd| opt.terminate}
+ begin
+ OptionParser.new(general) do |opt|
unless cmd
abort "#{$0}: command missing\n#{missing[opt]}"
@@ -40,6 +41,6 @@ begin
opt.define(" specific options for #{cmd}:")
subopt[opt]
- end
- opt.parse!(ARGV)
+ end.order!(ARGV) {|cmd| opt.terminate}
+ end until ARGV.empty?
rescue OptionParser:arseError => e
abort "#{e}\n#{opt}"

 

[Robert Klemme]

Hi,

At Wed, 10 Nov 2004 23:23:31 +0900,
Robert Klemme wrote in [ruby-talk:119714]:

Somehow I though Massimiliano will have multiple sub commands - apparently
that was wrong. Out of curiosity: What does the code above do in the
light of multiple sub commands? As far as I understand it, once a
subcommand is encountered options for this subcommand are defined and
would remain valid once another subcommand is seen. Is that right? Or is
this scheme incapable of dealing with multiple sub commands.

Do you mean this?

prog.rb --host example.com --port 1024 login --pass secret list --open

Yes, that's exactly the scenario I was thinking of ("login" and "list" as
sub commands).

Then loop subcommand parser until ARGV becomes empty using
#order! instead of #parse!.

I got lost somewhere in the diff below. What eaxtly is the difference
between "order!" and "parse!"?

Kind regards

robert

--- prog.rb 2004-11-12 10:33:03.000000000 +0900
+++ mprog.rb 2004-11-12 10:47:16.000000000 +0900
@@ -6,5 +6,5 @@ require 'pp'

# general options
-opt = OptionParser.new do |opt|
+general = OptionParser.new do |opt|
opt.banner = "Usage: #{$0} [general options] subcommand [specific options]"
opt.define(" general options:")
@@ -30,6 +30,7 @@ missing = proc {|opt|
begin
cmd = nil
- opt.order!(ARGV) {|cmd| opt.terminate}
- opt = OptionParser.new(opt) do |opt|
+ (opt = general).order!(ARGV) {|cmd| opt.terminate}
+ begin
+ OptionParser.new(general) do |opt|
unless cmd
abort "#{$0}: command missing\n#{missing[opt]}"
@@ -40,6 +41,6 @@ begin
opt.define(" specific options for #{cmd}:")
subopt[opt]
- end
- opt.parse!(ARGV)
+ end.order!(ARGV) {|cmd| opt.terminate}
+ end until ARGV.empty?
rescue OptionParser:arseError => e
abort "#{e}\n#{opt}"

 

[nobu.nokada]

Hi,

At Fri, 12 Nov 2004 20:28:27 +0900,
Robert Klemme wrote in [ruby-talk:120033]:

I got lost somewhere in the diff below. What eaxtly is the difference
between "order!" and "parse!"?

OptionParser has two kinds of behaviors; "in order" and
"permutation" mode. The former (#order! method) stops as soon
as a non-option argument is encountered and is same as
getopt_long(3) when the environment variable POSIXLY_CORRECT is
set, and the latter (#permute! method) processes all option
arguments at first and leaves non-option arguments. #parse!
method switches the two behavior by the environment variable.

 

[Robert Klemme]

Hi,

At Fri, 12 Nov 2004 20:28:27 +0900,
Robert Klemme wrote in [ruby-talk:120033]:

I got lost somewhere in the diff below. What eaxtly is the
difference
between "order!" and "parse!"?

OptionParser has two kinds of behaviors; "in order" and
"permutation" mode. The former (#order! method) stops as soon
as a non-option argument is encountered and is same as
getopt_long(3) when the environment variable POSIXLY_CORRECT is
set, and the latter (#permute! method) processes all option
arguments at first and leaves non-option arguments. #parse!
method switches the two behavior by the environment variable.

Thanks, Nobu! Now I see much clearer.

robert