ideas for a "parameter sweep" program?

Discussion in 'Ruby' started by Diego Virasoro, Nov 23, 2009.

  1. Hello,
    I often need to run my computational experiments with a combination of
    parameter values, and I was thinking of automatising it with Ruby. Any
    idea on what would be the best way to go about it?

    What would be the advantage of trying something like a DSL? I've never
    done it before so it looks a little bit daunting and it may be an
    overkill, but then this is a personal project so I don't have
    deadlines and I'd like to learn a bit in the process.

    Or maybe should I cook up something with Rake?

    Ideas?

    Diego
    Diego Virasoro, Nov 23, 2009
    #1
    1. Advertising

  2. Diego Virasoro

    Intransition Guest

    On Nov 23, 3:15=A0pm, Diego Virasoro <> wrote:
    > Hello,
    > I often need to run my computational experiments with a combination of
    > parameter values, and I was thinking of automatising it with Ruby. Any
    > idea on what would be the best way to go about it?
    >
    > What would be the advantage of trying something like a DSL? I've never
    > done it before so it looks a little bit daunting and it may be an
    > overkill, but then this is a personal project so I don't have
    > deadlines and I'd like to learn a bit in the process.
    >
    > Or maybe should I cook up something with Rake?


    Depends on what you want. If you just want to create an executable
    that can accept various options look at the built in OptionParser.

    http://ruby-doc.org/stdlib/libdoc/optparse/rdoc/classes/OptionParser.html
    Intransition, Nov 23, 2009
    #2
    1. Advertising

  3. Diego Virasoro wrote:
    > I often need to run my computational experiments with a combination of
    > parameter values, and I was thinking of automatising it with Ruby. Any
    > idea on what would be the best way to go about it?


    Is the computational experiment written in Ruby? Then wrap it in a
    class, and pass the parameters to your initialize method. This allows
    you to re-use the code in different ways. For example, you can create
    one runner program which takes the parameters from the command line;
    another runner program which repeatedly runs the experiment but sweeping
    across a range of values; and so on.

    Even if the experiment is not written in ruby, you can still wrap it in
    a ruby class and call it as an external program using %x{}, system() or
    IO.popen().

    > Or maybe should I cook up something with Rake?


    rake supports passing (string) parameters as environment variables:
    rake mytest FOO=xyz BAR=456

    so a 'rake runner' might look like this:

    require 'experiment'
    task :mytest do
    result = Experiment.new:)foo => ENV['FOO'], :bar =>
    ENV['BAR'].to_f).run
    puts result
    end
    --
    Posted via http://www.ruby-forum.com/.
    Brian Candler, Nov 24, 2009
    #3
  4. 2009/11/23 Diego Virasoro <>:
    > I often need to run my computational experiments with a combination of
    > parameter values, and I was thinking of automatising it with Ruby. Any
    > idea on what would be the best way to go about it?


    Not sure what you actually mean. Are you looking for something like

    p1 = [1,2,10,3]
    p2 = [1312,42345,235435,65]
    p3 = [123,23,476,43]

    p1.each do |param1|
    p2.each do |param2|
    p3.each do |param3|
    system "experiment", param1, param2, param3
    end
    end

    or rather

    params = [
    [1,11,2],
    [23,4,21],
    [5,1,4],
    ]

    params.each do |pa|
    system "experiment", *pa
    end

    ?

    Kind regards

    robert

    --
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
    Robert Klemme, Nov 24, 2009
    #4
  5. > Is the computational experiment written in Ruby? Then wrap it in a
    > class, and pass the parameters to your initialize method. This allows
    > you to re-use the code in different ways.


    I would like a generic solution that works with any language for any
    program (including Fortran that doesn't allow to pass arguments form
    the command line). Basically what it should do is look for some
    regexp, swap some values, and the rerun the experiment. Then rinse and
    repeat. All the output should be catalogued so that afterwards I can
    run some data analysis.

    Diego
    Diego Virasoro, Nov 24, 2009
    #5
  6. Diego Virasoro wrote:
    > I would like a generic solution that works with any language for any
    > program (including Fortran that doesn't allow to pass arguments form
    > the command line).


    So how do you pass parameters to your Fortran program - in a config file
    perhaps? If so, then just generate that file then run the program. It's
    up to you to code what you want.

    Does it read a flat file? XML? YAML? Something else?

    class FlatFileRunner
    def initialize(exe, conffile)
    @exe, @conffile = exe, conffile
    end
    def run(params = {})
    File.open(@conffile, "w") do |f|
    params.each do |k,v|
    f.puts "#{k}=#{v}"
    end
    end
    %x{"#{@exe}"}
    end
    end

    prog = FlatFileRunner.new("./flurble", "./flurble.conf")
    res = prog.run("foo"=>123)
    puts "Result of experiment: #{res}"
    --
    Posted via http://www.ruby-forum.com/.
    Brian Candler, Nov 24, 2009
    #6
  7. On 24 Nov, 12:36, Robert Klemme <> wrote:
    > 2009/11/23 Diego Virasoro <>:
    >
    > > I often need to run my computational experiments with a combination of
    > > parameter values, and I was thinking of automatising it with Ruby. Any
    > > idea on what would be the best way to go about it?

    >
    > Not sure what you actually mean.  Are you looking for something like
    >
    > p1 = [1,2,10,3]
    > p2 = [1312,42345,235435,65]
    > p3 = [123,23,476,43]
    >
    > p1.each do |param1|
    >   p2.each do |param2|
    >     p3.each do |param3|
    >       system "experiment", param1, param2, param3
    >   end
    > end
    >


    Yep, something like this in its simplest form.
    But I'd like to make it a little more generic and powerful, for
    example allowing for some action before or after the main call, for
    the user to express the name of the experiment, for actions before or
    after the whole script, etc...

    The question wasn't how to do it, but what would be the best way to do
    it, if you see what I mean. I've got a few ideas and I am not sure
    what will work because some (like a DSL) I've never done before and
    others (like a Yaml file) may not be enough.

    Diego
    Diego Virasoro, Nov 24, 2009
    #7
  8. On 24 Nov, 13:11, Brian Candler <> wrote:
    > Diego Virasoro wrote:
    > > I would like a generic solution that works with any language for any
    > > program (including Fortran that doesn't allow to pass arguments form
    > > the command line).

    >
    > So how do you pass parameters to your Fortran program - in a config file
    > perhaps? If so, then just generate that file then run the program. It's
    > up to you to code what you want.


    Yes, it would generally read a file. Don't get me wrong: I know how to
    do it. Currently I've got a bash script that does it for one
    particular executable by using sed and make. I just wanted to
    generalise it and given I hate bash Ruby seemed like the best
    alternative.

    However I would also like to make it a lot more generic and flexible,
    as maybe other people in my group may be interested in using it too.

    Which lead to the question: what would be the best way? A rake file, a
    yaml input file, some internal DSL?

    Diego
    Diego Virasoro, Nov 24, 2009
    #8
  9. Diego Virasoro wrote:
    > Which lead to the question: what would be the best way? A rake file, a
    > yaml input file, some internal DSL?


    Do The Simplest Thing Which Can Possibly Work, which I think in this
    case means: just start writing code.

    If you find yourself writing the same thing over and over again, then
    write a shortcut for that particular pattern. That shortcut or set of
    shortcuts will become your "DSL".

    And don't be afraid to refactor later.
    --
    Posted via http://www.ruby-forum.com/.
    Brian Candler, Nov 24, 2009
    #9
  10. 2009/11/24 Diego Virasoro <>:
    > On 24 Nov, 12:36, Robert Klemme <> wrote:
    >> 2009/11/23 Diego Virasoro <>:
    >>
    >> > I often need to run my computational experiments with a combination of
    >> > parameter values, and I was thinking of automatising it with Ruby. Any
    >> > idea on what would be the best way to go about it?

    >>
    >> Not sure what you actually mean. =A0Are you looking for something like
    >>
    >> p1 =3D [1,2,10,3]
    >> p2 =3D [1312,42345,235435,65]
    >> p3 =3D [123,23,476,43]
    >>
    >> p1.each do |param1|
    >> =A0 p2.each do |param2|
    >> =A0 =A0 p3.each do |param3|
    >> =A0 =A0 =A0 system "experiment", param1, param2, param3
    >> =A0 end
    >> end
    >>

    >
    > Yep, something like this in its simplest form.
    > But I'd like to make it a little more generic and powerful, for
    > example allowing for some action before or after the main call, for
    > the user to express the name of the experiment, for actions before or
    > after the whole script, etc...


    Then you should probably start with filling on those dots. If the
    required feature set is unclear the proposed solution might be
    inappropriate.

    > The question wasn't how to do it, but what would be the best way to do
    > it, if you see what I mean. I've got a few ideas and I am not sure
    > what will work because some (like a DSL) I've never done before and
    > others (like a Yaml file) may not be enough.


    As I said: first it needs to be clear what needs to be done. What
    kind of inputs do you need or want (a single configuration file or a
    number of files, should it be controlled by some command line
    arguments etc.) and what actions do you need. Are the actions before
    and after the experiment uniform or do they depend on the command line
    args of that particular execution. Things like that can make a
    difference for the viability of any solution.

    Kind regards

    robert


    --=20
    remember.guy do |as, often| as.you_can - without end
    http://blog.rubybestpractices.com/
    Robert Klemme, Nov 24, 2009
    #10
  11. > As I said: first it needs to be clear what needs to be done.  What
    > kind of inputs do you need or want (a single configuration file or a
    > number of files, should it be controlled by some command line
    > arguments etc.) and what actions do you need.  Are the actions before
    > and after the experiment uniform or do they depend on the command line
    > args of that particular execution.  Things like that can make a
    > difference for the viability of any solution.
    >

    You are completely right.

    I hoped it was something people had already faced, or maybe even
    solved, or maybe had ideas to give (I doubt I'll get all the
    requirements right now). I think the real solution is to follow
    Brian's advice and just start coding... it will mean a lot of
    refactoring, but that's life, I guess. :)

    Diego
    Diego Virasoro, Nov 24, 2009
    #11
    1. Advertising

Want to reply to this thread or ask your own question?

It takes just 2 minutes to sign up (and it's free!). Just click the sign up button to choose a username and then you can ask your own questions on the forum.
Similar Threads
  1. Jason  Taylor

    Ideas for first program?

    Jason Taylor, Aug 31, 2007, in forum: C++
    Replies:
    3
    Views:
    341
    Jude Naveen Pinto
    Aug 31, 2007
  2. George Ogata

    Ideas for program

    George Ogata, Mar 10, 2006, in forum: Ruby
    Replies:
    3
    Views:
    106
    Gregory Brown
    Mar 10, 2006
  3. Jasper
    Replies:
    17
    Views:
    175
    David H. Adler
    Jul 8, 2004
  4. eli m

    Small program ideas

    eli m, Feb 16, 2013, in forum: Python
    Replies:
    20
    Views:
    349
    Neil Cerutti
    Mar 29, 2013
  5. eli m

    Program ideas

    eli m, Mar 18, 2013, in forum: C++
    Replies:
    1
    Views:
    254
    Christopher Pisz
    Mar 19, 2013
Loading...

Share This Page