Command Pipeline (pipes & filters) in Ruby

[eastcoastcoder]

Anyone know of any generic implementation of a pipeline in ruby? That
is, one which takes a bunch of GoF Command objects, and strings them
together in the classical pipeline configuration.

Any advice on that?

I have a process with a lot of sequential processing, and think that a
pipeline of Command objects (not just Proc's) might be the way to do
it.

The downsides of pipelines:
1) You can't do selections (if) or sequences (loops). If you can
handle this, though, this becomes a virtue, as it keeps everything
crystal clear and simple. Witness the ever useful Unix command line
pipelines.

2) They don't encapsulate very well, as each stage totally overwrites
the previous ones. I'm not sure if I should just accept this, or try
to use some modification (append only pipeline, etc.).

Any experience or ideas on this appreciated.

 

[James Britt]

Anyone know of any generic implementation of a pipeline in ruby? That
is, one which takes a bunch of GoF Command objects, and strings them
together in the classical pipeline configuration.
=20
Any advice on that?

Does Rake approach this?

Define each command as a task. Let Rake handle the chaining and=20
dependency logic.




--=20
James Britt

=93Design depends largely on constraints.=94
=97 Charles Eames

 

[Robert Klemme]

Anyone know of any generic implementation of a pipeline in ruby? That
is, one which takes a bunch of GoF Command objects, and strings them
together in the classical pipeline configuration.

Any advice on that?

I have a process with a lot of sequential processing, and think that a
pipeline of Command objects (not just Proc's) might be the way to do
it.

The downsides of pipelines:
1) You can't do selections (if) or sequences (loops). If you can
handle this, though, this becomes a virtue, as it keeps everything
crystal clear and simple. Witness the ever useful Unix command line
pipelines.

2) They don't encapsulate very well, as each stage totally overwrites
the previous ones. I'm not sure if I should just accept this, or try
to use some modification (append only pipeline, etc.).

Any experience or ideas on this appreciated.

There are several ways you can do pipelining. Do you want concurrency
with that? Does every stage have a single output the next stage wants to
operate on? etc.

Probably the simplest thing you can do is to use #inject:

commands = [
lambda {|x| x * 2},
lambda {|x| 0 - x},
lambda {|x| x + 10},
]

commands.inject(0) {|x,cmd| cmd[x]} => 10
commands.inject(1) {|x,cmd| cmd[x]}

=> 8

Kind regards

robert

 

[Adam Sanderson]

That's neat, this might be handy too:

class Proc
def |(other)
proc{|*a| other.call(self.call(*a)) }
end
end

I think I got the idea from something on Why's site, though I can't
find it now. Anyways if you could now do:

f = lambda {|x| x * 2} | lambda {|x| 0 - x} | lambda {|x| x + 10}

or more verbosely:

a = lambda {|x| x * 2}
b = lambda {|x| 0 - x}
c = lambda {|x| x + 10}

f = (a|b|c)

Think of it as unix pipes
.adam

 

[Robert Klemme]

That's neat, this might be handy too:

class Proc
def |(other)
proc{|*a| other.call(self.call(*a)) }
end
end

I think I got the idea from something on Why's site, though I can't
find it now. Anyways if you could now do:

f = lambda {|x| x * 2} | lambda {|x| 0 - x} | lambda {|x| x + 10}

or more verbosely:

a = lambda {|x| x * 2}
b = lambda {|x| 0 - x}
c = lambda {|x| x + 10}

f = (a|b|c)

Think of it as unix pipes
.adam

Depending on the number of processors you might even run into stack size
limits which won't happen with an #inject based solution (see my other
posting). (Ok, I know that I'm being obsessive about Enumerable#inject -
it was love at first sight. ) )

Although this looks nice, when associating Unix pipes you might be tempted
to expect those to execute concurrently which they don't.

We can combine both solutions:

class ProcChain
def initialize(*procs) @chain = procs end
def |(proc)
@chain << proc
self
end
def call(a)
@chain.inject(a) {|x,pr| pr[x]}
end
alias [] call
end

class Proc
def |(proc)
ProcChain.new(self, proc)
end
end

irb(main):039:0> f = lambda {|x| x * 2} | lambda {|x| 0 - x} | lambda {|x|
x + 10}
=> #<ProcChain:0x4a6fed8 @chain=[#<Proc:0x04a70238@(irb):39>,
#<Proc:0x04a70148@(irb):39>, #<Proc:0x04a6ffc8@(irb):39>]>
irb(main):040:0> f[0]
=> 10
irb(main):041:0> f.call 1
=> 8

)

Kind regards

robert